Your question goes to the heart of debates about personal identity, and even goes back to the early modern starting point for those debates, the chapter on personal identity in John Locke's Essay Concerning Human Understanding. Much discussion about personal identity has turned on the question of whether personal identity is to be located in psychological continuity or in bodily continuity of some sort; if one is inclined to think that the identity of a person is to be identified with the identity of her brain, then personal identity does indeed seem to consist in bodily continuity.
Your question goes even further, for you wonder whether the brain is to be identified as the locus of personhood, so that a person just is her brain. One thing that this view has going for it is that it seems that all human persons have had--it's not clear whether we can know, without doing brain scans, that all human persons have brains--brains. So having a brain would seem to be necessary for being a human person. But is having a brain sufficient for being a person? It's not clear to me that it is. For one thing, if one takes a person to be an agent, capable of having long term goals and forming life plans, then many human beings with brains do not count as persons. Moreover, even if one thinks that personhood is tied to being a human being (and human beings certainly have brains), it's not clear that personhood consists essentially in the identity of one's brain: the rest of one's body, and even one's psychology, could also be bound up with being a human being. As for whether a person would be the same person if her brain were transplanted into a different body, I don't really know what to say. Assuming that such a transplant could be done, if the rest of the body helps to determine the nature of a person, as much as her brain, then merely transplanting a brain into a different body would not preserve the identity of a person. (Maybe if the body into which the brain was transplanted was relevantly similar enough to the first body, then identity might be preserved.) Even if identity just consists in one's psychology, only if one's psychology could be preserved when one's brain was transplanted into another body, would identity be preserved. But I hesitate to draw conclusions about personal identity on the basis of a consideration of this sort of science fiction scenario.
I myself am inclined to think that the brain is importantly related to personal identity, although this is only an intuition, and I'm not at all sure just what role the brain plays in determining personal identity; I'm not, however, inclined to think that a person just is her brain. But cashing out this intuition is a matter for further, deeper, reflection. (Much of the philosophical literature on personal identity is devoted to exploring just such issues: if you're interested in exploring further, there are numerous good, relatively introductory starting points. I recommend Amelie Rorty's anthology, The Identities of Persons, and John Perry's Dialogue on Personal Identity and Immortality.)