Not necessarily. Within the comics - specifically the 616 continuity - the mansion located at 1407 Graymalkin Lane, Salem Center, NY has been destroyed and rebuilt numerous times.
On many occasions, the mansion that houses the X-Men's famous School for Gifted Youngsters has been entirely demolished, often at the hands of some villain or destructive event. In each case, the mansion is rebuilt. In most cases, the opportunity is taken to expand the mansion itself, thus making room for additional students and facilities.
It's important to note that both the interior and exteriors of the mansion have varied quite a bit over the years. With some rebuilds, great care was taken to replicate the previous appearance, while with other rebuilds the layout and/or style of the mansion was changed drastically. On at least three occasions, the entire mansion was rebuilt but you'd only know it once entering the lower levels - the mansion proper was identical to the previous building.
Note: At one point in continuity, the school was relocated to a different mansion in Snow Valley, Massachusetts. The information in this answer pertains to the Xavier family home, in which Professor X originally started his X-Men team.