I have good news: you're not crazy.1
Both Clara's apartment block and Rose's were filmed at the same out-of-universe location: the Lydstep Flats in Cardiff.
Here it is as it appears in the episode Rose, with Rose and the Ninth Doctor hurrying down the stairs and through the estate:
And the Doctor Who Locations site confirms that the same location appears in the episodes The Zygon Invasion and The Time of the Doctor. Here's a Youtube video showing the latter episode, Matt Smith's final Christmas episode, actually being filmed on location at Lydstep:
And another video from the production of Series 9:
Judging from the makeup Jenna Coleman is wearing in this one, she's playing Bonnie (aka "evil Zygon Clara") rather than the real Clara, so this is The Zygon Invasion filming we're looking at.
In-universe, they may both have lived in the Powell Estate, but it's never confirmed either way.
We know that Rose Tyler lived in the Powell Estate in Peckham. We don't know exactly where Clara lived, but she taught at Coal Hill School in Shoreditch, around half an hour's drive north of Peckham.
There has been assorted speculation, but no confirmation, that Clara and Rose live in the same estate in-universe. The best we can say is that it's possible. Since there's been no in-show confirmation, to get more than that you'd have to ask someone like Steven Moffat.
1 No footnote here; but I had you going, didn't I? (Credit to Jason Baker for the joke.)