The Prisoner of Pineapple Place by Anne Lindbergh? There are more books about the same character by the way (The People in Pineapple Place).
Blurb found on Google Books:
Pineapple Place, an invisible street that moves from city to city and keeps its inhabitants the same age forever, is threatened with change when nine-year-old Jeremiah becomes bored and makes contact with the outside world.
Blurb found on Amazon:
The invisible residents of Pineapple Place have lived on the same block for fifty years, where nobody ages and nothing changes - and Jeremiah can’t stand it anymore! Then Mr. Sweeny accidentally moves them to Athens, Connecticut, instead of Athens, Greece, and Jeremiah meets Ruby, a girl from the outside world who can see him in reflections. Will things ever be the same?
It's a long shot, but without details, the only thing I can think of.