Sci-fi book I read as a kid with a house and a caretaker. The house was used by alien races as a station when travelling thru space. The house looked normal from a distance but could not be entered because of some invisible force field. The man was chosen by the aliens as a caretaker of this house.
Are you thinking of Way Station, by Clifford Simak?
Enoch Wallace, an American Civil War veteran, is chosen by an alien called Ulysses to administer a way station for interplanetary travel. Travelers arrive by a form of teleportation by duplication, where the original body remains at the source and a new live copy is created at the destination. Enoch's job is to monitor the machinery of the way station and make sure arrival conditions meet the biological needs of the wide variety of aliens. (Wikipedia)