I'm desperate to find a book that I read ages ago.
It started off with aliens contacting Earth and then anyone who wanted to leave with them had to meet them by a large expanse of water (sea or lake). When the humans got on board the aliens didn't appear to be there. The humans had to build their own camp out of the space ship's walls which were mouldable.
At some point one of the humans begins to communicate with the aliens through drawings on the walls.