I'm trying to identify a science fiction novel that I read in the UK around 1980. It was a library book, so the book was probably published sometime during the 1970s.
The story was basically a coming of age story. The protagonist was male. I don't recall much about the incidents in the novel. I think he had various lessons and agonized a bit about what he would do in the future. I do recall that at one point he sheltered, with a woman from some severe storm in a cave. Maybe there was a scene in a swimming pool?
The setting was some colonized world. The people on this world were human, and were well established on the world. They no longer had the technology (or interest) to travel to other worlds. They viewed the world that they lived on as their home. They had lost all contact with other human worlds.
The colonists were not the only sentient people on the planet. There were a few aliens. These aliens (who might or might not have been native to the world, but probably not I think) lived in their own building within a city. The building was described as being spiral in shape. The building was like a ring, that rose up into the sky, getting narrow as it went up. The colonists were not able to build anything like it themselves. As far as I can recall they were unable and unwilling to enter the alien's building.
The colonists knew very little about the aliens. They got on well with them, but mostly because they left them alone. They viewed them as a mystery.
I think that towards the end of the novel the world was disturbed by the arrival of spaceships from Earth. The people on these ships were imperial in nature - they expected the colonists to join their federation or empire or whatever and no was not an option. They forced their way into the aliens' building. I think there was a description of some kind of acid splashed on the lock. I think the end was generally depressing - neither the colonists nor the aliens could do anything to stop the humans from Earth.
I don't have a clear image of the book. I think it was hardback. I think that the cover might have shown a cityscape in silhouette, with the spiral building standing taller than the others.