I read this book somewhere around 1996/97, probably on the earlier side, and I'd like to track it down. It was in the school library (Durban High School, South Africa), but for the policies of that school at the time, it probably should not have been there, as it did definitely contain a mention of sex (strongly against school policy, which is why it stuck in my mind).
It's set on Earth, but in a much different societal model than we have. The children are taken away from parents, and raised collectively, so that they would not have the biases of their parents. This was apparently driven by aliens that came to help structure Earth in a better way. However the aliens communicate with the humans extremely little, and are recluses in a tower structure.
The first part covers the three main characters, all teenagers: two boys and a girl, covering about growing up, including a fumbling virgin threesome in a wheat field. Soon thereafter, the girl is then taken away to be the next alien translator. The boys then spend most of their life trying to get the girl back, only succeeding near the end of their lives.
Edit 2015/05/13:
I dug up Native Tongue as suggested as possible duplicate, and it's definitely NOT the same.