Read this story many years ago in high school (around 2009), and it's been bugging me since. I can't say for sure when the book was published, but it feels recent (so the 90's or 00's). In the story, humans were already space faring. They'd formed an alliance with another species of aliens; a genetically engineered servant caste that had broken free from their creators (this wasn't important to the main story).
The humans find a star system containing two important planets. The first planet was home to a very violent race (let's call them species A) who had only recently obtained space travel. Unlike humans, these aliens didn't sleep daily and instead hibernated periodically.
The second planet was a wasteland, and to their horror humans realized this was because species A of the first planet had used nukes to annihilate the surface and kill whatever intelligent life lived there (species B). Later, it's learned that some of species B survived. It was also discovered that species B were somewhat psychic. They feed on the "intelligence" of species A, and because of this every time their planets approached each other species A would experience massive spikes in violence. Eventually realizing this, A nuked B in retaliation, and to allow themselves to develop a society that wouldn't try and kill itself.
Because of A's violent nature, instead of contacting them humans instead setup a secret military base on an asteroid to observe them quietly. The novel begins when species A launches a surprise attack on the base.
