I'm pretty sure I read this story at Baen Free Library, but as they've taken down most stories to re-organize, I don't remember the title. I'm not sure if it was a book by itself or just a story in a collection - I think it was a in collection.
The story is set in the future and computers and AI-programs are controlling very much. Some wants to let it control more, but are vary that it may become intelligent and turn against it's human masters. It's quite a philosophical debate on the issue, among other thing how logic and rationality (ie. a computer program), could cause a response indistinguishable from aggression and a "evolutionary" fight for resources - but really for quite different reasons.
Eventually they decide to set the proposed system up in an abandoned space-station and let it control most aspect of it. Military-and science-personnel will be on-board, partly to monitor and partly to disrupt and "attack" the system. The hope is to see how the system responds, if it will "evolve" and become truly sentient, and if it will attack humans. The theory being that if they can't get the test-system to evolve and/or attack humans, then there is no danger for the real system they plan back on earth ever will either.
Of course, things doesn't go quite after plan, and for a while humans and computer is at war. Finally a truce is reach, and the program actually understand why the humans did it, that humans are fragile and must be protected - and that it soon will return to earth.