Ser Barristan Selmy was a member of Aerys II's kingsguard, and one of the best fighters in the world. Considering he is sworn to defend Aerys, the man who Robert is waging a war against, Ser Barristan is a very dangerous man indeed. Even unarmed and in captive he is a dangerous man. This is evidenced by (A Storm of Swords Spoilers):
his ability to defeat the Titan's Bastard, a dangerous man, powerful fighter and commander of the Second Sons, with just a wooden staff. Roose considers it safer to kill him than to keep him captive.
Additionally Roose has no qualms with violence. He comes from a family that used flaying as a torture device. In A Storm of Swords/Season 5:
he betrayed and killed his Liege Lord/King Robb Stark at his uncle's wedding
And he supports his bastard son Ramsay Bolton who has an unhealthy obsession with violence and torture.
Clearly killing and violence come somewhat easy to him, so killing a dangerous servant of his enemy is an understanding course of action for him.