This came up on another question, where in the Game Of Thrones books, The Unsullied have all of their genitalia removed, not just the penis.

>"In Yunkai and Meereen, eunuchs are often made by removing a boy's testicles, but leaving the penis. Such a creature is infertile, yet often still capable of erection. Only trouble can come of this. We remove the penis as well, leaving nothing. The Unsullied are the purest creatures on the earth." He gave Dany and Arstan another of his broad white smiles. "I have heard that in the Sunset Kingdoms men take solemn vows to keep chaste and father no children, but live only for their duty. Is it not so?"
> <sub>-- *A Storm of Swords*, chapter Daenerys II</sub>

Removal of the penis makes sense in such a context, both to force obedience and to degrade the soldiers, but if you castrate a man then he loses most/all(?) of his ability to produce testosterone. Testosterone, being desirable for a soldier that relies on strength and fighting prowess, would be foolish to get rid of.  
So for a (supposedly) elite army, why would it make sense to do so?