I don't think there's any canon answer, but I can answer from a real world perspective. There are a couple of things at play here, I think.
Firstly, I imagine that they did extensive testing on a smaller scale, and it probably worked fine. It was only when the testing was expanded to a larger scale that the problems arose.
Why did the problems arise on a large scale? A couple of points there as well...read the Challenger investigation chaired by Richard Feynman. There's a lot of discussion of the engineering culture whereby you rose in your career if you were optimistic, and were filtered out otherwise. Secondly, read about Agent Orange (a large scale defoliant) use in Vietnam. The chemical engineers come up with a safe way (for the deployers, anyhow) to use the product, the company relaxes that standard, the contractor relaxes that standard, the military relaxes that standard, and so on and so on, until you have soldiers exposed to toxic levels of dioxin.
Finally, and most significantly, the Alliance believed in the boundless good of their own intentions. They believed themselves, to paraphrase Jefferson, booted and spurred to ride mankind - people were merely pawns on their chessboard, and not human beings - that's the ultimate message of the series and the movie. This arrogance, borne from the self-regard of their own piety, made them careless.