One also must remember that most armies are comprised of common folk who have been called by their liege lords, and tat be a long chain of lords until you reach a great house. The great houses are fighting for their own interests and not those of the common folk. Why should they continue the fight once the reason the fight was started is no longer an issue?
In a open field battle, even if the knights and direct retainers wanted to avenge their lord who lost the single combat they would most likely be out numbered as the common folk of the lord who won would still be there. For a castle being sieges it is much the same philosophy, except at that the common folk inside know it will only be a matter of time before they starve. Most would see the only option as opening the gate and praying the attackers are merciful.
For an in-universe perspective look to what happened when Renly was killed prior to meeting Stannis in battle. Renly is killed and his army divides and scatters. It is very demoralizing when your commanding officer is taken from the fight. The commanding officer is usually the best tactician as well, so an army missing their leader will quickly lose ranks and order. I will admit Renly's case is a bit different because his army was supposed to have sworn fealty to Stannis from the start. Once Renly was dead most of the lesser houses promptly went to Stannis' side.