There's several possible reasons.
Firstly, human mouths are dirty. A bite from a normal human is likely to become infected in many cases even today, when we have over-the-counter disinfectants, bandages, etc on hand in almost all locations. Left untreated, this alone could be fatal. This is unlikely today, in modern society, but not nearly as unlikely if society has broken down and medical treatment is unavailable. This is discussed to some degree in The Zombie Survival Guide by Max Brooks, which is based (as is TWD) on Romero-style zombies.
Next there's the reanimation. The virus is dormant, as you say, until death. If the zombie has any form of saliva (which they seem to) or moisture in their mouth (without which verbalizing would be nearly impossible) their bites will almost certainly introduce the live, active virus into the bitten person's bloodstream.
There's a HUGE difference between a dormant virus and an active one.
The active virus seems to cause fever, death, and reanimation. The dormant virus activates upon death, causing reanimation. It's also possible that the dormant virus activates upon contact with an active sample of the virus, which would help account for the rapid onset of symptoms.
In either case, amputation could potentially save the life of someone bitten upon a limb - if the active virus doesn't have time to make it to the heart, it can't propagate (either normally or by activating the dormant virus) throughout the rest of the body.