You've actually asked three different questions. I'll try to cover them all;
Are Agents invulnerable?
No, Agents are most definitely not invulnerable. In the films we see that Agents can be shot and killed. A good example of this is where Agent Jones is dispatched with a shot to the head in the excellent "Dodge this..." scene.
We see that they can also suffer non-mortal wounds in the Matrix comic "Day In, Day Out" (where an unnamed Agent loses his arm to falling glass) and "Reloaded" (where Morpheus cuts an Agent's face with his sword).
Are Agents Superhuman?
Yes, when a human is taken over by an Agent we see that their strength, agility and speed are instantly amped up to superhuman levels. This seems to include the amount of physical impact trauma they can receive. A good example would be in the Subway Fight where Smith punches through solid concrete without damaging his knuckles.
Are Agents immortal?
The short answer to this one is yes. Whilst there's no canon evidence that the Agents are or aren't susceptible to mundane ways of dying (such as illness and old age) I think we can be reasonably certain that they would be.
In terms of violent death, a few seconds after Agent Jones is dispatched on the roof (courtesy of a bullet to the head) he turns up again, having possessed a new human body. If they can discarnate and reincarnate at will, for all intents and purposes they are immortal.