In the current version of the Matrix (current as of the start of the first movie), the machines did not create any monsters or other fantastical elements.
However, between Agent Smith, Persephone, and the Architect's expositions, we learn that this is the third iteration of the Matrix. Smith explains the first one -- a paradise matrix where every human got everything they wanted. The human mind rejected that version as too unreal ("whole crops were lost"), so they scrapped that version.
The second version is a little more mysterious, since we have to wade through the Architect's dialog to get to it. It was a matrix where the machines tried to build in the concept of cause and effect. This is the matrix where The Merovingian came into play (thus his fascination with cause/effect in his dialog). Persephone mentions that many of his henchman came from an "older version" of the matrix --The Twins, for example, are supposedly the second matrix's version of Agents.
This second matrix was a "nightmare" world, the exact opposite of the paradise world. It was populated by all of the horrors from the human psyche. Presumably, this was the machine's natural reaction to the first failure -- do everything exactly the opposite. In that matrix, all of the fantasy monsters from our mythologies were real. Thankfully, humans must have rejected that Matrix too, because they made a third one.
In the third, current matrix, the Oracle introduced the concept of choice ("free will"), allowing the attached minds to behave on their own. That was the final key element that kept the humans invested in the illusion (but is also what led to the eventual need for The One). This third matrix is a faithful replication of late-20th-century human culture, so there won't be any monsters in it. Those old programs should have been hunted down and terminated ("returned to the source") in the same way that The Keymaker, Sati, etc. should have been. But obviously, some obsolete programs escaped, and The Merovingian likes to collect them.
So yes, there were monsters in the matrix but they were anomalies that should have been wiped out, but managed to escape.