This one's actually rather complicated to explain, so bear with me. In the original novel, the horde was evidently protected by a magical door.
They followed the tracks up the hill, until hidden by bushes they came
on a big door of stone leading to a cave. But they could not open it,
not though they all pushed while Gandalf tried various incantations.
"Would this be any good?" asked Bilbo, when they were getting tired
and angry. "I found it on the ground where the trolls had their
fight." He held out a largish key,
The Hobbit, or There and Back Again
When the screenplay was originally written by Jackson and Boyens, the scene with Gandalf saying to be careful was apparently going to be said directly after they made their way through the enchanted door (e.g. be careful, there might be magical traps) however, they then changed the order of events and moved the cave entrance into a descending tunnel rather than having them enter through a door. The line about being careful remained, but the rest of the dialogue was scrapped.
Boyens: In the sense of Gandalf, with things not being really as they should [e.g. finding trolls in this part of the world]
Jackson: We wanted to start the feeling that all is not as it should be, and to give Gandalf this disquiet. We originally designed this cave in a slightly different way, and when we found the location for the forest, we found the natural rock that seemed to lend itself to being the entrance to the cave and so that then caused us to redesign it to be like a tunnel that goes down a hill under the ground, which really ended up being designed that way because of this big overhanging rock which we used on the location. You see more of it when we come back to the panning over of Sting in a minute.
Audio Commentary: The Hobbit - An Unexpected Journey
Obviously there are also dangerous creatures living in the cave and rotting (infected) carcasses strewn about with broken and protruding shards of bone to catch the unwary. General caution is well advised, even if there's no evidence of magic.