This might be A Day for Damnation by David Gerrold. It's the second book in his series The War Against the Chtorr.
The Chtorr are aliens that look like giant worms and do indeed burrow into the ground. In A Day for Damnation one of the Chtorr has been captured and is being experimented on to see how intelligent it is. One of the tests is to see if the Chtorr can solve a series of problems to open a cage with a rabbit in it. The scientists have nicknamed the Chtorr Tiny:
"For your information," Fletcher said, "Tiny has never seen this puzzle before. It is not the most complicated one we've assembled, but for the purposes of this demonstration we thought we'd keep it short. All of our puzzles are rigged to keep a record of Tiny's moves-and once Tiny goes to work the life-expectancy of the rabbit can be measured in minutes. The longest Tiny has ever taken was half an hour."
That means Tiny's solved the puzzle," said Fletcher. - the glass case popped open. Tiny grabbed the rabbit with one dark claw, lifted it up high, it squealed, I didn't know that rabbits could scream-and shoved the creature into its gaping maw. There was a wet slobbery crunching sound, and then Tiny uttered a soft trill of pleasure, and looked around for more.