He didn't find a Silmaril in Orthanc, he found a palantír:
Will you, Aragorn, take the Orthanc-stone and guard it? It is a dangerous charge.'
'Dangerous indeed, but not to all,' said Aragorn. 'There is one who may claim it by right. For this assuredly is the palantír of Orthanc from the treasury of Elendil, set here by the Kings of Gondor. Now my hour draws near. I will take it.'
The Two Towers Book IV Chapter 11: "The Palantír"
By the Third Age (when Lord of the Rings takes place), the three Silmarils are lost, considered beyond the reach of Elves, Men, and Orcs:
- One is flying through the sky, borne by Eärendil
- One is at the bottom of the sea, thrown there by Fëanor's son Maglor
- The last is somewhere underground; Fëanor's son Maedhros flung it (and himself) into a crack in the Earth's crust
It's also worth noting that Gandalf himself didn't find it; strictly speaking Pippin did, and he only did that because Wormtongue flung it at Saruman and nearly decapitated Gandalf:
'Saruman, your staff is broken.' There was a crack, and the staff split asunder in Saruman's hand, and the head of it fell down at Gandalf's feet. 'Go!' said Gandalf. With a cry Saruman fell back and crawled away. At that moment a heavy shining thing came hurtling down from above. It glanced off the iron rail, even as Saruman left it, and passing close to Gandalf's head, it smote the stair on which he stood. The rail rang and snapped. The stair cracked and splintered in glittering sparks. But the ball was unharmed: it rolled on down the steps, a globe of crystal, dark, but glowing with a heart of fire. As it bounded away towards a pool Pippin ran after it and picked it up.
The Two Towers Book IV Chapter 10: "The Voice of Saruman"
The palantíri are significantly larger than the Silmarils were; we're told in Unfinished Tales that, at minimum, they were about a foot in diameter:
At smallest they were about a foot in diameter, but some, certainly the Stones of Osgiliath and Amon Sûl, were much larger and could not be lifted by one man.
Unfinished Tales Part 4 Chapter III: "The Palantíri"
If Wormtongue managed to huck it out of the tower with as much accuracy as he did, the Orthanc stone is probably sitting at the low end of that scale; so it's not quite so implausible that Pippin found it as easily as he did.