As we see in "Deathly Hallows" there are such things as anti-summoning charms;
‘There’s an easier way,’ said Hermione, as Harry wiped his inky
fingers on his jeans. She raised her wand and said, ‘Accio locket!’
Nothing happened. Ron, who had been searching the folds of the faded
curtains, looked disappointed. ‘Is that it, then? It’s not here?’ ‘Oh,
it could still be here, but under counter-enchantments,’ said
Hermione. ‘Charms to prevent it being summoned magically, you know.’
‘Like Voldemort put on the stone basin in the cave,’ said Harry,
remembering how he had been unable to Summon the fake locket.
Harry's cloak also seems to have an enchantment that prevents anyone other than the owner from summoning it;
Harry seized Ron’s wrist as he raised his wand. There were too many of
them to Stun: even attempting it would give away their position. One
of the Death Eaters waved his wand and the scream stopped, still
echoing around the distant mountains. ‘Accio Cloak!’ roared one of the
Death Eaters. Harry seized its folds, but it made no attempt to
escape: the Summoning Charm had not worked on it.
It's likely that anyone who was deliberately hiding something would put such a charm on the hidden object.
Additionally, accio'd items still need to travel physically. If you hid something inside something relatively immovable (such as by putting it under a flagstone) it wouldn't come when summoned.