A palantir has to be placed in the right position for it to work
Pippin, it's noted in Unfinished Tales, by sheer "coincidence" sat the Orthanc Stone in its right position, facing it towards Mordor in the East while he himself sat on the West.
So it was "by chance" as Men call it (as Gandalf would have said) that Peregrin, fumbling with the Stone, must have set it on the ground more or less "upright," and sitting westward of it have had the fixed east-looking face in the proper position.
Unfinished Tales, Part 4, III, The Palantiri
And coming back to that of course, when Pippin first picked up the Stone he didn't place it in the right position, so it didn't work then.
Though without any external markings of any kind they had permanent poles, and were originally so placed in their sites that they stood "upright:" their diameters from pole to pole pointed to the earth's centre, but the permanent nether pole must then be at the bottom.
Unfinished Tales, Part 4, III, The Palantiri
Finally coming to when he does get it right, he established a link between the stone now in Sauron's possession.
'"So you have come back? Why have you neglected to report for so long?"
The Lord of the Rings, The Two Towers, The Palantir
The movies got it wrong (duh) by not showing the Stone being placed on the ground for it to work.