After the short duel between Gandalf and Saruman in The Fellowship of the Ring, Gandalf gets trapped on the roof of Orthanc. But it looks to be a smooth top without any doors or stairs (according to the films). Do the books go into any detail on how Saruman managed to get Gandalf trapped on the roof?
From "The Lord of the Rings", chapter "The Council of Elrond".
Gandalf tells of his imprisonment in Orthanc (or ON Orthanc is better said...):
They took me and they set me alone on the pinnacle of Orthanc, in the place where Saruman was accustomed to watch the stars. There is no descent save by a narrow stair of many thousand steps, and the valley below seems far away.
As far as I know that is the only reference to what the top of Orthanc is like anywhere in the books.
According to this answer, and my own recollections of reading the appropriate passage in the books (from the chapter called "The Council of Elrond" in The Fellowship of the Ring), Gandalf would have been led up "many thousands of steps" which wound their way around the inside of the tower, and then imprisoned at the summit, which would have most likely (though no-one knows for certain) been accessed via a hatch or trapdoor in the ceiling of the topmost chamber of Orthanc. I always imagined it was similar to how Sam found Frodo when he was kept prisoner in the tower of Cirith Ungol, since both those towers were originally Gondorian in origin.
In the film, unlike the books, Gandalf was forcibly transported (levitated) to the top of the tower via Saruman's magic, after Gandalf discovered the White Wizards's treachery.
To answer directly your question: The powers of Gandalf and Saruman were far, far beyond "boom", "bam", "light and camera", "action" and "Gandalf is thrown by invisible power on the top of tower" :) Take it as infinite amount of bullets in the gun's magazine or jumping'n'flying cars in action movies.
There was no battle in Orthanc in the book. And Gandalf said:
"They took me and they set me alone on the pinnacle of Orthanc, in the place where Saruman was accustomed to watch the stars.There is no descent save by a narrow stair..."
So he was ordinarily taken by some sort of guards - probably normal humans - because as Saruman was stronger than Gandalf, Gandalf never stooped so low to kill Saruman's servants for no real purpose. And Saruman had men as servants - as he probably didn't trust orcs, as is mentioned elsewhere in the book (Merry's or Pippin's speech about food they found in Isengard?).
There is a description of Orthanc and Isengard in pre-Saruman times:
four mighty piers of many-sided stone were welded into one, but near the summit they opened into gaping horns. their pinnacles sharp as the points of spears, keen-edged as knives. Between them was a narrow space, and there upon a floor of polished stone, written with strange signs, a man might stand five hundred feet above the plain. This was Orthanc, the citadel of Saruman, the name of which had (by design or chance) a twofold meaning; for in the Elvish speech orthanc signifies Mount Fang, but in the language of the Mark of old the Cunning Mind. A strong place and wonderful was Isengard, and long it had been beautiful; and there great lords had dwelt, the wardens of Gondor upon the West, and wise men that watched the stars.
So there had to be a way for a mortal "wise man" to reach the top observatory to watch the stars.
There are no mentions of any duels, although it's hinted at.
If anyone wants to see an alternative portrayal of that moment have a look at the earlier animated version, Lord of the Rings (1978). It's a very psychedelic piece of animation, so it might be interpreted as a duel.
The feeling I get from the book is that although Gandalf is almost Saruman's equal, the use of the ring, given to him by Sauron, overcomes any resistance Gandalf could have mustered. Remember at this time his ownership of one of the three rings is still secret.