*Earlier in the film*<br />
"He will join us or die." - Vader

*later*<br />
"Join me." - Vader<br />
"NEVER!" - Luke<br />
"So...let my son be tortured to the breaking point by Sidious, or kill him...oh look, he's falling.  Decision made." - Vader's thoughts<br />
"[Eat pit, Jedi.][1]" - Vader

Edit: It has been pointed out that the link no longer works.  The link was to a description of the Jedi Pit Theory. A full explanation really doesn't add anything to the answer (or I would have copied in relevant parts) but the 'Jedi Pit Theory' states that Jedi are nearly unassailable warriors except in the presence of pits. It points out how, in the OT, every time a Jedi is in trouble, they are near a pit (Ben dies on the Death Star, near a shaft; Palpatine dies when he falls into a pit, Luke is in trouble near the Sarlaac pit, etc)

  [1]: http://www.galacticsenate.com/showthread.php?10294-The-Jedi-Pit-Theory