Only Ilúvatar has the power to give, or to deny, his gift. But oaths done in the name of Ilúvatar may have consecuences beyond the power of the Valar. Fëanor's oath, as an example. Ilúvatar's will needn't to be perceived by men or elves clearly; his designs may appear in the history just as "Destiny" or "Doom". "Frodo was destined to find the Ring".

There's no text evidence that the Dead Men of Dunharrow made his allegiance to Isildur in the name of Ilúvatar, but the consecuences of his perjury say so. Not Isildur's power doomed them, but "Destiny".