I've read the question Can't the Doctor just land the TARDIS outside New York during 1938? and could not found a satisfying answer.
At the start of the S07E05 The Angels Take Manhattan, the Doctor, Rory and Amy were in NYC 2012. As the plot went on, it appeared there were baby Angels in NYC 2012 too. Rory got zap back to NYC 1938, which is a battery farm for the Angels. Rory witnessed his own death and his second death created a paradox so strong that undone the whole event and push them back to cemetery of Queens 2012.
At this point, all of them were safe. Except that there was a tombstone with the name of Rory on it. Then Rory saw it and coined his death once again. No one asked why there was a tombstone.
So why there is a tombstone for Rory in the cemetery of Queens 2012?