First thing to say is, that it's not just about LotR. In most stories about fantasy worlds, with elves or similar superior species, there are different ages.
Something like the dark ages (gods, ghosts and other mystic creatures), followed by the rise of the elves and other "old" races. But at the end humans arrive, expand and push them to the edge of existence or for whatever reason, the elves have since "disappeared".
Of course, not every elvish race in SciFi is the same, but still there are many similarities. Lets call it a stereo type.
So, judging by plethora of the most prevalent SciFi, what is the most common reason that elves always disappear?