From an interview with Robert Jordan himself (item 13 in the linked list; emphasis mine):
Are all of Mat’s memories from his past lives?
RJ: No, Mat’s “old” memories are not from his past lives at all. The “sickness” he got from the Shadar Logoth dagger resulted in holes in his memory. He found whole stretches of his life that seemed to be missing. When he passed through the “doorframe” ter’angreal in Rhuidean, one of the things he said – not knowing that the rules here were different than in the other ter’angreal he had used – was that he wanted the holes in his memory filled up, meaning that he wanted to recover his own memories. In this place, however, it was not a matter of asking questions and receiving answers, but of striking bargains for what you want. What he received for that particular demand was memories gathered by the people on that side of the ter’angreal, memories from many men, all long dead, from many cultures. And since not everyone passing by has the nerve to journey through a ter’angreal to some other world, the memories he received were those of adventurers and soldiers and men of daring.
Footnote: RJ is obviously talking about the memories Mat received from the Eelfinn, so this quote does not rule out past life memories as an explanation for the Aemon memories in The Dragon Reborn Chapter 19 (before Mat ever visited the Aelfinn or the Eelfinn), nor does the Dromen and Demonen chat rule out the Old Blood as an explanation. Also, RJ meant to say that most of the men who provided the memories went through the Tower of Ghenjei; he corrected himself later in the TOR Questions of the Week.