On this video, we can see Jory and Jaime having a nice talk like old friends. Jaime says Jory don't need to apologize, then they remember the siege at Pyke, where they both fought for the same side. Jory and Jaime
Then suddenly, Jaime remembers he should't be friendly with Jory... and disrespects him. At this time the Lannisters and the Starks did not have the "Tyrion problem" between them. So I don't see any reason for Jaime being unfriendly with Jory.
This questions here reveals more details about History between the Lannisters and the Starks : What was the original cause of enmity between Stark and Lannister?