Bran had a long, dangerous and arduous journey. From recovering from being pushed out a window and learning to deal with being crippled. Being chased out of Winterfell and then North of the Wall. He lost comrades and faced near-death while needing to rely almost completely on the aid of others. He personally was marked by the Night King and was his primary target. And Jon wasn't there for any of it.
As for what Bran meant:
Things needed to end the way they did. If Jon had been guarding Bran through the story he never would have rallied the army that ultimately defeated the Night King or Daenerys and Cersei for that matter.
The impression is that Jon was apologizing for not being there to protect Bran and prevent his attempted murder when he was thrown from the tower and everything that came after it. Bran's reply is that without Jon going to the Wall, everything that came from that, which included the killing of the Night King which saved the living of Westeros, would not have happened.
Most likely not being there when Bran & co travelled north of the wall.
Sam knew about it but kept it quiet from Jon, presumably until later. It would have been handy to have some of the Night's Watch with them, since they are the only ones south of the wall with any experience of travelling beyond it.
But since the Night's Watch were drawing all the attention of the White Walkers, it would have made Bran's journey less dangerous. This could be what Bran meant with Jon being where he was supposed to be.
Jon was referring to the events happening in Winterfell after Robb went south. Jon ended up choosing duty over love by staying at the wall and subsequently going beyond the wall with Mormont. He was with Ygritte and the Free Folk at the time when Theon took Winterfell and Bran fled.
The ‘real North’ is where Jon is supposed to be and Bran knows it. That’s why he’s sending him back there.
I think Jon was referring to the fact that he wasn’t there to protect him from the night king at winter fell when Arya was. It’s probably something that's played on Jon’s mind since in the same way any big brother would want to protect their younger crippled sibling. Also that scene was never explained to fans, lots of people were saying Jon should’ve killed the NK and not Arya so it was the writers predicting this reaction and tying the lose ends up surrounding this I think, it’s almost as though Bran is saying to Jon you were meant to distract the undead dragon so Arya could do her job.