One factor was the difficulty Eragon would have reconciling his role as the new leader of the Dragon Riders with his sworn loyalty to Nasuada and his deep connections and friendships with the Elves (and Dwarves). He says as much in Inheritance, p. 804-805:
"Eragon! What are you thinking! You know you have to join. All of the magicians who serve me have to. There can't be any exceptions. Not one! I can't have people think that I'm playing favorites. It would sow dissent among the ranks of the magicians, and that is exactly what I don't want. As long as you are a subjeect of my realm, you have to abide by its laws, or my authority means nothing. I shouldn't have to tell you that, Eragon."
"You don't. I'm well aware of it, which is why Saphira and I have decided to leave Alagaësia."
...
"I don't understand."
...
"It never would have worked for me to take charge of the magicians. Saphira and I have to raise the dragons and train the Riders, and that must take precedence before all else. Even if I had the time, I couldn't lead the Riders and still answer to you - the other races would never stand for it. Despite Arya's choice to become queen, the Riders have to remain as impartial as possible. If we start to play favorites, it will destroy Alagaësia. The only way I would consider accepting the position would be if the magicians were to include those of every race - even the Urgals - but that's not likely to happen. Besides, it would still leave the question of what to do about the eggs and the Eldunarí."
In the same passage, Eragon also lists several other logistical factors for leaving:
The need for physical barriers to defend the Dragons, most importantly the distance
The need to keep the Dragons fed with livestock other than that from farms and Dwarven hunt animals
But then Eragon reveals the most important reason for leaving, somewhat similar to the first reason I mentioned, but with a twist - it wouldn't be right for him to stay, but not because of the loyalty issues. As he says:
"I've become too powerful. As long as I'm here, your authority - and that of Arya, Orik, and Orrin - will always be in doubt. If I asked them to, most everyone in Surda, Teirm, and your own kingdom would follow me. And with the Eldunarí to help me, there is no one who can stand against me, not even Murtagh or Arya."
"...because I have this power at my disposal, I have a responsibility to act...If I was wrong, though, who could stop me? I could end up becoming Galbatorix, despite my best intentions...My presence here unbalances things, Nasuada. If I am to avoid becoming what I hate, then I have to leave."
TL;DR: It wasn't just fear for the dragon's safety, it was the need to let the lands be free from his influence, to not be ruled by another Rider King. The fact that there were logistical, safety, and loyalty questions as well were just icing on the cake.