There are more humans in the Milky Way compared to Pegasus
When you're an ascended being that thrives on people worshiping you, you want as many as you can to join your side. There are thousands of worlds full of humans and Jaffa in the Milky Way galaxy, and they are at relative peace. The Pegasus Galaxy has far fewer worlds populated by humans, and there are smaller numbers of them due to the Wraith cullings.
More People = More Worshippers = More Power
The Wraith don't make good worshippers
The Wraith aren't interested in eternal life; they can already live unnaturally long by feeding on humans. The Wraith aren't interested in worshipping a higher power; they are already that higher power to the "Wraith worshippers" that revere them. And the Wraith will probably just try to feed on any Priors who visit them.
Oh, and let's not forget that the Wraith already beat the Ancients once. Even with their powers, I imagine that's got to give the Ori pause in leading a crusade against them.
Finding a suitable place to send a Prior in the Pegasus Galaxy is a crapshoot
Let's assume that the Ori have the ability to determine which gate addresses exist in another Galaxy (we see that Adria is able to use her power to manipulate a DHD, and the Ori are related to the gatebuilders after all). Daniel Jackson knows many planets in the Milky Way galaxy that have humans on them, but I find it hard to believe that he really had time to study any of the planets in the Pegasus Galaxy. So it seems the Priors would have to guess which planets to send the Priors to.
In the Milky Way, virtually every gate is located on an inhabitable world. But in the Pegasus Galaxy, the gates that the Atlantis team had come across by that point were:
- Atlantis (has a shield, Prior can't come through)
- Wraith-controlled planets (Wraith make bad worshippers)
- Completely desolated planets (no worshippers)
- Outer space (a one way trip for any Prior who goes through)
- Planets with small populations of humans (not really worth their time)
Later on, they also discover planets of human Replicators. I highly doubt that them worshipping the Ori would do any good (a similar question was brought up later when some tried to ascend, to the doubt of the Atlantis expedition).
Really, trying to go to the few planets with humans is a huge risk when so many other Stargate destinations are unfruitful at best or fatal at worst.
At the time of the Ori's discovery, Atlantis' fate was uncertain
It's a little unclear where Atlantis' "The Siege" three-parter takes place in relation to SG-1, but "The Siege, Part 3" aired after "Avalon, Part 1". Let's assume for simplicity that the two episodes occurred simultaneously. When Daniel Jackson left to find Avalon, a rescue team was just sent to save Atlantis from the Wraith and it hadn't been resolved yet. Daniel didn't return to the SGC until a few episodes later.
So I'm going to go out on a limb and say that at this point, the Ori didn't even know if Atlantis would be around for much longer.
The Jaffa and humans under Goa'uld rule are in a power vacuum, and already predisposed to worship supposed gods
Less than eight years ago, the vast majority of Jaffa and humans fully embraced the idea that the Goa'uld were living gods. It wasn't until the Tau'ri showed up and started killing them, and the few doubters like Teal'c rallied support for the Free Jaffa movement that this changed.
In the midst of a power vacuum and chaos, a number of people could be persuaded to fall back into old habits and accept the leadership of a race of beings that were even more powerful than the Goa'uld. Sure, Apophis could make his eyes glow and Anubis looked all scary. But the Priors can perform miracles and spread plagues. If any Jaffa was having a crisis of faith, turning to the Ori would seem pretty appealing.
Compare this to the Pegasus Galaxy humans. Really the only faiths we see are Wraith Worshippers and those who pray to the ancestors (e.g. the Athosians, Teyla's people). While praying to the Ori as ancestors might be appealing, this seems to be a minority view given that most planets don't have an active faith. Convincing them to worship gods might be more difficult.
Pegasus already has at least one ascended being worshipped the same way the Ori are
In the Season 1 episode "Sanctuary", Sheppard's team met Chaya Shar, an ascended Ancient who broke the rules to not interfere with unascended beings. As a result, she was sentenced to only be able to protect the people of the planet Proculus.
The people worshipped her as a goddess (and she was her own high priestess), so she likely is already gaining the same sort of power that the Ori did. I can't imagine the Ori are too keen on having competition, so it seems prudent to stay away from another such ascended being for the time being.
Bonus answer: Why didn't the Ori invade the Ida galaxy as well?
There is one other galaxy that Daniel Jackson knows about: the Ida galaxy where the Asgard live. However, this isn't a good place for the Ori to invade either:
- The Asgard impersonate religious deities and are well aware of ascension, since they are former allies to the Ancients. I can't imagine a single Asgard actually wanting to worship the Ori.
- They are dying out. The Ori seem to thrive on continued worship, and it's not really worth it if your people aren't expected to live much longer (and in just two years…)
- The Replicators are winning over control of the galaxy, and I highly doubt that worship from human Replicators gives the Ori any power.
We don't know much else about the Ida galaxy, but what we do know suggests that the Ori wouldn't get a single worshipper there.