This is what Eloise had to say in 316 (Season 5, Episode 6):
ELOISE: Jack... the binder in your hands shows all the air routes that fly over the coordinates where I believe the Island will be in
little more than a day from now. There's a commercial airliner flying
from L.A. to Guam. It's going to go right through our window. Ajira
Airways, Flight 316. If you have any hope of the Island bringing you
back, it must be that plane. You all need to be on it. It must be that
flight. [Sighs] If you... want to return, you need to recreate as best
you can the circumstances that brought you there in the first place.
That means as many of the same people as you are able to bring with
you.
JACK: And what if we can't get anyone else to come with us? What if we're it?
ELOISE: All I can tell you is the result would be... unpredictable.
JACK: So that's it? We just get on that flight, and we just hope that it works? That's all?
ELOISE: No, that's not all, Jack... at least, not for you.
We know from later in season 5, The Incident (Season 5, Episode 16,17), that Eloise knew that Jack had an important job to do with respect to the bomb, Jughead. Additionally in that episode, she learns about Daniel's journal.
After leaving the Island, she spends her time making sure that what happened, happened. E.g. her handling of Desmond in Flashes Before Your Eyes and Daniel in The Variable, so it is likely that she has prepared for this moment and returning the Candidates back to the Island.
One of the themes of Lost was Science vs Faith.
ELOISE: Ohh... there are many reasons, I'm sure, but the only one that
matters is this--he is going to help you get back. John is going to be
a proxy. A substitute.
JACK: A substitute for who?
ELOISE: Jack... who do you think? You need to, as best you can,
recreate the conditions of the original flight... which is why you
need to give John something of your father's. You have to get
something that belonged to your father, and you need to give it to
John.
JACK: What?
ELOISE: That's what you have to do.
JACK: My father... is gone. My father has been dead for three years.
You want me to... [scoffs] a--and to give it to Locke?! He's in a
coffin! This is ridiculous!
ELOISE: Oh, stop thinking how ridiculous it is and start asking
yourself whether or not you believe it's going to work. That's why
it's called a leap of faith, Jack.
Jack has long been the Man of Science, this is part of his transition to a different point of view. This leap of faith is important to him being accepted back by the Island, and why his previous attempts (along with the Island's movement) have failed.
As noted, other people such as Ben and Richard have been shown to leave and return to the Island at will.
However, after the events of There's No Place Like Home (Season 4, Episode 12,13,14) and when it was done properly in This Place is Death (Season 5, Episode 5) the Island is moving, to protect it from Widmore. Now they need a device such as the Lamppost Station in 316 to find it.
When Richard, Ben, Mr Happy etc... were leaving the island it was stationary and so "easy" to find.
It's worth noting after moving the Island, Ben couldn't return and Ajira 316 was his only way back.
Now, why was the recreation necessary? Why was it important they did this rather than just being in the right place at the right time?
We can assume that right place alone isn't enough, as although the Island is moving there must be multiple aircraft and ships going over it all the time without incident. Even the original Oceanic 815 was only brought down by Desmond's negligence.
So what was different about Ajira 316? It had Candidates expressing a desire to return. Additionally, for the timeline to work Jack, Hurley, Kate and Sayid needed to do what they did in 1977. (Sorting out Jughead, and their interactions with Ben, for example.)
In summary:
- It had to happen that way, because that's how it has always happened (predestination).
- It made the survivors/Candidates stand out more to the Island, and proved their desire to return. (leap of faith)
- Eloise had learnt that this particular set up was necessary for them to travel back to 1977. (which can be a mixture of the two above)
- The Island was course correcting, ensuring the 1977 events happened as needed.