Why was Luke Skywalker missing in The Force Awakens?

Question

In The Force Awakens, we learn in the opening crawl that Luke Skywalker has disappeared.

He is in fact

absent for most of the movie except for a few seconds at the very end (and in a vision / flashback scene).

Why was this?

I am looking both in and out-of-universe explanations.

1. In-universe: Why did this character disappear, choosing to essentially go into hiding?

2. Out-of-universe: Why did the filmmakers choose to

leave this Luke Skywalker out of the main story?
_{(i.e. Did Abrams think Mark Hammill was a bad actor? Was he busy or sick? Anything like that...)}

Answer 1

Out of universe:

Because they thought Luke would draw the attention away from the other new lead characters

According to an article from ScreenRant.com:

Early on I tried to write versions of the story where [Rey] is at home, her home is destroyed, and then she goes on the road and meets Luke. And then she goes and kicks the bad guy’s ass. It just never worked and I struggled with this. This was back in 2012.

It just felt like every time Luke came in and entered the movie, he just took it over. Suddenly you didn’t care about your main character anymore because, ‘Oh [wow], Luke Skywalker’s here. I want to see what he’s going to do.’
_{- Michael Arndt}

In universe:

Sometime after the events of Return of the Jedi, Luke

started training a new generation of Jedi. When one of them turned to the Dark Side and killed the others, Luke held himself responsible so he went into exile.

Answer 2

In universe:

As Han explains. The new Jedi order had been destroyed. He had failed and unleashed a new terror on the Galaxy. He needed some space

Out of universe:

I think there needed to be space for the new characters to grow.

It was also some misdirection to keep us guessing. Mark was part of all the PR and very excited about the film. We were meant to believe he was a full part of the film.

It wasn't till the later trailers came out that people started asking 'where is Luke?' And JJ said

These are good questions to be asking. I can't wait for you to find out the answers,

Answer 3

Out of universe: because no beard.

From a very recent interview with Mark Hamill:

Hamill was just starting to regrow his Jedi beard for the follow-up, Episode VIII — which would soon need him on set. He had shaved to play the Trickster on the new season of The Flash, and had to ask Lucasfilm's permission to do so. "The bottom line was if I couldn't grow it back fast enough, I couldn't do the part," says Hamill, 64. "Luckily, I have 10 weeks, so fingers crossed!"

Answer 4

In-universe

Han Solo explains it to Rey:

• Luke basically pulled an Obi-Wan and trained Vader Jr. Almost literally.

• And then he pulled a Amazing Disappearing Yoda trick.

  Luke learned well from his past Masters :)

WGA script:

REY
Why'd he leave?
HAN
He was training a new generation of Jedi. One boy, an apprentice turned against him, destroyed it all. Luke felt responsible... He walked away from everything

Alan Dean Foster's novelization:

Rey spoke while drinking in the details of the marvelous but imperfect chart. “Why’d he leave, anyway?”
Han pursed his lips; thinking back, remembering.
“He was training a new generation of Jedi. There was no one else left to do it, so he took the burden on himself. Everything was going good, until one boy, an apprentice, turned against him and destroyed it all. Everything Luke had worked toward: gone. Luke felt responsible. He walked away from everything.”
Finn’s tone was respectful. “Do you know what happened to him? Does anyone?”
Han turned to him. “There’ve been all kinds of rumors and stories. When people don’t have access to facts, they invent what they’d like to believe, or what they think others would like to hear. The people who knew him the best think he went on a personal quest, looking for the first Jedi temple.”

Out of universe

First of all, @RickSanchez's answer is totally awsome and probably better at addressing OOU than what I found.

But I found a slightly different, complementary reasoning to the one he cited. During Guild Screenings Q&A with the screenwriters, J.J. Abrams said about his reason to agree to work on the reboot, that it was an exciting idea to explore in terms of...:

“Luke Skywalker is potentially an unknown. Luke Skywalker is potentially a myth. Luke Skywalker, is like, you know, King Arthur. To someone who’s 19 years old, what does that mean?”

Admittely, most of the famous Arthuriana deals with actually-active King Arthur, not a shut-off in exile King Arthur, but we'll forgive that inaccuracy because NO MIDICHLORIANS!