Edited to reflect new info:
By that point they had exhausted every option. The flyby was very, very fast and the sheer level of calculations necessary meant a certain amount of art and flying by the seat of the pants. But by that point, the Martian was running out of food, water, air, and options.
So yes, these things didn't occur to folks until later, but the circumstances were such that they couldn't even happen until later. Modifying the MAV, for example, took the boffins at NASA quite some time to work out, and the necessities of that alone were such that they had to plan accordingly.
In short: they exhausted every other option which would have been saner and less desperate, but as a result all of those options took time and resources which didn't pan out.
Edit: Adding relevant quote here from Chapter 16:
Mitch rubbed the back of his head. “Wow…549. That’s thirty-five sols before Watney runs out of food. That would solve everything.”
Teddy leaned forward. “Run us through it, Venkat. What would it entail?”
“Well,” Venkat began, “if they did this ‘Rich Purnell Maneuver,’ they’d start accelerating right away, to preserve their velocity and gain even more. They wouldn’t intercept Earth at all, but would come close enough to use a gravity assist to adjust course. Around that time, they’d pick up a resupply probe with provisions for the extended trip.
“After that, they’d be on an accelerating orbit toward Mars, arriving on Sol 549. Like I said, it’s a Mary flyby. This isn’t anything like a normal Ares mission. They’ll be going too fast to fall into orbit. The rest of the maneuver takes them back to Earth. They’d be home two hundred and eleven days after the flyby.”
“What good is a flyby?” Bruce asked. “They don’t have any way to get Watney off the surface.”
“Yeah…,” Venkat said. “Now for the unpleasant part: Watney would have to get to the Ares 4 MAV.”
A little bit later in the spaceship:
“Would this really work?” Martinez asked.
“Ja.” Vogel nodded. “I ran the numbers. They all check out. It is brilliant course. Amazing.”
“How would he get off Mars?” Martinez asked.
Lewis leaned forward. “There was more in the message,” she began. “We’d have to pick up a supply near Earth, and he’d have to get to Ares 4’s MAV.”
“Why all the cloak and dagger?” Beck asked.
“According to the message,” Lewis explained, “NASA rejected the idea. They’d rather take a big risk on Watney than a small risk on all of us. Whoever snuck it into Vogel’s e-mail obviously disagreed.”
“So,” Martinez said, “we’re talking about going directly against NASA’s decision?”
“Yes,” Lewis confirmed, “that’s exactly what we’re talking about. If we go through with the maneuver, they’ll have to send the supply ship or we’ll die. We have the opportunity to force their hand.”
Later, just After Mark arrives at the MAV:
“So is it ready?” Venkat asked.
“Yes, it’s ready. But you’re not going to like it.”
Bruce steeled himself and stood, picking up his briefcase. He pulled a booklet from it. “Bear in mind, this is the end result of thousands of hours of work, testing, and lateral thinking by all the best guys at JPL.”
“I’m sure it was hard to trim down a ship that’s already designed to be as light as possible,” Venkat said.
Bruce slid the booklet across the desk to Venkat. “The problem is the intercept velocity. The MAV is designed to get to low Mars orbit, which only requires 4.1 kps. But the Hermes flyby will be at 5.8 kps.”
Venkat flipped through the pages. “Care to summarize?”
“First, we’re going to add fuel. The MAV makes its own fuel from the Martian atmosphere, but it’s limited by how much hydrogen it has. It brought enough to make 19,397 kilograms of fuel, as it was designed to do. If we can give it more hydrogen, it can make more.”
“This is getting really risky, Bruce.”
Bruce sighed. “I know. There’s just no other way. And I’m not even to the nasty stuff yet.”
Venkat rubbed his forehead. “By all means, tell me the nasty stuff.”
“We’ll remove the nose airlock, the windows, and Hull Panel Nineteen.”
Venkat blinked. “You’re taking the front of the ship off?”
“Sure,” Bruce said. “The nose airlock alone is four hundred kilograms. The windows are pretty damn heavy, too. And they’re connected by Hull Panel Nineteen, so may as well take that, too.”
“So he’s going to launch with a big hole in the front of the ship?”
“We’ll have him cover it with Hab canvas.”
“Hab canvas? For a launch to orbit!?”
Bruce shrugged. “The hull’s mostly there to keep the air in. Mars’s atmosphere is so thin you don’t need a lot of streamlining. By the time the ship’s going fast enough for air resistance to matter, it’ll be high enough that there’s practically no air. We’ve run all the simulations. Should be good.”
“You’re sending him to space under a tarp.”
“Pretty much, yeah.”
“Like a hastily loaded pickup truck.”
“Yeah. Can I go on?”
“Sure, can’t wait.”
Mark reacts as you might imagine
>[08:41] MAV: You fucking kidding me?
>[09:55] HOUSTON: Admittedly, they are very invasive modifications, but they have to be done. The procedure doc we sent has instructions for carrying out each of these steps with tools you have on hand. Also, you’ll need to start electrolyzing water to get the hydrogen for the fuel plant. We’ll send you procedures for that shortly.
>[09:09] MAV: You’re sending me into space in a convertible.
[09:24] HOUSTON: There will be Hab canvas covering the holes. It will provide enough aerodynamics in Mars’s atmosphere.
[09:38] MAV: So it’s a ragtop. Much better.
The crew, meanwhile, has been practicing their maneuvers using the simulators, which haven't been going well:
Beck held a coil of metal wire in one hand and a pair of work gloves in the other. “Heya, Commander. What’s up?”
“I’d like to know your plan for recovering Mark.”
“Easy enough if the intercept is good,” Beck said. “I just finished attaching all the tethers we have into one long line. It’s two hundred and fourteen meters long. I’ll have the MMU pack on, so moving around will be easy. I can get going up to around ten meters per second safely. Any more, and I risk breaking the tether if I can’t stop in time.”
“Once you get to Mark, how fast a relative velocity can you handle?”
“I can grab the MAV easily at five meters per second. Ten meters per second is kind of like jumping onto a moving train. Anything more than that and I might miss.”
“So, including the MMU safe speed, we need to get the ship within twenty meters per second of his velocity.”
“And the intercept has to be within two hundred and fourteen meters,” Beck said. “Pretty narrow margin of error.”
“We’ve got a lot of leeway,” Lewis said. “The launch will be fifty-two minutes before the intercept, and it takes twelve minutes. As soon as Mark’s S2 engine cuts out, we’ll know our intercept point and velocity. If we don’t like it, we’ll have forty minutes to correct. Our engine’s two millimeters per second may not seem like much, but in forty minutes it can move us up to 5.7 kilometers.”
As it approaches, they discuss it more and more but ultimately there's not much more they can do. Then:
Directly in his field of view, the Hab canvas patch flapped violently as the ship exponentially gained speed. Concentration became difficult, but something in the back of his mind told him that flapping was bad.
The remotely controlled MAV begins to have difficulty being controlled because of the loose canvas.
“VELOCITY SEVEN hundred and forty-one meters per second,” Johanssen called out. “Altitude thirteen hundred and fifty meters.”
“Copy,” Martinez said.
“That’s low,” Lewis said. “Too low.”
“I know,” Martinez said. “It’s sluggish; fighting me. What the fuck is going on?”
“Velocity eight hundred and fifty, altitude eighteen hundred and forty-three,” Johanssen said.
“I’m not getting the power I need!” Martinez said.
“Engine power at a hundred percent,” Johanssen said.
“I’m telling you it’s sluggish,” Martinez insisted.
The canvas finally rips away and the controls stabilize.
“I’M GETTING more response now,” Martinez said.
“Back on track with full acceleration,” Johanssen said. “Must have been drag. MAV’s out of the atmosphere now.”
“It was like flying a cow,” Martinez grumbled, his hands racing over his controls.
“Can you get him up?” Lewis asked.
“He’ll get to orbit,” Johanssen said, “but the intercept course may be compromised.”
“Get him up first,” Lewis said. “Then we’ll worry about intercept.”
“Copy. Main engine cutoff in fifteen seconds.”
“Totally smooth now,” Martinez said. “It’s not fighting me at all anymore.”
“Well below target altitude,” Johanssen said. “Velocity is good.”
“How far below?” Lewis said.
“Can’t say for sure,” Johanssen said. “All I have is accelerometer data. We’ll need radar pings at intervals to work out his true final orbit.”
“We’re working on getting you,” Lewis said. “There was a complication in the launch.”
“Yeah,” Watney said, looking out the hole in the ship. “The canvas didn’t hold. I think it ripped early in the ascent.”
So in the book at least, this was discussed back at NASA at the beginning. They knew it was a Hail Mary Pass as I understand the term. They don't super work out the details so much as "This is how it's going to have to happen, lets get them moving in the direction they need to go and in the meantime we can figure out how we're going to get this to work."
In the end with all the MAV modifications, the tarp tore loose screwing up the aerodynamics significantly such that it wasn't able to attain the velocity necessary to get to the higher orbit that was necessary.
Going to stop adding more quotes for fear of copyright issues at this point :/