I thought it was a good movie, yet I'm always hearing people complain about how the plot was terrible and so on. What critical reactions were there to the film and which elements failed in the eyes of the Star Trek community?
Rand al'thor has done a great job of collecting reviews on the issue of The Motion Picture. However, I thought you might also appreciate the input of a die-hard Trek fan. While critics deplored the plodding pace of the film, Star Trek fans are no stranger to episodes with a contemplative pace (even if TMP is uniquely slow amongst Trek films).
Going from The Original Series to The Motion Picture, here are some of the biggest issues that Trekkies must contend with:
The core friendship was off-balance. The core of the original Trek was arguably the friendship between Kirk, Bones, and Spock. In TMP, this is broken. McCoy offers his usual criticism of Kirk's decisions, but not in a friendly way. Spock, having gone through Kolinahr, is now taciturn and offers only brief dialogue through the entire film.
Spock was not Spock. Building on the previous point, Spock was arguably the most popular character in TOS. Despite his purported lack of emotion, he would often engage in a kind of spirited and witty banter with the others. Here, he was literally a shell of himself. Even if it was part of the plot, it didn't help to sell the film to its audience, old or new.
Kirk was a buffoon. Shatner's acting aside, the Kirk of TMP was incompetent. He had none of the leadership skills that made Kirk who he was in TOS. This was a major stumbling block for fans. We Trekkies had never seen a Kirk before who had almost destroyed the ship from sheer incompetence. It just doesn't compute.
There was no real villain, and the characters were oddly passive aggressive. TOS episodes had some marvellous villains, who the crew dispensed with while maintaining their poise. On the other hand, this film had a strange sort of pent-up rage in many of its characters — Kirk, McCoy, Dekker — but no one to use it against. The film is a ship of passive aggressive Starfleet officers flying through a never-ending screensaver, and there is no villain whose presence causes their humanity to shine. They are as mechanical as V'Ger.
Regarding the last point, the lack of a strong villain to counter Kirk and the others was what led to producer Harve Bennett and director Nicholas Meyer scouring TOS for a formidable villain to use for Star Trek II. (They ended up making an excellent decision, of course, by choosing Khan. The rest is history.)
Wikipedia has a section on "Critical reception". Quoting the negative parts:
Gary Arnold and Judith Martin of The Washington Post felt that the plot was too thin to support the length of the film.
Time's Harold Livingston wrote that the film consisted of spaceships that "take an unconscionable amount of time to get anywhere, and nothing of dramatic or human interest happens along the way". Livingston also lamented the lack of "boldly characterized" antagonists and battle scenes that made Star Wars fun; instead, viewers were presented with lots of talk, "much of it in impenetrable spaceflight jargon".
David Denby said that the slow movement of ships through space was "no longer surprising and elegant" after films such as 2001: A Space Odyssey, and that much of the action consisted of the crew's reacting to things occurring on the viewscreen, which the New York Magazine critic considered to be "like watching someone else watch television".
Arnold felt that the acting of the main cast (Shatner in particular) was poor; "Shatner portrays Kirk as such a supercilious old twit that one rather wishes he'd been left behind that desk", he wrote. "Shatner has perhaps the least impressive movie physique since Rod Steiger, and his acting style has begun to recall the worst of Richard Burton."
Many critics felt that the special effects overshadowed other elements of the film. Canby stated that the film "owes more to [Trumbull, Dykstra and Michelson] than it does to the director, the writers or even the producer". Livingston felt that Trumbull and Dykstra's work on the film was not as impressive as on Star Wars and Close Encounters due to the limited amount of production time. Godfrey called the effects "stunning", but conceded that they threatened to overpower the story two-thirds of the way into the film. Kroll, Martin, and Arnold agreed that the effects were not able to carry the film or gloss over its other deficiencies; "I'm not sure that Trumbull & Co. have succeeded in pulling the philosophic chestnuts of Roddenberry and his co-writers out of the fire," Arnold wrote.
James Berardinelli, reviewing the film in 1996, felt that the pace dragged and the plot bore too close a resemblance to the original series episode "The Changeling".
The slow pacing, extended reaction shots, and the film's lack of action scenes led fans and critics to give the film a variety of nicknames, including The Motionless Picture, The Slow Motion Picture, The Motion Sickness, and Where Nomad [the probe in "The Changeling"] Has Gone Before.
All of this, naturally, is well referenced; there are plenty of citations you can go and look at if you want more detail.
In short, the main criticism is that everything was too slow.
A few other things that Memory Alpha has to say on the movie's reception:
Shatner thought it was the end: Shatner commented:
"Well, that's it. We gave it our best shot, it wasn't good, and it will never happen again." But having recalled his reaction fifteen years later, he has added, "Shows you what I know." (Star Trek Movie Memories, 1995, p. 124)
Though eagerly awaited, Star Trek fans were by and large in agreement with Nimoy's assessment at the time, especially where the lumbering pace of the movie was concerned, and endowed the movie with humorous, if unflattering, sobriquets such as "Star Trek: The Motion Sickness", "Star Trek: The Motionless Picture", or "Star Trek: The Slow-Motion Picture". (The World of Star Trek)
Fans also thought it was too much like TOS: 'The Changeling':
Another sobriquet given to the movie was "Where Nomad Has Gone Before", which reflected the criticism that the story was too reminiscent of several Original Series episodes, first and foremost the second season episode "The Changeling", in which the sentient robot Nomad was featured