I reviewed Prisoner of Azkaban, Goblet of Fire, Order of the Phoenix, and Half-Blood Prince. Here's what seems to be going on.
I think Divination was compulsory through fifth year, until students were post-O.W.L.s¹. In Prisoner of Azkaban, Hermione is allowed to drop Divination halfway through the school year. Just as an exception was made to allow Hermione to use a Time Turner to attend ten classes, as I looked through the available info, it seems another exception was made and Hermione was allowed to drop a compulsory class. It would seem certain lessons were compulsory during years three, four, and five, but in order to move on to the N.E.W.T. level courses, minimum grades had to be achieved.
distribution of timetables was more complicated than usual
this year, for Professor McGonagall needed first to confirm
that everybody had achieved the necessary O.W.L. grades to
continue with their chosen N.E.W.T.s.
Half-Blood Prince - page 165 - Bloomsbury - chapter 9, The Half-blood Prince
If Harry could have dropped Divination during his third year, he surely would have -- he hated Divination with the passion of a thousand fiery suns! So this may be another indicator that Divination was a compulsory class.
I think the main point of J.K. Rowling purposefully showing us a lot of scenes of Professor Trelawney and Divination, and Harry's reaction to Trelawney and all that she represents was set up a guise that Trelawney was a fraud, inept, ridiculous, and ditzy. So when it's revealed that it was Sybill Trelawney who made the prophecy that the entire series is built around, it's a bit of a surprise and a Whoa! moment to both the reader and Harry.
A figure rose out of it, draped in shawls, her eyes magnified to
enormous size behind her glasses, and she revolved slowly, her feet
in the basin. But when Sybill Trelawney spoke, it was not in her
usual ethereal, mystic voice, but in the harsh, hoarse tones Harry
had heard her use once before:
‘The one with the power to vanquish the Dark Lord approaches ...
born to those who have thrice defied him, born as the seventh month
dies ... and the Dark Lord will mark him as his equal, but he will have
power the Dark Lord knows not ... and either must die at the hand of
the other for neither can live while the other survives ... the one with
the power to vanquish the Dark Lord will be born as the seventh month
Order of the Phoenix - page 741 - Bloomsbury - chapter 37, The Lost Prophecy
I think if all the pieces are put together, Harry took Divination in year four because it was 1) compulsory and 2) an important part of the build-up to the revelation of Trelawney's prophecy.
ETA: ¹At the end of their second year at Hogwarts, students are required to choose a minimum of two more subjects from the following list: Arithmancy, Muggle Studies, Divination, Study of Ancient Runes and Care of Magical Creatures. POTTERMORE - BOOK 1 - CHAPTER 6 - *Talons and Tea Leaves (screenshot).
So, when Harry picked Divination as one of his two elective classes, Divination became compulsory for him through the end of book five. As this is when the prophecy is revealed, and Harry earned a "P" (Poor) on his Divination O.W.L., there is no need for J.K. Rowling to continue showing the same scene of Harry hating Divination to the reader. McGonagall would never have let Harry into N.E.W.T.-level Divination anyway, even if -- for some reason that I can't even begin to fathom -- Harry had wanted to continue with Divination.
So, a small distinction, but one nonetheless: Divination was compulsory if it was a chosen elective. This is consistent with my existing answer, so I'll leave it as is.