The position, not the instructor, is cursed.
It means that next year they'll have to find someone new, for whatever reason. So, I think events transpired in a way that made it necessary to find a new instructor.
Obviously, Crouch couldn't be it. Him being an impersonator in that position to begin with seems like a result of the curse.
Moody not being it the following year, due to prior plans or due to or a result of everything else, wouldn't be there or wouldn't have stayed anyway. (I personally believe that once Voldemort returned, there'd be no chance at all of Moody sticking around to teach anything.)
I think all these events are quite intertwined with one another, and the curse, and you couldn't really separate out a single victim. As I said, the position is cursed.
And while it's generally revealed to us at the end of the books why the DADA instructor can't stick around, those circumstances are generally set in motion long before then. Quirrel wasn't going to last long with Voldemort in him. Dumbledore suspected Lockhart was a fraud from the get go. Crouch could have kept his cool after the Cup incident, leaving none the wiser, and better served his Dark Lord by killing Moody and impersonating the next DADA instructor. Instead, he went all fanboy and gloating.
So, it's entirely possible that Moody's initial one-year plan was influenced by the curse, and then when he was removed a new set of limitations were created to affect Crouch. (Even if it wasn't the real Moody to be offered the position to begin with, we'd see that there'd need to be some restructuring for the following year.)