Do we know if it is indeed naturally curly, or is it straight with a perm(anent?
Though I don't think there's an ironclad proof one way or another (unless it comes from one of the books), I think the evidence suggests it is naturally curly but that he has it straightened.
We know from Schisms that Worf makes regular trips to the barbershop, at least while serving on the Enterprise. While only the length of his hair is addressed in the episode, knowing that he has it cut (and that he cares about its appearance, warning Mot not to take off too much again) makes it extremely plausible that he would straighten or curl it too.
The reason I believe it is naturally curly and he straightens it, as opposed to the other way around, is when we see him with curly hair. I believe the top right image in your question is the best indicator, because this is a screenshot from Insurrection, in which Worf (and others) are experiencing the rejuvenating effects of the planet within the Briar Patch. (He had to deal with a Klingon pimple in an earlier scene.)
Certainly, we see Worf with both straight and curly hair in situations where he has access to 24th century comforts, but this is to be expected. (People change their hairstyles periodically, and Worf in particular struggles with his identity as a Klingon in a human environment throughout the franchise.) But I don't know of a time when he was without the comforts of the Federation and still has straight hair. (The exception being that if he goes into a situation with a straightened pony tail, it appears to stay that way if he doesn't touch it.)
I agree with @Josh that Worf's hair may be naturally curly if we look at the family lineage.
Mogh (Worf's father)
Seems quite curly (or atleast fuzzy).
Kurn (Worf's brother)
Young Alexander (Worf's son)
During his time on the Enterprise, he may have also visited the barber quite often like his father, hence the straight hair.
Older Alexander (Worf's son)
After leaving the Enterprise, he not have had time (or perhaps no longer wanted) to style his hair anymore. So ends up looking fairly curly.
I have a theory.
Keratin is a protein that is a major building block in hair and fingernails, and it is also used to straighten hair when added in salon treatment processes.
Keratin is one of a family of fibrous structural proteins. Keratin is the protein that protects epithelial cells from damage or stress that has potential to kill the cell.
If the rejuvenation planet is constantly refreshing cells, then the body may have no need for keratin and might halt production of it, thereby resulting in curlier or wavier hair for those under its effects.