For those of you unfamiliar with this minor celebrity vegetable, here's a picture of it:
As far as I know, the Fifth Doctor wore it for his entire run, but we didn't get anything resembling an in-universe explanation for it until his very last episode, The Caves of Androzani.
PERI: Doctor, why do you wear a stick of celery in your lapel?
DOCTOR: Does it offend you?
PERI: No, I'm just curious.
DOCTOR: Safety precaution. I'm allergic to certain gases in the praxis range of the spectrum.
PERI: How does the celery help?
DOCTOR: If the gas is present, the celery turns purple.
PERI: And then what do you do?
DOCTOR: I eat the celery. If nothing else, I'm sure it's good for my teeth.
Sadly, the celery never did turn purple, so we never found out if it was legitimately useful.
But considering it was around for a very long time before the writers felt the need to acknowledge it, its presence probably has more to do with making the costume look ever-so-slightly abnormal rather than any in-universe benefits. Which begs the question: Is there any particular reason they chose celery as the random weird thing to put on Peter Davison's costume?