EDIT: In a copy of the script I found Jellico says:
Oh, by the way...
I prefer a certain... formality on
(beat, then with a
I would appreciate it if you wore
a standard uniform when you're on
My reading of this, is that Troi wearing a standard uniform is not a regulation or rule, but simply a preference of Jellico's. Troi doesn't protest either because Jellico's the captain or because she does not want to make the transition to a new captain any harder.
The reason out-of-universe is because it made the character more interesting, in-universe it may have just been that Troi had a desire to be taken more seriously as an officer. This seems like a situation where the writers attempted to merge the in-universe and out-of-universe reasons, to create a holistic plot solution, if you will.
Troi appears to be shedding a more feminine or motherly appearance for a more commanding professional appearance. What this says about Federation culture is difficult to speculate about.
Here's my old answer, I still feel like the scene seems pretty weird and oddly pro- and anti-feminist as far as the Star Trek universe goes.
This is the kind of question that writers try to avoid coming into their viewers minds, because there's no-way to arrive at any good in-universe explanation. It's just poor writing and character planning, plain and simple.
Why would such a progressive and large institution as Star Fleet have dress-code issues on their flag-ship centuries after the founding of Star Fleet?
Troi either would have been corrected on the first day or never been asked to change, but not one and then the other. The ornery evil captain that caused the change probably wouldn't have said anything if she was in compliance with the dress code to begin with.
Why have a dress-code if you're going to allow the leadership to ignore it?
Wouldn't that cause problems in any institution?