There's two (related) questions here. I'll try to answer both;
Does Harry need to carry on wearing the armband?
No, absolutely not. The armband seems to be needed to prevent the user from dying while crossing between the two quantum states, not to continue living on the other side. Think of it as a key rather than a scuba suit.
JANEWAY 2: Rig another one, Ensign. I'm going with her.
KIM 2: This should protect you from the spatial transition.
Note that once he's on the other side, he's freely able to walk around without the armband, as we see in the final scene of the same episode:
Is he out of phase forever?
Yes. Although it's not made clear in the show, he and the baby never return to phase. This is referenced in the follow up novel trilogy "String Theory", set between Seasons 3 and 4:
“Without more data as to the source of the altered engrams, it would
be difficult,” the Doctor said simply. “There are many differences
between your species which could account for… What is this?” the
Doctor interrupted himself.
“There is a subtle phase variation in the molecular structure of Naomi
Wildman which she does not share with you, her mother, or Mr. Neelix.
Can you account for this?”
A theory was formulating in Janeway’s mind and the Doctor’s discovery
crystallized it. “Almost two years ago, the day Naomi was born, in
fact, our ship encountered a subspace scission and every particle of
matter on board was duplicated, including the crew. The Naomi who was
born on this ship did not survive, but the duplicate child did, and
before the other ship was destroyed, she was brought on board.”