We're not told when and why, but we can make a guess.
It's possible he wanted to give his seat to his son, Jorah, and handed his blade Longclaw to him before his departure. After Jorah shamed himself and his house, he sent Longclaw back to his father in the Night's Watch.
“My son brought dishonor to House Mormont, but at least he had the grace to leave the sword behind when he fled. My sister returned it to my keeping, but the very sight of it reminded me of Jorah’s shame”
A Game of Thrones - Jon VIII
Jeor was evidently coming of age (as noted below) and decided he would bring his House honour by "taking the black"
Father asked if there were any knights in the hall who would do honor to their houses by taking the black
A Game of Thrones - Sansa III
It's not known, but it is known he arrived before Alliser Thorne did, who was sent there at the end of Robert's Rebellion. He is said to have been Lord Commander of the Night's Watch before Alliser's arrival.
"The Watch has no shortage of stableboys," Lord Mormont grumbled. "That seems to be all they send us these days. Stableboys and sneak thieves and rapers. Ser Alliser is an anointed knight, one of the few to take the black since I have been Lord Commander. He fought bravely at King's Landing."
A Game of Thrones - Tyrion III
But they've been contradicted in more recent canon and semi-canon sources:
"You were just a boy, and I was all in black, one of a dozen riding escort to old Lord Commander Qorgyle when he came down to see your father at Winterfell. I was walking the wall around the yard when I came on you and your brother Robb. It had snowed the night before, and the two of you had built a great mountain above the gate and were waiting for someone likely to pass underneath."
A Storm of Swords - Jon I
From the above, it's worth noting that Jon was born at the end of Robert's Rebellion and was not brought back to Winterfell till after the Rebellion.
So we know he was in the Night's Watch from at least around 14 years before the events of ASOS, and we know he'd left after Jorah was wed and before the Greyjoy Rebellion.
Ten years we were wed, or near enough as makes no matter. She was a plain-faced woman, but not unkind. [...[
Ser Jorah nodded. “By then my father had taken the black, so I was Lord of Bear Island in my own right.
A Clash of Kings - Daenerys I