Maege Mormont has following daughters:
- Dacey Mormont
- Alysane Mormont
- Lyra Mormont
- Jorelle Mormont
- Lyanna Mormont
Dacey was murdered at Red Wedding. Alysane apparently came back to Bear Islands because she was part of Stannis's liberation of Deepwood Motte. Lyra Mormont was last with her Lady Mother who went on Robb's orders to find Lord Reed, they have disappeared. Jorelle is also with them.
Which leaves Lyanna the Mormont in Bear Island as Alysane is on the march with Stannis. She was on Bear Island because she was not with her Mother in Riverlands, and because she replied to Stannis' summons in ADWD:
Bear Island knows no king but the King in the North, whose name is
But since she is the youngest daughter and second daughter of Lady Maege is alive and free, it appears she is filling the place of Alysane in books, rather than being in charge in her own right.
You must keep it in mind that when Lyanna replied Stannis, it was before He moved against Ironborn. Only explanation for that can be that Alysane must be away.
Afterwards when Stannis moved against the Ironborn, It was Alysane who lead the surprise Mormont attack on Ironborn and later swore fealty to Stannis on behalf of her house.
In any case, Alysane is the heir, not Lyanna.
In the show, No other Children of Lady Maege Mormont are mentioned and Lyanna is mentioned as Lady of Bear Island. So it appears, in show canon, Lyanna is the only Child of Lady Maege and thus inherited the Island on her Mother's demise.
Maege's death is not shown on the Show but it is implied because in Season 5, Stannis refers to Lyanna as Lady of Bear Island.
As pointed out by BCdotWEB, Lyanna said this in Episode "Broken Man" to confirm her mother's death:
"My mother wasn't a great beauty or any other kind of beauty. She was
a great warrior, though. She died fighting for your brother, Robb."