TL;DR: We don't know but less than the full 100,000 are left.
Some wildlings were killed when the Nights Watch ranged north of the wall and some were killed when the wildlings attacked Castle Black.
Not all of the wildlings came to Hardhome in the first place.
That's it, come on.
How many are with us?
I'm not good at counting.
We're leaving too many behind.
The free folk are stubborn.
You know how long it took Mance to band them together? And he knew them better than I ever will.
They're running out of food and there's nothing to hunt.
They'll come around.
Game of Thrones, S05 E08, "Hardhome"
It's reasonable to assume more came but not all of them. Out of those that came not all made it either, as we get the iconic Night King raising the dead scene.
When the wildlings are let through the gates they then go straight through the gates and free into the north:
It's safe to assume that at least some of these never returned to help Jon in his next battles. As Jon himself states that the wildlings can occupy the Gift:
Look, well, there is good farmland in the Gift. Land that no one uses now. A dozen abandoned villages.
Game of Thrones, S05 E05, "Kill the Boy"
At the Battle of the Bastards a lot of wildlings are again killed. As you can see from this picture of the Vale saving the day.
Also in the full scene you see body piles forming, obviously not all are wildlings but a lot of them are.
We also know that not all of the wildlings are shown on screen at any one time so it's hard to guess at those left. As far as I know we are also never given numbers.
A handful also died in Season 7 Episode 6, "Beyond the Wall", when the merry men set off to capture a wight.
We know there are still some wildlings left as they are stationed at Eastwatch in Season 7. Though in the finale some died here as well.
So in conclusion we don't know but less than the full amount of Mance's army.