This is The Cave King Will Live a Paradise Life.
From MangaUpdates:
Hiel was ridiculed for having a useless crest and was exiled to a remote island. When he grabbed an ice axe to reclaim the island, the true effect of his crest was revealed. With his production cheat he will reclaim this uninhabited island and start his life in paradise.
The MC is a prince, and on his fifteenth birthday he is deemed to have become an adult and granted his own 'dominion', which happens to be a remote, rocky island. He travels there alone by boat, with instructions from the king to 'heal' it.
Everyone in the kingdom the MC came from is born with a crest on their body, and each crest signifies a different, magical power. In the case of the MC, his crest and power were unknown, and he was consequently looked down upon by his peers. As a result, he views the assignment he's been given as a form of exile.
Soon after arriving on the island, he finds a cave with several human-like skeletons inside it. When he pulls out a small pickaxe from the supplies he brought with him, he hears a voice in his head informing him that he's acquired a mining tool. He soon figures out that he has the crest of a "Cave King", that comes with a built in advisor function which he's now activated for the first time.
The Cave King skill allows him to dig huge tunnels at great speed, and he has night vision and can see valuable ores in the rocks, which he can store an unlimited amount of in an inventory. Also, the more he digs, he more his abilities level up, and he discovers magical rocks and crystals with specific effects, like one that extends your lifespan by a day.
After digging deep enough, he discovers slimes, and later a group of goblins, including a small goblin princess suffering from a curse. He uses some of his magic stones to lift her curse and heal her, and in gratitude, the goblins become his helpers.
He later finds another magic stone which has the ability to evolve monsters. The goblin princess says she wants to evolve, and when he gives her the stone, she transforms into a young, elf-like woman. She also reveals that her previous body was stuck with a child-like appearance as the result of another curse, so she's pleased with her new look.
The MC subsequently discovers that he can combine rocks and sand to create gravel, and produce cuboid blocks with which he can build structures. Also, he and the goblins are attacked by killer birds at one point, and he creates a large stone golem to protect them.
This technically isn't an isekai manga, since the MC never leaves his original world, he just moves to a new environment within the same world. But that aside, this manga seems to have everything else you mentioned.
Here's a page where the MC's night vision is mentioned.
And here's a page where they talk about evolving one of the goblins.
