The manga is about a magician who gets reincarnated as a noble but gets abandoned by his father. he was later adopted by a goblin and he turns his village into a monster kingdom.
Suterareta Tensei Kenja - Mamono no Mori de Saikyou no Daima Teikoku o Tsukuriageru aka The Abandoned Reincarnated Wise Man: Creating the Strongest Empire in the Forest of Demons
A man once called a great wise man was reincarnated just before his death, retained his memories because of the magic he applied to himself. But at times he was born, magic was unnecessary; therefore, the baby was abandoned in the demon forest. He was worried that he would eventually die, but he was lucky enough to be picked up by a gentle goblin and raised at a demon village. The matured wise man tries to develop the village to pay back to the demons, but things start to happen. One who was praised as a wise man leads demons to create a great empire!?
Found with a search for
manga magician "adopted * goblin"