The short story you are looking for is Mountain, written by Liu Cixin and published in Apex Magazine in September 2015. (it may have been published elsewhere before in Apex Magazine, so still could fit the 2012 - 2014 frame).
This story seems to fit all the criteria:
If I remember correctly, the above describes a narration by one character describing his civilization's beginnings to another.
Two lines of text appeared on the screen. They contained all the alphabets and characters Fan had ever seen. Recognizing the words for “English”, “Chinese”, and “Japanese”, he surmised that they must spell out the names of all the world’s languages. He also spotted a dark frame quickly moving between the different words. It all appeared rather familiar. His guess was soon proven right, as he discovered that this frame actually did follow his gaze! He fixed his eyes on the characters for “Chinese”, causing the dark frame to stop over them. He blinked once, but there was no response.
Maybe it needed a double-click, Fan thought and blinked twice. The dark frame flickered and the giant sphere’s language menu closed. In its stead, a huge word appeared in Chinese.
Hello!
“Hello!!” Fan shouted his response into the sky. “Can you hear me?”
We can hear you; there is no need to shout. We could hear the wings of a mosquito anywhere on Earth. We picked up the electromagnetic waves leaking from your planet and so learned your languages. We want to have a little chat with you, the text on the sphere now read.
The story contains a civilization which forms under the surface of a planet (hollow core type) that, over eons, discovers it is under the surface of a planet and then tries to send expeditions to try and uncover the outer surface of the planet.
Our world is a very simple place. It is a spherical space, somewhat more than 3,500 miles in diameter, according to your units of measurement. This space is completely surrounded by layers of rock. No matter what direction one chooses to travel, the journey will always end with a solid wall of rock.
Naturally, this shaped our first model of the cosmos: We assumed that the universe was made of two parts. The first was the 3,500-mile space in which we lived; the second was the surrounding layers of rock. We believed the rock to stretch endlessly in all directions. Therefore, we saw our world as a hollow bubble in this sold universe and so we gave our world the name “Bubble World”. We call this cosmology the Solid Universe Theory. Of course, this theory did not deny the possibility of other bubbles existing in these infinite layers of rock. However, it gave no indication how close or far those other bubbles might be. That became the impetus for our later journeys of exploration.
The problem with these expeditions is that each expedition takes a pocket of air with it and this reduces the civilization's living space in case these are lost. The last viable expedition seems to reach less dense strata but I don’t remember what happens next.
Our first journeys of exploration into the rock layers began in earliest antiquity. Exploration for us meant tunneling into the walls in an attempt to find other bubbles in our solid universe. We had spun many fascinatingly alluring myths around these distant spaces and almost all of our literature dealt with the fantasy of other bubbles. Soon, however, exploration became outlawed, forbidden on pain of death by short-circuiting.
“Outlawed? By your church?” Fan assumed.
No, we have no church. A civilization that cannot see the sun and stars will be without religion. There was a very practical reason for our senate to forbid tunneling: We were not blessed with the near infinite space you have at your disposal. Our existence was limited to that 3,500-mile bubble. All the debris that the tunneling produced ended up within this space. As we believed in infinite layers of rock stretching in all directions, those tunnels could have become very long indeed; long enough even to fill the entire bubble space at the core of our would with rubble! To put it another way: We would have transformed the empty sphere in the core of our world into a very long tunnel.
Read the story again to find out the ending for yourself!
Note: Samveen (the question asker) provided me with the name of the story after I had recognised it and summarised it in the comments. A lot of credit to him.
