This might be the planet Mesklin (en.wikipedia article, contains spoilers) in one of Hal Clement's novels. The planet first appears in Mission of Gravity (1953), then in other novels by Hal Clement.
(This one paragraph is an update.) You can find almost all the details you asked about from the very first chapter of Mission of Gravity, so it is possible that what you remember is based on that.
Mesklin is a world with very high gravity. It is inhabited by the Mesklinites, an intelligent species that looks a bit like worms, and are adapted to the high gravity.
The Mesklinites never go far from the ground, have a fear of objects falling on them, and build only single-level buildings with very light roofs. At least, this is the situation near the poles. The gravity is much lighter near the equator, where gravity is light enough that Mesklinites can actually fly with the aid of gliders.
The inhabitants have science, and believe they live on the inside surface of a bowl (half of an oblate spheroid). That is, until one day, when a spaceship of humans arrives from outside Mesklin. The Mesklinites learn about the outside world for the first time, but what's more disturbing though is what they learn of their own world.
Mesklinites are living on the outside of a flattened sphere, not the inside. I hide the explanation for their false belief below, which is revealed in chapter 15.
Mesklin is actually a super-dense giant planet. The atmosphere of the planet is dense, but concentrated very close to the surface, and gets less dense as you go up. This bends light in such a way that the ground appears to curve upwards. The inside of a sphere was a good enough approximation for all purposes, because they never go far from the surface of the planet.
There are indeed oceans, and the Mesklinites can travel them on ships, though they rarely do so.
I don't remember anything about digging or what they believe about the Sun, but I have only read Mission of Gravity.
The plot of that novel is the following. The spaceship of humans had arrived (a little before the start of the novel). The humans are co-operating with the Mesklinites to recover a human rocket that has crashed on the planet.