Relating specifically to the settlement in the episode, there are a few factors involved.
The walls are likely there to keep out the native fauna. We see near the start of the episode, when the Mandalorian lands on the planet, that there are large creatures in the distance munching on trees. Tall, solidly built walls would work to keep these creatures from stomping through the village and damaging property (whether intentionally, or just by accident).
The people of the town are effectively enslaved by the Magistrate as part of some unspecified industrial work, which is the reason that the forest is completely burnt out. This could provide a justification for the lack of advanced technology and general squalor - giving the people access to anything too advanced could lead to an uprising, and an oppressor willing to cage dissidents is unlikely to expend resources on improving the lives of their workers.
Also, some people in Star Wars are just poor. We saw the small mining town in the first episode of this season - those people don't have a lot of money, so they don't have a lot of access to the best technology. Even on Coruscant, the capital of the Republic, a huge number of people were living in dirty, dangerous districts analogous to slums below the surface. The shiny part we see in the films is just where the rich people live - the planet has a trillion people, most of whom are packed into tiny box-like apartments and rarely if ever see the sky at all. While they might have holograms and flying cars, for most of the citizenry, Star Wars is more Cyberpunk than futuristic utopia.