In the films, both Anakin and Luke want to leave Tatooine, which indicates how harsh life was on the dusty ball.
Moisture farming takes place against a background of slavery, illegal smuggling, gangsterism, gambling and poverty. Their nearest neighbors are hostile aborigines and the climate is unpleasantly hot.
If the planet is as bad as it seems, why don't the moisture farmers simply leave Tatooine and go somewhere else?
Firstly, the Moisture Farming was not just a self-sustaining activity. It was done for the entire population of Tatooine. The Lars family were just one farm in the system, not doing it for their own sustenance. Farming is hard work, but it's a living.
However, that is not the reason Tatooine was initially populated. People came to mine it's plentiful resources, though that turned out to eventually be a bit of a dead end. Even though this venture was a non-starter, as with any industry, there is always a need to set up support for the industry, which drew in other industries to provide support, food, and any other things required to maintain the industry, and thus, a population is born.
However, now that the mining industry is a bust and the mob is out in force, the only reason to stay is basically just because no one can really ever afford to leave. Tatooine is all but ignored by the Galactic Federation, so the law is not really enforced on this planet. The only reason the Galactic Federation came there at all is because they were chasing droids. So the only way off the planet is to buy your way off, and that isn't cheap*.
HAN: Well, that's the trick, isn't it? And it's going to cost you something extra. Ten thousand in advance.
LUKE: Ten thousand? We could almost buy our own ship for that!
I couldn't say much for the income of the lowly moisture farmers, but from context, they can't afford much outside of their own equipment to run the farm, and keep themselves fed.
* This does come on the coattails of the potential of avoiding the Feds, but even then, it does require a lot to fuel a spacecraft
