The major plot of Star Wars: Rebels gives us a really good insight into the 'behind the scenes' working of the Empire.
There are multiple examples of unsustainable practices such as strip-mining and the removal of ores and minerals from planets on the outer rim, away from the prying eyes of pesky senators. In the episode "Homecoming" we learn that they've basically turned the planet Ryloth from a paradise into a starving semi-wasteland dustbowl.
Corruption / Unfavourable Supply Arrangments / Taxation
With the senate on its knees, local governors are free to make deals that are personally favourable (offering their planet's labour to build TIE-Fighters, for example) even when it's not in the planet's best interests to do so. On top of that, the Empire seems to be top-slicing every single deal that takes place and funneling that money into building the apparatus of a police state.
It's pretty clear that the Empire is built on a foundation of slavery. The Twi'leks and the Wookiees are especially well known for their forced labour being used, as sexual slaves and as construction workers respectively (don't get them mixed up).
As you've said, the Empire does a pretty fair trade with the Hutts. Hutt space is a major provider of slaves, vice goods and raw materials looted from worlds without advanced defence systems. There's also a fair amount of indentured/slave mining operations.