I remember this scene from one of the original trilogy movies. I also remember the droid was "screaming" as it was tortured.
Why would someone torture a droid? How could it be effective? It's a Robot!
TL;DR See at the bottom of this post after both pictures.
The reason is two-fold. First of all, what works for humans also seems to work for (sentient) droids. 3PO is clearly shocked by the torturing, motivating him to do good work for Jabba (which he clearly despises). The picture I took was the script version of this shot from Return of the Jedi:
Where a droid that very likely was only designed to feel pain, is tortured by the assistant droid 8D8.
The true reason for this whole mess is a certain droid, EV-9D9, that started its existence in Cloud City. It later worked for Jabba as some sort of administrator for his droids. I cannot describe it better than wookiepedia, so here's the important excerpt:
She [EV-9D9] kept a laboratory deep in the bowels of Jabba's Palace, where she built grotesque droids from extra parts that existed only to feel pain. During her years spent there she added several upgrades to herself so that she could feel more of the droid equivalent of pleasure while watching the coolant from dismembered droids pour out of them while using her droid assistant 8D8 to burn the helpless victims.
... and here's the handsome lady:
So to summarise, the reason EV-9D9 tortures droids is because she genuinely enjoys it. And the reason for Jabba to allow and encourage this is because it works a "great" way to motivate his droid-servants.
I would put 2 reasons why robots were tortured in SW:
The fact that robots were tortured to pleased Jabba (or who ever ordered them to be tortured). He was just a bad "guy" and he just liked to see people, robots, creatures or whatever suffer simply because he ordered it to be so.
From SW it is obvious that robots have some sense of self preservation (R2D2 was hiding in a cave when the sand people came, C-3PO was actually scared all the time). So let’s say that you are the programmer and you put some self-preservation mechanism into a robot to make him "live" longer. There are also other protocols which the robot has to obey and due to that it sometimes doesn't do what you want. So maybe the torture was just a way of testing which protocol has the higher priority :). Just to test if you can actually override some lower priority protocol.
The great answer posted by "bitmask" and revealed in the wookiepedia is originally conceived in the short story, "A Bad Feeling: The Tale of EV-9D9" by Judith and Garfield Reeves-Stephens. It's one of a collection of short stories in the Star Wars anthology, "Tales from Jabba's Palace" by Kevin J. Anderson for Bantam books (1994). This book is part of the great collection of Star Wars books written before the prequel trilogy (and later, by corporate executive fiat, reduced to non-canon status after George Lucas sold the franchise to the Disney Corporation).
The EV-9D9 droid was torturing others for sadistic pleasure. "She" was a droid that was unique in manufacture, a freak of a mistake in construction that allowed her to experience pleasure from others' pains. This following large sample, excerpted from a pdf form of the story, shows how "Ninedenine" experienced the torture.
Given a year or two or five or ten within this dark domain, even the best of them [prisoners in Jabba's Palace] would succumb to ravaging their own tentacles or clawing at their own light sensors.
To Ninedenine, such actions were the addictively elegant expressions of a higher-dimensional logic pathway which only she among droids had the gift to comprehend--first by an accident of manufacture, it had seemed, but now augmented by her own deliberate and ongoing modifications. To organics, such acts of self-inflicted, physical alteration were second nature, a state which Ninedenine yearned to achieve and often felt maddeningly close to experiencing. Indeed, there was much within the organic mind which Ninedenine felt certain was comparable to her own. Not in quality of intellect--she was positive she had no equal among cellular-based processors in that regard.
But in appreciation of sensation--that was how Ninedenine preferred to characterize her avocation.
The savoring of the sine waves of discomfort. Plunging into the algorithms of despair. Racing through the oscillating peaks and valleys emitted by circuits strained far past their design and logic loads. True, for now, her internal receptors allowed her only the binary nature of droids to work with, but once she had accessed enough datastorage space and enough coprocessors at sufficiently wide bandwidths, there would be no limit to the sensations she would be able to induce, record, digitize, and play back to the nth repetition, all exactingly coaxed from her mechanistic brethren.
Simply put, and Ninedenine did cherish simplicity, she knew that what she did was an act of creation--an art form. Though trying to explain to an organic that a droid such as she could appreciate art was like trying to explain that a droid could feel pain.
Droids could feel pain, of course. One of the two new prisoners coming her way was proof of that--a golden protocol droid from the looks of it, buffed to a courtly gleam, completely out of place in this warren of dank tunnels, decaying power conduits, and scurrying, fur-covered, organic scavengers.
"Ah, good," Ninedenine said as the prisoners approached, "new acquisitions." She fixed her inner optic scanner on the golden droid.
She knew how unnerving it could be to other droids when they noticed that she--a humanoid model--possessed that third optic scanner, just in from the standard left unit.
It was not in the design specs of EVs or any other model. Some called it a design flaw. Proof that she had been put together the wrong way, as if that might explain her ambitions and her most undroidlike appetites.
But Ninedenine understood that third scanner for what it truly was-the gift that allowed her to sense beyond what any other droid could sense, to never-before-quantified dimensions of experience, completely bypassing the signal-to-noise ratio of ordinary droid sensation.
Ninedenine made her third optic scanner blink deliberately out of sync with her main scanning cycle.
"You are a protocol droid, are you not?"
The new prisoner did not even have to begin to speak for Ninedenine to know the answer to that question.
His supercilious pose and posturing proclaimed him to be a protocol droid of the highest, most irritatingly officious order.
"I am See-Threepio," the droid began, redundantly.
Already Ninedenine was growing tired of it.
"Yes or no will do," Ninedenine said sharply.
Give a protocol droid its way and half the shift would be taken up with meaningless gabble. Binary was best in dealing with such units.
"Well, yes," the golden droid replied more satisfactorily.
"How many languages do you speak?" Ninedenine called up the household's duty roster on her command console. She hoped there would be no opening for a protocol droid. She would enjoy showing this one the wonders of her workshop . . .
"I am fluent in over six million forms of communication, and can readily--"
"Splendid," Ninedenine snapped, cutting off the droid again as she saw an opening did exist. "We have been without an interpreter since our master got angry with our last protocol droid and disintegrated him."
Ninedenine tried to detect any reaction to that news on the droid's part, but was momentarily distracted by the snorting guffaw from the second Gamorrean guard sitting behind her, and then by the transmission of circuit-shivering pain from the silver courier droid on the traction-test bed, whose right-side appendages suddenly failed with twin bursts of live current.
"Disintegrated . . . ?" the golden droid repeated, trying to make sense of what was going on.
Ninedenine wondered if it too had picked up the pain transmission from the dismembered droid, and was experiencing the first touch of disturbance. Pain-simulator buttons were supposedly restricted technology, typically installed only in those droids who had to interact with organics at the most personal level. Strike a protocol droid on the head, for instance, and it would respond that the blow had hurt. Such empathy toward potentially damaging physical sensation was supposed to give them deeper understanding of organics. But as far as Ninedenine was concerned, it just made protocol droids better subjects for her experiments.
And Ninedenine did like to experiment.
Also, I would add that after the events of the Clone Wars, robots are mostly seen as devices used by enemy combatants. I am sure this was used by Lucas to show that overall, citizens of the SW universe were jaded or biased towards robots/droids, so they probably had less issue with torturing them.