TL;DR - The Summary
Dumbledore himself happens to have explained this to Harry in Harry Potter and the Order of the Phoenix. Dumbledore tells Harry that he knew his decision was between abuse and celebrity. He then goes on to say that he believes the choice he made - for Harry to stay at the Dursleys' - was the right one.
The Detail
The Choice
“Five years ago you arrived at Hogwarts, Harry, safe
and whole, as I had planned and intended. Well — not quite whole.
You had suffered. I knew you would when I left you on your aunt and
uncle’s doorstep. I knew I was condemning you to ten dark and difficult
years.”1
Not really a choice
“You might ask — and with good reason — why it had to be so.
Why could some Wizarding family not have taken you in? Many
would have done so more than gladly, would have been honored and
delighted to raise you as a son.
“My answer is that my priority was to keep you alive. ..."1
Dumbledore had no other choice at this point2, the Death Eaters were going to be after Harry. He also believed that Voldemort was not truly gone; that he would return and seek to kill Harry.
So the protection of Lily's sacrifice out-weighed any other option for Harry's safety. For what wizarding family could protect Harry if Dumbledore didn't believe that even he could:
I knew that even my most complex and
powerful protective spells and charms were unlikely to be invincible if he [Voldemort] ever returned to full power.1
The Outcome
Dumbledore goes on to explain that he is proud of the way that Harry turned out:
“Five years ago, then,” continued Dumbledore, as though he had not paused in his story, “you arrived at Hogwarts, neither as happy nor as well nourished as I would have liked, perhaps, yet alive and healthy. You were not a pampered little prince, but as normal a boy as I could have hoped under the circumstances. Thus far, my plan was working well.
“And then . . . well, you will remember the events of your first year at Hogwarts quite as clearly as I do. You rose magnificently to the challenge that faced you, and sooner — much sooner — than I had anticipated, you found yourself face-to-face with Voldemort. You survived again. You did more. You delayed his return to full power and strength. You fought a man’s fight. I was . . . prouder of you than I can say.1
The Outcome - Explained
There is evidence that Harry's greatest strength was his character3, 4. This is crucial, as the story sums up how Harry's overwhelming love is what conquers the hatred of Voldemort.
Harry endures hardship and suffering, mainly at the hands of the Dursleys, and in turn becomes a patient and caring person. He is often called "remarkable" and "exceptional", and not just for his feats but for his character:
[Dumbledore]: "I never dreamed that I would have such a person on my hands. ..."
...
"Young you might be, but you had proved you were exceptional. ..."1
We see this right from the start where he refuses to befriend Draco Malfoy5 and instantly becomes good friends with Ron Weasley.
Harry is on many occasions compared to his father, and are quite similar, but what James didn't have was that same selfless character which Harry possessed - right from his early life. Would James have done the same things that Harry ends up doing? Would Harry have done the same thing if he had grown up a "jock" like his father?
At the end Harry chooses his friends and family over himself6 - a choice that not every person could or would take. This is what makes him special.
1. -Harry Potter and the Order of the Phoenix, Chapter Thirty-Seven (The Lost Prophecy).
[All emphases added are mine]
2. As pointed out by Zibbobs