Yes, Harry was special and unique, and love had a lot to do with it. That doesn't mean that he was more loving than other characters.
This is a complex question. I think the relationship between love and the prophecy (and, by extension, Harry's unique circumstances) is the key to understanding it. Dumbledore and Harry move between discussing love and the prophecy freely in this chapter so it's easy to mistake the relationship between the two, in relation to Harry. Essentially, the answer to your question is that Harry was unique because of the circumstances in which he found himself rather than because of any innate, remarkable qualities in his personality. What was important was that he loved in the circumstances in which he found himself rather than loving to a greater extent than others.
Here's the passage you refer to:
"It will take uncommon skill and power to kill a wizard like Voldemort even without his Horcruxes.”
“But I haven’t got uncommon skill and power,” said Harry, before he could stop himself.
“Yes, you have,” said Dumbledore firmly. “You have a power that Voldemort has never had. You can -”
“I know!” said Harry impatiently. “I can love!” It was only with difficulty that he stopped himself adding, “Big deal!”
“Yes, Harry, you can love,” said Dumbledore, who looked as though he knew perfectly well what Harry had just refrained from saying. “Which, given everything that has happened to you, is a great and remarkable thing. You are still too young to understand how unusual you are, Harry.”
(Half-Blood Prince, Chapter 23, Horcruxes).
It's important that, while Harry and Dumbledore are talking about the power of love, they always do so with reference to the particular circumstances of the prophecy, Lily's sacrifice and Harry's connection to Voldemort. They aren't speaking about love as a generic concept. They're talking about it in relation to Harry - his story, his choices and his role in defeating Voldemort. Dumbledore makes it clear that Harry is remarkable, yes, but only because of his choice to continue loving in the circumstances in which he's placed.
I'll go through this exchange point by point to try and make this clear, which will inevitably mean a lot of quotations.
First, Dumbledore clarifies that love is indeed the power that Voldemort knows not.
“So, when the prophecy says that I’ll have ‘power the Dark Lord knows not,’ it just means - love?” asked Harry, feeling a little let down.
“Yes - just love,” said Dumbledore. “But Harry, never forget that what the prophecy says is only significant because Voldemort made it so. I told you this at the end of last year. Voldemort singled you out as the person who would be most dangerous to him - and in doing so, he made you the person who would be most dangerous to him!”
(Half-Blood Prince, Chapter 23, Horcruxes).
They then go into the particular circumstances of the prophecy. Dumbledore makes it clear that it was Voldemort, not the prophecy, which has made Harry the person he has become. Voldemort set store by the prophecy, which would otherwise be meaningless.
"If Voldemort had never murdered your father, would he have imparted in you a furious desire for revenge? Of course not! If he had not forced your mother to die for you, would he have given you a magical protection he could not penetrate? Of course not, Harry! Don’t you see? Voldemort himself created his worst enemy, just as tyrants everywhere do!"
(Half-Blood Prince, Chapter 23, Horcruxes).
Harry can therefore be said to be the product of Voldemort's bad choices. Voldemort killed Harry's parents. Naturally, that had a major impact on the kind of person that Harry went on to be. Voldemort also empowered Harry.
"[Voldemort] heard the prophecy and he leapt into action, with the result that he not only handpicked the man most likely to finish him, he handed him uniquely deadly weapons!”
(Half-Blood Prince, Chapter 23, Horcruxes).
So Harry was shaped by his circumstances, through what Voldemort did to his parents. However, he also shapes his own destiny through his choices. Namely, Harry chooses to reject the Dark Arts and Voldemort's ways and continues to love.
"...Harry, despite your privileged insight into Voldemort’s world (which, incidentally, is a gift any Death Eater would kill to have), you have never been seduced by the Dark Arts, never, even for a second, shown the slightest desire to become one of Voldemort’s followers!”
“Of course I haven’t!” said Harry indignantly. “He killed my mum and dad!”
“You are protected, in short, by your ability to love!” said Dumbledore loudly. “The only protection that can possibly work against the lure of power like Voldemort’s! In spite of all the temptation you have endured, all the suffering, you remain pure of heart, just as pure as you were at the age of eleven, when you stared into a mirror that reflected your heart’s desire, and it showed you only the way to thwart Lord Voldemort, and not immortality or riches. Harry, have you any idea how few wizards could have seen what you saw in that mirror? Voldemort should have known then what he was dealing with, but he did not!"
(Half-Blood Prince, Chapter 23, Horcruxes).
What Harry saw in the Mirror of Erised is an example of him choosing love over power, and that love giving him the ability to triumph over Voldemort. Remember that Harry was able to get the Philosopher's Stone for this reason whereas Quirrell/Voldemort couldn't. A further example of Harry's love overcoming Voldemort was when Voldemort tried and failed to possess Harry at the Ministry.
"You have flitted into Lord Voldemort’s mind without damage to yourself, but he cannot possess you without enduring mortal agony, as he discovered in the Ministry. I do not think he understands why, Harry, but then, he was in such a hurry to mutilate his own soul, he never paused to understand the incomparable power of a soul that is untarnished and whole.”
(Half-Blood Prince, Chapter 23, Horcruxes).
Harry's choice to love despite containing a part of Voldemort's soul was a powerful one. Remember that Dumbledore (and, by extension, the series) places a very high value on the importance of choice.
“Voldemort put a bit of himself in me?” Harry said, thunderstruck.
“It certainly seems so.”
“So I should be in Slytherin,” Harry said, looking desperately into Dumbledore’s face. “The Sorting Hat could see Slytherin’s power in me, and it -”
“Put you in Gryffindor,” said Dumbledore calmly. “Listen to me, Harry. You happen to have many qualities Salazar Slytherin prized in his hand-picked students. His own very rare gift, Parseltongue - resourcefulness - determination - a certain disregard for rules,” he added, his moustache quivering again. “Yet the Sorting Hat placed you in Gryffindor. You know why that was. Think.”
“It only put me in Gryffindor,” said Harry in a defeated voice, “because I asked not to go in Slytherin...”
“Exactly,” said Dumbledore, beaming once more. “Which makes you very different from Tom Riddle. It is our choices, Harry, that show what we truly are, far more than our abilities.”
(Chamber of Secrets, Chapter 18, Dobby's Reward).
So Harry is indeed remarkable because he loves. But only because he loves in the particular circumstances of having been orphaned by Voldemort, having a part of his soul inside him and being tied throughout his childhood to the threat of Voldemort. The question suggests that Harry is an everyman because he responds in the same way that anyone else would respond to those circumstances. However, Dumbledore is saying the precise opposite. Many others, facing the same circumstances, would not make the choice to love. They would become bitter, angry, hateful, spiteful and cut themselves off from others. Harry is not like this, despite starting off in life with a deck of cards which may very well have caused him to turn to the Dark Arts. This is why Dumbledore says that Harry's ability to love, "given everything that has happened to [him], is a great and remarkable thing".
Of course, this is not to say that other characters in the series don't love, and love sacrificially. There are many, many examples that could be cited of different characters loving just as much as Harry, or more so. Sirius broke out of jail and into Hogwarts (although that task was thought to be impossible) in order to save Harry from Pettigrew. He also showed his love for Harry by fighting and dying at the Ministry. Neville showed sacrificial love by standing up to Voldemort, rejecting evil and killing Nagini on Harry's orders. Hermione showed love by wiping her parents' memories to keep them safe from Voldemort. Ron showed love by breaking into the Chamber of Secrets to try and save Ginny. And, of course, in perhaps the supreme example of love Snape gave up his whole life, risking (and eventually facing) certain death because of his love for Lily. All of these are examples of love which were costly and not at all superficial. Many other examples could be given. But none of these characters faced the same circumstances as Harry of being raised as an orphan and having a direct internal link to Voldemort.
That's why Harry was special and unique. He was the only person to have chosen love over power in the specific circumstances in which he found himself. You could argue that all of Voldemort's opponents had "a power the Dark Lord knows not" by choosing love and rejecting the Dark Arts. This is true, according to Dumbledore's interpretation of the prophecy. However, only Harry was in the position of being able to end Voldemort due to that love, because he was the only one which Voldemort picked out to attack in response to the prophecy.
So you could argue that Harry was significant because of his decision to love. But many other characters made this decision. He was special ultimately because of his circumstances, and for being the only person to face those circumstances and choose love.
