The only wizard he feared seemed to be Dumbledore.
McGonagall said that Dumbledore is the only one the Dark Lord feared. She would most likely know, and she tends to be precise and isn’t prone to either exaggeration or hyperbole.
“I know you haven’t,’ said Professor McGonagall, sounding half-exasperated, half-admiring. ‘But you’re different. Everyone knows you’re the only one You-Know – oh, all right, Voldemort – was frightened of.”
- Harry Potter and the Philosopher's Stone, Chapter 1 (The Boy Who Lived)
Hagrid also says that Dumbledore was the only one the Dark Lord feared, but he has a tendency to overstate things because of his own emotions. McGonagall saying it seems more trustworthy, because she’s shown to be logical and accurate.
He never seemed to fear Harry - though he wanted his threat eliminated.
The Dark Lord believed in the prophecy, so considered Harry a threat because he was the one foretold to have the power to defeat him. Because of that, he wanted to make sure Harry was destroyed, but he never thought of Harry himself as someone worth fear. Even when Harry had survived multiple attacks and two Killing Curses, the Dark Lord attributed it to his own errors.
“Again, Voldemort looked up at the slowly revolving body as he went on, ‘I shall attend to the boy in person. There have been too many mistakes where Harry Potter is concerned. Some of them have been my own. That Potter lives is due more to my errors, than to his triumphs.”
- Harry Potter and the Deathly Hallows, Chapter 1 (The Dark Lord Ascending)
He’s described as afraid a few times, like when he first lost his body, when he realized no Death Eaters were searching for him, and when he failed to get the Philosopher’s Stone. However, none of those times is he ever shown as afraid of Harry himself.
He did fear other things, though - mainly things related to death.
Though he only feared one wizard, the Dark Lord did have other fears - mostly of death and things related to it, such as dead bodies. Dumbledore also says he fears darkness.
“Not while they are merely drifting peacefully below us,’ said Dumbledore. ‘There is nothing to be feared from a body, Harry, any more than there is anything to be feared from the darkness. Lord Voldemort, who of course secretly fears both, disagrees. But once again he reveals his own lack of wisdom. It is the unknown we fear when we look upon death and darkness, nothing more.”
- Harry Potter and the Half-Blood Prince, Chapter 26 (The Cave)
From what Dumbledore says, it may also imply that the Dark Lord fears the unknown as well.