There's a famous scene in the Justice League Unlimited cartoon in which Lex Luthor has taken over Flash's body, and he triumphantly stands before a mirror declaring, "Now I'll learn the Flash's secret identity!" But when he pulls the mask off, he just stares into the mirror and murmurs, "I have no idea who this is..."
Oliver Queen is a public figure, but it's not like he's a movie star. Plenty of people on Arrow have to ask, "Are you Oliver Queen?" His face isn't instantly recognizable, and that's when he's just walking down the street.
Now, if you were buying drugs one night, and someone in a mask with a distorted voice who had a jawline that was kind of similar to Conrad Hilton, son of the billionaire Hilton family, started beating the hell out of you, would you ever make the connection? Probably not. That's the kind of person Oliver Queen was before he disappeared: a ridiculous wreck of a spoiled brat. Plenty of rich people are in the news, but aren't instantly recognizable to everyone they encounter.
What if it was someone with a more recognizable face, like Mark Zuckerberg? Maybe (though certainly not gauranteed), but that's crazy! Mark Zuckerberg attacked you last night? Mark Zuckerberg is the Vigilante? Psh, no way. Maybe the guy kind of looked like him, but it's OBVIOUSLY not Mark Zuckerberg, dude. Give me a break.
As for the people Oliver has met, like Detective Lance, when all you've got is "a strong jaw with stubble," it's easy not to make the connection. Especially when, as they do on Arrow, you're given conclusive proof that Oliver Queen CAN'T be the Arrow.
And you have to remember, unlike Batman, "the Arrow" doesn't necessarily need to be rich. He could be literally ANYONE in the city, just someone with access to a bow and some arrows. Frankly, connecting such a character to a public figure would probably be MORE of a stretch than assuming he's some crazed loner no one has ever heard of.