"Dogwood is one of my own personal favourites, and I have found that matching a dogwood wand with its ideal owner is always entertaining. Dogwood wands are quirky and mischievous; they have playful natures and insist upon partners who can provide them with scope for excitement and fun. It would be quite wrong, however, to deduce from this that dogwood wands are not capable of serious magic when called upon to do so; they have been known to perform outstanding spells under difficult conditions, and when paired with a suitably clever and ingenious witch or wizard, can produce dazzling enchantments. An interesting foible of many dogwood wands is that they refuse to perform non-verbal spells and they are often rather noisy."
— Ollivander, Pottermore
That seems all very well and charming, until you look at a very crucial point.
(...) they refuse to perform non-verbal spells (...)
Non-verbal spellcasting is one of the most important skills a wizard/witch will learn, up there with Apparition and counterspelling. Not having to waste breath speaking incantations is invaluable. Out of breath? Underwater? Hit with a tongue-tying curse? Need to be stealthy? Don't want someone to know what specific spell you just used? Can't spare the time needed to blurt out a phrase? Hermione herself says, when asked why non-verbal spellcasting is important:
"Your adversary has no warning about what kind of magic you are about to perform, which gives you a split-second advantage."
— Hermione Granger, Harry Potter and the Half-Blood Prince
In Fantastic Beasts and Where to Find Them, and many of the later Harry Potter movies, we see duels use non-verbal spells almost exclusively.
To have a wand that refuses to be non-verbal would be like having a wand that refuses to cast Protection charms, or use Transfiguration. It's not a "quirk", it's a handicap, a terrible disadvantage, and I'm bewildered why the Potterverse would even have it.
So, within the Potterverse, wouldn't Dogwood be considered the worst wand wood? Am I blowing the idea of non-verbal spellcasting out of proportion?