Book
In the original Roald Dahl book, the chocolates came out of a serving hatch, ready to be loaded onto Post Office trucks for delivery to the shops.
‘But Grandpa, who,’ cried Charlie, ‘who is Mr Wonka using to do all
the work in the factory?’
‘Nobody knows, Charlie.’
‘But that’s absurd! Hasn’t someone asked Mr Wonka?’
‘Nobody sees him any more. He never comes out. The only things that
come out of that place are chocolates and sweets. They come out
through a special trap door in the wall, all packed and addressed, and
they are picked up every day by Post Office trucks.’
‘But Grandpa, what sort of people are they that work in there?’
Films
In the 1971 film Willy Wonka & the Chocolate Factory, we see a Wonka-branded van driven by a non-Oompa-Loompa.

And in the 2005 reboot Charlie and the Chocolate Factory, the gates open and automated packing arms extend to load trucks driven by what are presumably non-Oompa-Loompa lorry drivers (because otherwise, why the need for secrecy?)
When Charlie says that "the gates are always closed", he seems to be referring to the factory doors, not the loading-yard gate.
