Your reasoning is correct.
My only reasoning is he meant the Elves of Lothlórien had no dealings with the dwarves, whereas the Elves of Mirkwood (led by Thranduil in The Hobbit) did.
Haldir was referring specifically to Lothlorien, which certainly isolated itself from Dwarves and Men and even from other Elves, to an extent.
Consider what he said in the book:
‘Welcome!’ the Elf then said again in the Common Language, speaking slowly. ‘We seldom use any tongue but our own; for we dwell now in the heart of the forest, and do not willingly have dealings with any other folk. Even our own kindred in the North are sundered from us. But there are some of us still who go abroad for the gathering of news and the watching of our enemies, and they speak the languages of other lands. I am one. Haldir is my name.
Yet so little faith and trust do we find now in the world beyond Lothlórien , unless maybe in Rivendell, that we dare not by our own trust endanger our land.
Boromir had heard only rumors:
‘A plain road, though it led through a hedge of swords,’ said Boromir. ‘By strange paths has this Company been led, and so far to evil fortune. Against my will we passed under the shades of Moria, to our loss. And now we must enter the Golden Wood, you say. But of that perilous land we have heard in Gondor, and it is said that few come out who once go in; and of that few none have escaped unscathed.’
Legolas only knew old tales of Lothlorien:
When all the Company had crossed, they sat and rested, and ate a little food; and Legolas told them tales of Lothlórien that the Elves of Mirkwood still kept in their hearts, of sunlight and starlight upon the meadows by the Great River before the world was grey.
Legolas himself didn't actually know what the Elves of Lothlorien were like anymore:
‘It is told that she had a house built in the branches of a tree that grew near the falls: for that was the custom of the Elves of Lórien, to dwell in the trees, and maybe it is so still.
And he'd never been there either, in his long life:
‘I will climb up,’ said Legolas. ‘I am at home among trees, by root or bough, though these trees are of a kind strange to me, save as a name in song.
The Elves of Mirkwood, meanwhile, commonly traded and allied themselves with both Dwarves and Men, at least until Smaug drove the Dwarves out of Erebor. Lothlorien too used to have extensive dealings with the Dwarves (particularly, one imagines, when Eregion prospered), thus why Haldir framed his statement as he did.
Strangers were not generally allowed in Lothlorien at all:
We allow no strangers to spy out the secrets of the Naith. Few indeed are permitted even to set foot there.
The Elves of Lorien were apparently so isolated the location of the Havens was not even common knowledge:
‘Happy folk are Hobbits to dwell near the shores of the sea!’ said Haldir. ‘It is long indeed since any of my folk have looked on it, yet still we remember it in song.
So, as other commenters noted, there's Elves, and there's Elves. The Elves of Lorien are not representative of the Elves of Rivendell, Mirkwood, or the Havens, and Haldir was not speaking for them.