It's "The Skills of Xanadu" by Theodore Sturgeon, one of his many, many classic stories. In the essay "Terrible Angels: The Singularity and Science Fiction" Damien Broderick summarizes it thusly:
In ‘The Skills of Xanadu’ (1956), a primitive world is found by a hightech militarist. The naked savages wear a curious belt but little else, and inexplicably their disorganized play fetches them everything they require in life. This harmony, maddening to the rigid newcomer, proves to be a by-product of their single crucial item of technology: the belt, which links all brains into a shared knowledge base. Faced with a problem, you simply understand the answer. The skill of the best practitioner of any craft is instantly available. When the warmonger leaves, taking a belt, he hopes to use it to transform his rigid society into a perfect dictatorship. Instead, the belts convert his world into a harmonious network of free individuals, each of them experts in the skills of ‘logic and love; sympathy, empathy, forbearance’ (Sturgeon, 1979, p. 259). Years later, Sturgeon wrote wistfully: ‘I yearn to live on Xanadu, and wear their garment, and join with them in their marvelous life-style’
In an epilogue the "agrarians" of Xanadu discuss the progress of their plan to conquer the Galaxy with their insidious devices:
In each other's arms, Tanyne and Nina were singing softly, when the goblet in the mossy niche chimed.
"Here comes another one," said Wonyne, crouched at their feet. "I wonder what will make him beg, borrow or steal a belt."
"Doesn't matter," said Tanyne, stretching luxuriously, "as long as he gets it. Which one is he, Wo--that noisy mechanism on the other side of the small moon?"
"No," said Wonyne. "That one's still sitting there squalling and thinking we don't know it's there. No, this is the force-field that's been hovering over Fleetwing District for the last two years."
Tanyne laughed. "That'll make conquest number eighteen for us."
"Nineteen," corrected Nina dreamily. "I remember because eighteen was the one that just left and seventeen was that funny little Bril from the Sumner System. Tan, for a time that little man loved me." But that was a small thing and did not matter."