It's a word made to represent an inarticulate sound of gurgling. From LotR:
He took to thieving, and going about muttering to himself, and gurgling in his throat. So they [his relatives] called him Gollum, and cursed him, and told him to go far away.
And from The Hobbit:
"Bless us and splash us, my precioussss! I guess it's a choice feast; at least a tasty morsel it'd make us, gollum!" And when he said gollum he made a horrible swallowing noise in his throat. That is how he got his name, though he always called himself 'my precious.'
It became both his name and a mannerism. I always picture it as sort of a chocked gargling noise...if that helps any!