Yes, there are references to sports in Middle-earth.
First off, in The Hobbit, there are two references to football (or as Americans insist on calling it, soccer), the first of which you have already quoted:
"This won't do at all!" said Thorin. "If we don't get blown off or drowned, or struck by lightning, we shall be picked up by some giant and kicked sky-high for a football."
And:
There was a muffled yelp inside, and a toe shot up and kicked the spider straight and hard. There was life in Bombur still. There was a noise like the kicking of a flabby football, and the enraged spider fell off the branch, only catching itself with its own thread just in time.
I take this to mean that football, or something like it (perhaps something more similar to rugby?) does indeed exist in Middle-earth.
Tulkas, one of the Valar, is basically a god of wrestling. From a passage referring to him, published in The History of Middle-earth, Volume X: Morgoth's Ring:
Greatest in strength and deeds of prowess is Tulkas, who is surnamed Astaldo, the Valiant. He came last to Arda, to aid the Valar in the first battles with Melkor. He delights in wrestling and in contests of strength; and he rides no steed, for he can outrun all things that go on feet, and he is tireless. His hair and beard are golden, and his flesh ruddy; his weapons are his hands. He has little heed for either the past or the future, and is of no avail as a counsellor, but is a hardy friend...
His wife Nessa, also a Vala, is an incredible runner. From the wiki page on Valar:
Entitled the Dancer. She is the wife of Tulkas and is noted for her agility and speed, able to outrun the deer who follow her in the wild, and for her love of dancing on the ever-green lawns of Valinor.
From the book Unfinished Tales, in the chapter about Númenor:
Axes and spears and bows they had, and shooting with bows on foot and on horseback was a chief sport and pastime of the Númenóreans... Beyond all other pursuits the strong men of Númenor took delight in the Sea, in swimming, in diving, or in small craft for contests of speed in rowing or sailing.
Archery competitions were common as well. In The Hobbit, the stone-Giants also play a game involving throwing boulders at each other; the movie makes it seem like they are having a fight, but the book quite clearly says that they were doing it "for a game".
In Unfinished Tales, we read the following passage about Galadriel:
Her mother-name was Nerwen ("man-maiden"), and she grew to be tall beyond the measure even of the women of the Noldor; she was strong of body, mind, and will, a match for both the loremasters and the athletes of the Eldar in the days of their youth.
So it would seem that Elves have their own sports as well.
As for halfling sports, we have this passage from The Hobbit, which is found in the scene in which Bilbo uses stones to fend off a large number of giant spiders who are trying to eat the Dwarves:
As a boy [Bilbo] used to practise throwing stones at things, until rabbits and squirrels, and even birds, got out of his way as quick as lightning if they saw him stoop; and even grownup he had still spent a deal of his time at quoits, dart-throwing, shooting at the wand, bowls, ninepins and other quiet games of the aiming and throwing sort - indeed he could do lots of things, besides blowing smoke-rings, asking riddles and cooking, that I havent had time to tell you about.
And in The Lord of the Rings: The Return of the King, Pippin is challenged to a wrestling match by a young boy named Bergil, son of Bergond, Guardian of the Citadel of Minas Tirith, whom he has befriended. Although he declines the challenge, he does say that he is considered quite a talented wrestler in his hometown. He goes on to explain that, despite his youthful appearance, he is actually much older than the boy (Bergil is only 10 years old, whereas Pippin is 29), and any competition between them would therefore be unfair.
So hobbits obviously played many games, including (but probably not limited to):
Quoits: A game similar to the modern game of "horseshoes", in which the participants try to throw rings onto stakes.
Dart-throwing: It is not clear whether this refers to the version of the game played in bars and pubs today, with a numbered dartboard, or the version of the game played outdoors, with circular targets on the ground, sometimes called "jarts" or "lawn darts".
Bowls: Also known as lawn bowling, a game involving wooden balls.
Ninepins: The forerunner of modern bowling.
Shooting at the Wand: A form of archery competition in which the traditional round target is replaced by a narrow stake, making the task of hitting it far more difficult. Thanks to Garreth Rees for explaining this to me.
Wrestling: It is safe to assume that we're talking about something similar to the Greco-Roman variety of the sport, not the Hulk Hogan, Randy Macho Man Savage version.
Golf: According to the usually reliable site Tolkien Gateway, hobbits did indeed play golf, in a form more or less similar to the modern game. As a previous answer mentioned, it was invented when Bullroarer Took, wielding a club, knocked the head off of an Orc and sent it tumbling down into a rabbit hole. It is probably safe to assume that later hobbits used balls instead of Orc heads.
The Lord of the Rings actually says, in the introduction section regarding hobbits, that all hobbits are good at throwing stones, and spend a considerable amount of time practicing this skill.
In one of his letters, Tolkien actually said that one of the things that set Hobbits apart from other races in Middle-earth was the kind of sports in which they participated:
The chief way in which Hobbits differ from experience is that they are not cruel, and have no blood-sports, and have by implication a feeling for 'wild creatures' that are not alas! very commonly found among the nearest contemporary parallels.
- The Letters of J.R.R. Tolkien, Letter #154
This would explain why hobbit sports, with the exception of shooting at the wand, are largely peaceful and non-martial in nature, unlike many of the sports common among the other races of Middle-earth; Men and Elves, for example, are quite fond of hunting, which most hobbits would probably consider cruel and unconscionable.
We can also probably assume that many other sports were popular in Middle-earth: It is hard to imagine any culture with no concept of friendly competition in feats of strength, agility, speed, and accuracy. I would expect that villagers occasionally held competitive races, organized games of some sort (perhaps similar to football, cricket, stickball, kickball, etc), archery, javelin throws, shotput, that weird Scottish game where you flip a gigantic log end over end, and so on.
These kinds of games would probably be more common in the Shire, where people lived quiet lives far away from the shadow of Sauron, in blissful ignorance of the perils faced by Men in places like Rohan and Gondor. After Aragorn claimed the throne of Gondor and Arnor, however, and peace became the norm throughout Middle-earth, it is likely that these lands, too, found that they now had the time and inclination to engage in organized sporting events with nearby villages and towns.
Games that don't involve athletic ability are also mentioned in Tolkien's stories; aside from the obvious example of the "Riddle Game" in which Bilbo defeats Gollum, there are also references like this one, which mentions the game of chess:
Pippin looked at him: tall and proud and noble, as all the men that he had yet seen in that land; and with a glitter in his eye as he thought of the battle. ‘Alas! my own hand feels as light as a feather,’ he thought, but he said nothing. ‘A pawn did Gandalf say? Perhaps but on the wrong chessboard.’
- The Return of the King
And as this entry on the "Middle-earth & J.R.R. Tolkien Blog" shows, games of chance, i.e., gambling, were also well known in Middle-earth. To use an example related to the aforementioned challenge issued by Bergil to Pippin:
‘Twenty-nine!’ said the lad and whistled. ‘Why, you are quite old! As old as my uncle Iorlas. Still,’ he added hopefully, ‘I wager I could stand you on your head or lay you on your back.’
- ibid
And when Gandalf confronts Grima Wormtongue in Edoras, he says to Eomer:
‘Nay, Éomer, you do not fully understand the mind of Master Wormtongue,’ said
Gandalf, turning his piercing glance upon him. ‘He is bold and cunning. Even now he
plays a game with peril and wins a throw. Hours of my precious time he has wasted already...
- The Two Towers
The mention of a game, and to winning a throw, appear to be references to something like dice.
We also hear that Hobbit children play games like hide and seek, Tate, etc. at one point, Smaug actually scolds Bilbo for hiding, and tells him to "stop playing hide and seek".
If we add up all the sports and games mentioned in Tolkien's works - or at least the ones I have mentioned here - we come up with:
Quoits, dart-throwing, shooting at the wand, wrestling, archery, swimming, running, sailing, rowing, hunting, fishing, golf, bowls, ninepins, gambling (of various kinds), chess, dice, tag, hide and seek, football, riddles, and horseback riding.
This list is by no means exhaustive, and I will keep looking for more information, adding it as it appears.
