What is the Mathom House from the book The Lord of the Rings?
It's a museum at Michel Delving in the Shire.
From the Prologue of The Lord of the Rings, specifically the first part "Concerning Hobbits":
At no time had Hobbits of any kind been warlike, and they had never fought among themselves. In olden days they had, of course, been often obliged to fight to maintain themselves in a hard world; but in Bilbo's time that was very ancient history. The last battle, before this story opens, and indeed the only one that had ever been fought within the borders of the Shire, was beyond living memory: the Battle of Greenfields, S.R. 1147, in which Bandobras Took routed an invasion of Orcs. Even the weathers had grown milder, and the wolves that had once come ravening out of the North in bitter white winters were now only a grandfather's tale. So, though there was still some store of weapons in the Shire, these were used mostly as trophies, hanging above hearths or on walls, or gathered into the museum at Michel Delving. The Mathom-house it was called; for anything that Hobbits had no immediate use for, but were unwilling to throw away, they called a mathom. Their dwellings were apt to become rather crowded with mathoms, and many of the presents that passed from hand to hand were of that sort.
Interestingly Tolkien also gives us the origin of the term: a "mathom" is a Hobbit term for, essentially, the sort of thing you'd find in a museum: something with sentimental value but little use value. Like many Tolkienian terms, this word comes - perhaps more directly than most - from Old English roots. (Thanks to @chepner for pointing this out to me.)