This sounds like Hal Draper's "MS Fnd In a Lbry", from 1961. I have it in a collection called 17 X Infinity, edited by Groff Conklin. Conveniently, Wikipedia has an entry for this story, saving me the effort of summarizing:
The title of the short story comes from the fact that all redundancy - and vowels - had been removed from our language in order for the information volume to shrink. Finally the sum of all human knowledge (which was sort of finite) was stored away in a drawer-sized box by means of subatomic processes. However the access to that information required complicated indices, bibliographies etc., which soon outgrew the size of all knowledge.
The society described quickly discovers that while knowledge itself can be encoded in ever-more compact forms, the catalogs and indexing information required to use it fill up the same space as before, and so meta-catalogs are born, and then meta-meta-catalogs, and so on.
Here's a line from the story itself (p.53 in the aforementioned anthology) that fits your memories:
This building, 25 miles square and two miles high, was buried in one of the oceans to save land surface for parking space [. . .]
The full text is available online.