This is "They Don't Make Life Like They Used To" (1963) by Alfred Bester.
The story begins:
The girl driving the jeep was very fair and very Nordic. Her blonde hair was pulled back in a pony tail, but it was so long that it was more a mare's tail. She wore sandals, a pair of soiled bluejeans, and nothing else. She was nicely tanned. As she turned the jeep off Fifth Avenue and drove bouncingly up the steps of the library, her bosom danced enchantingly.
She lives in the boathouse in Central Park:
"I'm living in the house in Central Park where they used to keep the model yachts. It faces the boat pond. It's a darling place, and I've got it all fixed up. We could get the piano in together, Jim. It wouldn't be hard."
Eating the buildings:
"I always wanted a model boat when I was a kid. Once I even—" Mayo broke off. A penetrating pounding sounded somewhere; an irregular sequence of heavy knocks that sounded like the dint of stones under water. It stopped as suddenly as it had begun. "What was that?" Mayo asked.
Linda shrugged. "I don't know for sure. I think it’s the city falling apart. You'll see buildings coming down every now and then. You get used to it."
and
A crack of thunder split the quiet morning. Mayo looked up at the clear sky in astonishment. "What the hell was that?" he exclaimed.
"Watch," Linda ordered.
"It sounded like a sonic boom."
"There!" she cried, pointing west. "See?"
One of the westside skyscrapers crumbled majestically, sinking into itself like a collapsible cup, and raining masses of cornice and brick. The flayed girders twisted and contorted. Moments later they could hear the roar of the collapse.
"Man, that's a sight," Mayo muttered in awe.
"The decline and fall of the Empire City. You get used to it. Now take a dip, Jim. I'll get you a towel."
Insect-like aliens:
"Christ! You won't let it alone until you find out, huh? All right. You want the explanation for that bee smell, and them buildings falling down, and all the rest?" He turned Linda around with a hand on her neck, and directed her gaze at the Wonderland monument. "Go ahead. Look."
A consumate craftsman had removed the heads of Alice, the Mad Hatter, and the March Hare, and replaced them with towering Mantis heads, all sabre mandibles, antenna, and faceted eyes. They were of a burnished steel, and gleamed with unspeakable ferocity.
