If I remember correctly, the story begins as the last ships containing the last bit of human civilization are leaving the heavily polluted Earth to resettle elsewhere. A man and his grandson have decided to stay on the planet. The story is basically a conversation between the man and the grandson. At one point, one of the ships dumps a load of garbage back down onto the Earth. I believe the story ends when the boy (or the grandfather) finds a green plant growing up through broken concrete.
I read the story as a youngster, so it was probably in one of those literature compilations that one would have in their elementary/junior high literature classes.