Golden-age short story starts with a guy shutting his door on an inoffensive but boring alien going door-to door promoting his alien god. They are all over Earth, and bother people several times every day knocking on their front door, mouthing their bland patter, and handing out pamphlets.
Some dialog between the guy and his neighbor about how annoying this all is, and how daily life has degraded since the aliens came. Someone comes up with a stratagem that finally makes the aliens stop, or leave Earth, or whatever. Much celebration.
Closing scene of the story is the arrival of a brand new alien race in a threatening-looking spaceship burning across the skies, broadcasting, "Bow down before the flaming scimitar of [our own god]."
Edit: To answer a couple of the questions:
I can only tell you I read this story sometime in the '60s, possibly the '70s. I have no idea how old it was when I read it. I devoured anthologies during that period, so it could have been in any of those.
The "flaming scimitar" wasn't a quote, I just remember the original line conveying much truculence. The context of the story made it clear between the lines that the first aliens were meant to approximate Mormons or Jehovah's Witnesses, and the second aliens were meant to approximate Muslims.