Before its more modern application to aliens, little green men was commonly used to describe various supernatural beings in old legends and folklore and in later fairy tales and children's books such as goblins. Aubeck noted several examples of the latter in 19th and early 20th century literature. As an example, Rudyard Kipling had a "little green man" in Puck of Pook's Hill from 1906.
During the flying saucer "sightings" in the 1950s, the term little green men came into popular usage in reference to aliens.
And Chris Aubeck website tries to put a timeline to the term. Also according Aubeck the use of little green men was already deeply engrained in English vernacular long before the flying saucer era, used for a variety of supernatural, imaginary, or mythical beings. It also seems to have easily extended beyond the imaginary to real people, such as the reference to small actors in the Wizard of Oz or camouflaged Japanese soldiers. Similarly, Aubeck and others suspect that when flying saucers came along in 1947, with subsequent speculation about alien origins, the term naturally and quickly attached itself to the modern age equivalent.
What is also interesting is the fact that 1960s also marked a transition in the way people imagined a stereotypical alien. In alien abduction stories they are often small but grey beings. There was an article in Wall Street Journal, which by the way was commissioned by NASA, stating that "the little green men with the wiggly antennae" would be hostile term.
In 1942, Washington Post, used the term "little green man" in reference to a camouflaged Japanese sniper who nearly killed one of their war correspondents.