Does this term originate in Star Trek or was it used in previous sci-fi works or does it predate sci-fi in discussing initial contact between cultures.
Actually, the first use of this exact phrase, at least in printed works, was in 1875, according to Google's handy Ngram Viewer:
This was used in an astronomical sense, in the Monthly Notices of the Royal Astronomical Society :
But for usage in the desired sense of "initial contact between cultures", the very first appearance was in the (rather lengthy) title of the following book:
- With a Prehistoric People, the Akikuyu of British East Africa, Being Some Account of the Method of Life and Mode of Thought Found Existent Amongst a Nation on Its First Contact with European Civilisation: With Illustrations and a Map
by W. Scoresby Routledge and Katherine Routledge, first published 1910.
In the former, we have the passage
He had piloted the Adastra to its first contact with the civilization of another solar system...
which provides the first usage of the phrase in science fiction (in the desired sense).
(Thanks to @user14111 for the Proxima Centauri reference, and for pointing out that a 1916 instance reported in a prior version of this answer may not have been correct.)
According to the Historical Dictionary of Science Fiction, the earliest known occurrence of the term first contact in the science-fictional sense of "[t]he first meeting between two different intelligent species" was in the novella "Proxima Centauri" by Murray Leinster (pseudonym of William F. Jenkins), which first appeared in the March 1935 Astounding Stories (available at the Internet Archive). The following quotation is from page 21,column 2, of the original magazine publication, or p. 669 of Isaac Asimov's anthology Before the Golden Age:
He had piloted the Adastra to its first contact with the civilization of another solar system.
As far as I know, the first use of this exact phrase was in 1945 (~20 years before Star Trek), in a short story by Murray Leinster called First Contact. This story has long been considered one of the best science fiction stories ever (it's in the "Hall of Fame", and is in a number of anthologies such as this one.)
In fact, the author's estate tried to sue Paramount for trademark infringement over Star Trek: First Contact. The court found that Leinster probably did coin the term, but in the interim decades, it had become too generic to trademark any more.