Oliphaunt is the name the Hobbits use, and Mûmak is the name the Gondorians use.
Oliphaunt is used by the Hobbits.
`Were there any oliphaunts?' asked Sam, forgetting his fear in his eagerness for news of strange places.
The Lord of the Rings - Book IV Chapter 3 - "The Black Gate is Closed"
The language that the Hobbits speak is translated into Modern English in the book, so the name follows a similar etymology that it would follow assuming English speakers only new of elephants as a distant legend. As Tolkien explains it in his guide to translators:
It is an archaic form of elephant used as a ‘rusticism’, on the supposition that rumour of the Southern beast would have reached the Shire long ago in the form of legend. ... Oliphant in English is derived from Old French olifant, but the o is probably derived from old forms of English or German: Old English olfend, Old High German olbenta ‘camel’. The names of foreign animals, seldom or never seen, are often misapplied in the borrowing language. Old English olfend, etc. are probably ultimately related to the classical elephant (Latin from Greek).
Nomenclature of The Lord of the Rings (Published in The Lord of the Rings: A Reader's Companion)
Mûmak is used by the Gondorians
'Ware! Ware!' cried Damrod to his companion. 'May the Valar turn him aside! Mûmak! Mûmak!'
The Lord of the Rings - Book IV Chapter 4 - "Of Herbs and Stewed Rabbit"
Tolkien says that this was a direct loan word from the Easterling tongue, and that this was the name by which these creatures were known in Harad.
Of the speech of Men of the East and allies of Sauron all that appears is mûmak, a name of the great elephant of the Harad.
earlier draft of The Lord of the Rings Appendix F (published in The Peoples of Middle-earth)
mûmak, Harad word for great elephant; pl. mûmakil, III 118.
"Words, Phrases, and Passages in The Lord of the Rings" (published in Parma Eldabaron #17)