We don't know much about Yoda's species mostly because "George Lucas maintains a strict policy of keeping the history, name, origin, and whereabouts of this species secret". This is also the case for his native language.
From Yoda's Wookieepedia article :
Yoda spoke an unusual version of Basic. He usually tended to place verbs (especially auxiliaries) after the object and subject (an object-subject-verb format). An example of Yoda's speech pattern: "When nine hundred years old you reach, look as good you will not." Most agreed that this pattern of speech was convoluted, and while it seems as though others of his species (e.g. Yaddle) had the same penchant for rearranging sentences, not all of them did (e.g. Vandar Tokare).
His strange syntax has a high symbolism of Yoda's personality; he always put action last. This fits well with the old master and hermit guru archetypes.