Clearly, he speaks English not very well.
So he must speak another language natively. Which one?
We don't know much about Yoda's species mostly because "George Lucas maintains a strict policy of keeping the history, name, origin, and whereabouts of this species secret". This is also the case for his native language.
Yoda-speak closely resembles German in the way its sentences are constructed -- its grammar. German often puts verbs at the end of a sentence, though not always. I theorize that George Lucas (presuming he wrote Yoda's words) might know at least enough German that he thought to do this.
A common in-joke among native English speakers who learn German as a second language (including me) is to poke fun at German by sometimes speaking English with a German sentence structure/grammar. For example:
English: I wish to go home.
German: Ich wünsche zu hause gehen. (Literally: "I wish to home go.")
English words, German grammar: I wish to home go.
Yoda-speak: I wish to home go.