TL;DR: He didn't love it - it was just a job. A difficult one too.
From a post on StarWarsInterviews:
INTERVIEWER: What do you regard as the highlight of your Star Wars career? Maybe a special moment or scene?
BAKER: There weren’t any highlights; I was just there, in the droid. I was
mainly in the end scenes of every movie. I can’t remember any highs or
lows, it was just a job.
INTERVIEWER: Still, it made you famous!
BAKER: I’m not saying I didn’t enjoy it; it was just a job at that time. It
wasn’t a real acting job.
INTERVIEWER: Was it still comfortable for you?
BAKER: No! Although it was better than in the beginning it is still
definitely not comfortable. But then again, I got used to it.
INTERVIEWER: …And you still enjoy being R2…
BAKER: Well, it wasn’t enjoyable because I couldn’t react to the dialogue and
everything that was happening around me, so it wasn’t that easy.
INTERVIEWER: Most scenes you have done were with Anthony Daniels (C-3PO). How was
it to work with him?
BAKER: We were both in our droids; there was no interconnection at all. We
couldn’t hear or see each other. George (Lucas) used to shout “look
left, look right” and wanted me to project emotions like happiness and
sadness. It wasn’t very easy, which also applies to C-3PO. He had to
record his dialogue later on which had to match his movements. He had
way more movement because he had arms and legs. I didn’t have that, so
it was very hard to communicate. He came in an hour before me to get
in the costume, while I just needed 5 minutes to get into R2. When we
were done, they took the lid off and I am out! He needed another hour
to get rid of the costume. So, we never met a lot.
Bonus: While Baker seemed to be relatively OK with Star Wars, he didn't get along well with C-3PO actor Anthony Daniels.
From an article on Daily Mail:
R2-D2 star Kenny Baker, 83, once branded C-3PO, or Anthony Daniels as he's otherwise known, 'the rudest man I’ve ever met'.
This article from Telegraph goes on to describe the tension between Baker and Daniels.
It was the rudest thing anyone had ever done to me
It was very hard to communicate [with Daniels]
If he comes [to an interview], I won’t be there
He looked down his nose at me
I don’t hate Daniels…I just don’t like him