To the best of my recollection, the main character lives with a woman in a small house, possibly atop a hill. I don't think they were blood related but the woman had some connections to a magic association.
There is an attempted robbery by some bandits that are passing by, trying to make some fast money. The main character gets involved and I think they beat the bandits but then some people from the magic association turn up in zeppelins(?) and for a reason they take him with them.
The protagonist is sad to leave but agrees and they end up in a (hidden?) magic town where if you want to stay, you have to complete missions to extend your time. He gets lessons on how to use magic from an old ugly lady.
