This is Lawrence Watt-Evans's With a Single Spell
Tobas had been lucky to find a wizard to take him on as apprentice. But then the wizard died suddenly and unexpectedly after teaching Tobas only a solitary spell, and the youth was too old to find a new master. How could he earn a living when all the magic he could do was light fires?
A DESTINY UNFOLDING
In the Small Kingdoms there were dragons to be slain, princesses and gold to be won, magic castles, witches who knew the secret of immortality, and other treasures. But how could a wizard with a single, simple spell hope to find them and win them?
Tobas travels through a magic tapestry to a wizard's castle. Indeed, he finds a book of powerful spells from the prior owner, as well as the wizard's immortal wife. They are trapped because the return portrait is no longer accurate, but Tobas finds a way to fix that. And yes, at the end, Tobas tracks down the person whose boat he stole and gives them a replacement.