As Harry said in the comments on the Question. You're looking for the Millennium Rule trilogy. The first book is Thief's Magic.
Some of your details mix parts from book 1 and book 2, so everything below is likely to be a late arrival spoiler.
In book 1, the male viewpoint character is Tyen. He's an archaeology student on a dig with his professor when he finds a book in tomb. The book is Vella; originally she was a magician who made books using magic, but she is turned into a book by the most powerful magician in Tyen's history. Vella reveals that Tyen's world is just one of many and is low in magic. Tyen's discovery of Vella leads to him be accused of theft, having to flee his university and ultimately flee his world using magic.
The female point of view character, Rielle, lives on a world where magic is forbidden. Through a series of errors, including falling in love with an artist and attempting to avoid conceiving, she is discovered to be using magic and is banished from the city of her birth to a mountain temple. The use of magic doesn't involve colours, but colours play a big part in her life as a dyer and painter. She has an unusual ability to create large amounts of magic. Creating and Using are distinct things in this universe. A being she regards as an Angel ultimately takes her with her when he leaves her world, using all the magic on the planet.
In book 2, Tyen has taken a position at a magical university on another world. He's brought knowledge of Mechanical Magic to these other worlds, and has become a respect teacher and strong sorcerer. The return of "The Raen", the Angel from Rielle's world in book 1, causes the local society to break down. The Raen is the most powerful sorcerer in "The Worlds" and is regarded as the ruler. He's been missing for 20 years or so, having been tricked into travelling to a world that did not have sufficient magic to allow people to leave.
Meanwhile Rielle is going to be trained as a sorcerer by her Angel, this causes some significant jealously from The Raen's other followers and ultimately kickstarts the plot lines of book 2, revolving around a group of "Travelers", essentially magical traders and a rebellion against The Raen, based on the Millennium Rule, which states that powerful magician's who rule everything get replaced every 1000 years or so.