The Fantastic Adventures of Unico (1981).
From Wikipedia:
Unico's first movie, titled The Fantastic Adventures of Unico in English and simply Unico in Japan, was released in Japanese theaters on March 14, 1981, in Mexico on Canal de las Estrellas on September 27, 1982, and direct-to-video in the United States by RCA Columbia Pictures Home Video on May 12, 1983. This musical film, narrated by singer/songwriter Iruka, directed by Toshio Hirata, written by Masaki Tsuji, and with animation by Yoshiaki Kawajiri, presents the back story of Unico's banishment by the gods and his subsequent travels, as well as his friendships with Beezle (to whom he grants his own horn) and Chao/Katy (to whom he grants the wish of becoming a human girl).
The next creature in Germany Unico encounters is Chao (Katy), a black-and-white cat who dreams of becoming a witch. Katy and Unico befriend a lonely old woman, whom Katy mistakenly believes is a witch and will teach her some magic as well as transform her into a human girl. When Unico changes Katy into a girl, Katy at first believes that the old woman did it, until Unico proves it was his doing by changing her back into a cat; but Unico, seeing how selfish Katy has become, refuses to change Katy back to a girl, until one day when Katy saves the old woman from drowning. Katy then becomes entranced by a man posing as a lord (Danshaku in Japanese, Baron de Ghost in English). He invites her to his castle, gets her drunk and attempts to seduce her. Unico follows, rescuing Katy (with help from Beezle) after transforming into a majestic white winged unicorn and destroying the demonic monster that the "lord" had transformed into. Afterwards, the West Wind comes to take Unico away again, and Katy moves in with the old woman.
The main character in this film isn't the cat girl, but a baby unicorn named Unico.
Around half an hour into the film, he meets Katy, a mostly black cat (with a red bow) who longs to be human. He later grants her wish to become a human girl, after which she encounters a handsome baron on horseback, who invites her to his eerie castle. The baron is a sinister individual, and doesn't have white hair, but Katy also interacts with a white-haired old woman, so you could be conflating memories of her appearance with his.
You can view the scene where Katy is transformed into a girl at around the 43:10 mark in the video below. She first encounters the baron at around the 51:57 mark, and arrives at his castle at around the 59:34 mark.