Book about person using logic to leave our world and enter enter magical one

Question

The plot is about 2 characters one of which is male and a regular human with no super abilities but with above knowledge of science, the category theory to be precise, and is smart enough to create and prove new theorems in it.

He created some proof using category theory which proved impossible from the logical perspective thing. Then he finds it does not make sense from one side and error-free from another which makes him thinks about it as about something 'magical'. Since he is a scientist and knows that magic is impossible and magic is about making impossible and knows that his world has no magic in it and the whole world is only ruled by logic and not by magic. He comes after some reasoning to conclusion that what he is proved can be considered a magic in world of logic and an find his proof would be equal to a spell in a magical world but in logical world that would make him able to travel from logical world to world where lack of logic is possible - a magical world.

He shows his proof to his friend and tells him about that. His friend thinks he is crazy but still get himself convince him to do an experiment. They got prepared to leave to make sure their disappearance would go unnoticed and would not raise questions from authorities and have supplies to be able to survive in the world they wanted to travel.

After the character reads the proof they find themselves appearing in some magical world with a very low level of technology and prove themselves they have their magical abilities. They meet other humans some of whom are capable of magic, and to survive in this world they try to trick inhabitants about the greatness of their magical abilities when in reality they can do some magical things but their skill is quite lacking. In this world, one of the most magical creatures of dragons, and only the most powerful wizards can fight them off. As the plot progresses they face a situation where they need to prove that they are a real deal against one of such powerful wizards via the challenge. They invent a spell on the go to create(or summon, can't remember exactly) a dragon and when they try it they get a couple of tens of them, however, created dragons are subpar to genuine ones, they don't fly, eat grass and not breath fire and not aggressive at all. The sight of this many dragons terrifies the powerful wizard but their flaws makes him doubt their powers. They get away by lying to him that they could create a real deal, and they would as long as he is willing to fight this many dragons at once. He knows that no wizard that he was aware of ever created a dragon and since they are created, decides not to risk his life and accept that they are powerful wizards.

As the plot progresses they find at first how to scale down dragons out of safety concerns, then learn how to make them fire-breathing ones, and once they succeed in creating one in the birdcage, the small dragon uses fire and melts the cage escaping it, slightly burns main character and the character notably compares it to acetylene torch.

One of the goals of their knowledge up to the game is to be able to get back home for which they need to find a powerful enough wizard that knows the spell. If I'm not mistaken during the progress of their journey they even meet Merlin. And at least one of them manages to get home.

Dragons are not the main subject of the plot but play an important role in it as the story progresses and show the character's progress as they discover the world of magic through the lens of category theory and logic

If I'm not mixing things up one of the main characters is heartbroken by the loss of a loved one and the reason another main character invites this one to journey is to distract his friend from the loss

The book did exist before 2012 but the style wording and setting suggest it's likely much older than this.

The world described at the beginning of the novel is somewhere in our 20th century.

The novel was a translation likely from English to Russian and the book was popular enough that it was sold at a market in the open air in the small city.

The book was a dark-colored cover page with a drawing of a flying dragon and a person.

Answer 1

Aha, it was The Mathematics of Magic, the second novella in the Incomplete Enchanter series. In order to impress the wizard Dolon, Reed Chalmers tries to summon a dragon:

The oyster-colored smoke of the fire thickened and darkened. Chalmers bit off his chant in mid-stanza and scrambled back. A reptilian head a yard long was poking toward them out of the smoke. The head had a scaly neck behind it. Then came a foreleg and another.

The dragon seemed to be crawling from nothingness through an orifice somewhere in the smoke, ballooning out as it came. There it was, complete to stinger-tipped tail, gazing at them with yellow cat’s eyes.

Shea breathed, not daring to attract its attention by a movement: "If it starts for us, Doc, you get on Gustavus and I'll let go the reins."

Dolon's face was twisting as though he had swallowed too big a mouthful. The dragon lurched a few steps, not toward them but off at right angles, opened its terrible mouth, gave a whistling beeep and began to crop the grass contentedly.

"God bless my soul!" said Chalmers.

"He'd better," replied Shea. "Look!"

A second draconian head was pushed through the smoke. This one was squirted out in a few seconds. It looked at the three men, then wandered over to a clump of bright-colored flowers, sniffed, and began to eat them. Now a third and a fourth head were already in sight. As fast as the dragons were extruded, more followed them. The field down to the very confines of the trees was crowded with them, new arrivals butting the others to make room or scratching their sides on trees. Shea was counting: "Thirty-three, thirty-four—We better untie the animals and move or we’ll get stepped on. Thirty-six, thirty-seven—"

"Dear me," remarked Chalmers, fingering his chin, as they backed among the trees. "I rather feared this. The same thing happened with the mice."

"Fifty-two, fifty-three—" Shea continued. "My God, the country will be overrun with them!"

Dragons had overflowed the field and were lurching through the trees with their ungainly gait, munching everything green in sight, and mooing at each other with the same plaintive beeping sound. “Ninety-eight, ninety-nine, one hundred. Oh, boy!"

And they justify that they are herbivores:

"'Tis a thing conceivable; the Beast is in sooth of the lower orders, and cannot count beyond two." Dolon shook a finger and said with a slight leer: "Now about these dragons: tell me, fellow magicians, was't not by error you got eaters of grass? Eh? No secrets in the trade!"

"Ahem. No use taking unnecessary risks," said Chalmers, still looking a trifle wall-eyed.

"Doubtless," remarked Dolon with a glance that Shea just barely saw, "you can exorcise them as rapidly."

Then they make a small dragon that breathes fire that they fail to keep in a cage:

They had produced a dragon all right. One dragon, not a hundred. But this dragon was ten inches long, with batwings and a prominent sting on the end of its tail. It breathed fire.

The bars of the cage had been made strong enough to hold a dragon of conventional size. But this little horror fluttered up to them, squeezed through, and flew straight at the experimenters.