Origin of Arcane Magic

Question

I apologize for the length of this post, but it's a burning question. To see the actual questions, scroll down to the quoted text. I advise against this, though, as the intermediary text is pertinent background information.

I was intrigued by a question over on the worldbuilding stack exchange. To sum up, this question asks about "divine" magic, or magic that involves a deity as a patron who provides the magic effects. In answering that question, I realized that most ancient forms of magic were, at least according to most D&D and other classifications I could think of, "divine." (For those not in the know: Divine magic involves a deity, spirits, or other supernatural creatures, whereas arcane magic involves no deities, spirits, or other supernatural creatures.) Most "arcane" or "occult" spell casters of the (I.R.L.) ancient world were more like tricksters or profiteers who simply hid their tricks from common knowledge. Even in the ancient greco-roman world, magic was differentiated from religion only by social constructs, not source of power.

Even individuals in legend who we would think of as arcane spellcasters, often have divine inheritance or are divine themselves. Merlin, perhaps the first naïve choice for an example of an arcane spellcaster, may or may not have had demonic inheritance, which was then "fixed" (via divine magic) so to make him a mortal with supernatural powers. Such supernatural powers, especially when fixed by obviously divine intervention, could easily be considered divine. Even if Merlin's demonic inheritance is discounted, I cannot confidently place him as an "Arcane Caster" because we simply do not know what drives his magic. This is compounded by the observation that his character may have been inspired by druids, making him a druid, which is a divine spellcaster (according to a lot of categorizations)!

Gandalf, another naïve choice for an example of arcane spellcaster, was actually a demi-angel. Due to his divinity, he is able to cast spells. That's another example of a person we could consider an arcane caster but is actually divine.

Most other legendary, but still mortal, figures with magic are priests to a god, a druid, or have blood from mythical creatures, which are considered to have the divine magical hook-ups. Obviously we cannot use these individuals as the source of "arcane" magic. The source of "arcane" magic must be elsewhere.

Even in the earliest editions of D&D, which in turn influenced other works of fantasy, there was most definitely an arcane magician of some sort. Gygax wrote several appendices, with one pertinent one called appendix N, which cites sources of many books as his basis for formulating his influential game and the magic therein. (Thanks RPG.SE) This is little more than "I got my ideas from over there," and does not explain how this two-system view of magic came to be.

The move to have pure "arcane" casters certainly happened at some point, because we have wizards, like those in the Harry Potter Universe, who don't sacrifice rams to a deity for their powers. This brings me to my question, with an auxiliary question.

Where did the idea of Arcane Magic, magic you can use without the need of devotion or blood relation to a higher power, come from? Is arcane magic inspired from a class of tricky, god-manipulating/science people that we got "arcane" magic?

To be clear, an ideal answer will have the following:

• Quotes from or references to scholars or authors about the origin of "arcane" magic, as opposed to "divine" magic. Remember, arcane magic involves no spirits or deities.
• Mythological sources of verifiably arcane mages; the more ancient, the better. Remember that arcane magic means that this magic functions and exists independent of divinity. Verification would involve someone using their magic in spite of deities or omnipotent speakers specifically stating that magic does not come from any divine sources.
• Cultural traditions of magic use without the aid of spirits or other mythological/divine creatures.

Answer 1

Depending on how strictly you want to interpret "divinity", a number of societies from antiquity believed in forms of magic that were not strictly divine in nature. In particular, both the Classical Greek and Norse cultures had forms of magic that derived their power not from gods, but from "secret knowledge" of the working of the world -- the very definition of "arcane".

These practices were still very spiritual in nature, involving spirit realms and gods as actors in the practice of magic, but that's almost unavoidable. To those ancient societies, there was no real difference between the divine/spiritual and the scientific. In order to have something which approaches the "mechanical" magic we see in fantasy fiction today, I think you need to have a mechanical, secular understanding of science, which is a fairly new concept. Unfortunately, by the time that kind of awareness developed, magic had become a "thing of Satan" in Western culture, so I suspect (though I have no proof) that you would find a huge time gap where "arcane" magic disappeared from popular awareness until very recently, well within the last century, when it began making a comeback.

However, as mentioned, that concept of magic as merely channeling the power of evil spirits and pagan gods, is largely a misinterpretation of how ancient cultures thought of magic. It was often very similar to religion, in the sense that it had the same kinds of rites and writings, which is probably why it became so easy to demonize later on. But if you really look at how ancient cultures treated it, it becomes clear that at least some of them considered magic to just be part of the natural order of the world.

Classical Greek

To the ancient Greeks, there was a clear distinction between religion (the worship of the gods) and magic (attempting to gain the powers of the gods). In many cases, the two were tied closely together; the popular practice of divination, for example, was seen as the gods speaking through an oracle. But it's clear, from the language if nothing else, that the Greek idea of "magic" matches very closely what we could call "magic" in modern English, separate and distinct from religious worship.

Since the Greek gods were an integral part of nature, it's difficult to separate them completely from magic, or any other aspect of Greek life. However, there is evidence in pre-Classical Greek writings of magical practitioners that studied the forces of the world around them (what they would have called the "sympathies") in order to control them. The gods were still usually present in these stories, but not as suppliers of magic, but suppliers of knowledge of magic.

For example, in The Odyssey, Odysseus runs into the witch Circe. Circe herself is, of course, a god (actually a Titan), but Odysseus is fully mortal. To help him out, the god Hermes shows Odysseus the means to use magic to defend himself from Circe. This magic is as mundane as you can get: a magic wand and a magic herb. Hermes here is acting as just a messenger; the gods possess all of the arcane knowledge needed to use magic, but they have no exclusive power over it. In theory, Odysseus could have written down this knowledge in a tome and given it to anyone else, and they could gain the same control over magic as he did.

To expand on the comments such as @Jim2B, quoting Clarke's Law, there are a few famous "magicians" in Classical times that we would probably consider scientists or mathematicians today; Pythagoras, for example, was famous for performing a variety of magical abilities, as is the lesser-known Empedocles. Some of these acts are clearly fantastical (talking rivers, etc.), but others -- healing the sick or raising the dead -- could well be indicative of more advanced knowledge of health and medicine than those around him.

This fascination with magic in Greece lasted well into the Hellenistic period; there are written records of magic potions and spells right up through the Roman invasion and even afterwards; there are remnants of this Greek concept of magic even in the Bible (Simon Magus is described as practicing evil sorcery of the sort that the Greeks would have recognized). By that time, the Judaeo-Christian idea of witchcraft being a tool of the devil was starting to take over, though, so this is probably the last vestiges of "secular" magic you're likely to find.

Norse

Within Norse mythology, there is a form of magic known as Seidr, which was not powered by the gods, but by the forces of the spiritual world. In fact, in the Norse' minds, it was the other way around: the power of the gods like Odin and Frigga came from their use of magic -- the same kind of magic that mortals used. Of course, Odin was far more powerful than any human, but the women (it was mostly woman -- magic use was considered a primarily feminine pursuit) would gather together in large groups, often entire villages, to provide their spiritual essence to power magic.

As with the Greeks, seidr magic was closely associated with religion, though they were separate things. The mediums that practiced seidr used it to send their spirit throughout the Nine Realms of Norse mythology. They used their magic for more mudane uses, both good (predicting weather, promoting crop growth) and evil (cursing enemy villages.) The women practitioners were also often religious leaders, and they would sometimes use their magic to invoke the gods (similar to the concept of "summoning" or "binding" a demonic entity); but that was mostly because their use of magic gave them special insight into the workings of the gods and the spiritual world around them.