So, are either of these materials vulnerable to magic? I'm talking about the Comic book versions only.
Adamantium has been destroyed
First things first, Adamantium have slightly more complicated histories than just becoming a metal or finding a metal one day. The first "version" of Adamantium was Proto-Adamantium, which was accidentally discovered by Dr. Myron MacLain during the Second World War. It was then recreated in the 60s, but using a combinations of Resin as opposed to the Vibranium in the proto-Adamantium. More about Adamantium can be read at the bottom of this answer 1
Through a metallurgic accident, MacLain created the indestructible Vibranium-steel compound that was used to create the shield used by the super-soldier Captain America. MacLain spent decades attempting to duplicate the process, and although unsuccessful, he instead created True Adamantium in the 1960s.
Adamantium - Marvel.com
Vibranium has a simpler history, existing as a naturally occurring meteoric metal, which although rare exists naturally in the universe. Of which most of the Earth's Vibranium exists exclusively in Wakanda.
Now onto the question at hand. I found on example of magic breaking Captain America's shield, a iron-Vibranium alloy. It was destroyed by King Thor in Thor Vol. 2 #73. Thor, wielding the power of the Odinforce destroys the shield and kills Captain America, the timeline is however restored in #79. In the same comic just a few panels earlier, Thor melts Wolverine's Adamantium Beta with the Odin force.
On another Earth, where the Adamantium is noted as being weaker, Valkyrie, under the power of Enchantress shatters Cap's shield with a magic Sword. (Ultimate Comics Vol. 1 New Ultimate #2)
1 This metallurgical accident was shaped into a disc and used to make Captain America's shield. As stated above, MacLain continued to attempt to recreate the "proto-Adamantium" however was unsuccessful. True Adamantium was instead discovered in the 1960s and would become the standard for Adamantium, however, while proto-Adamantium was formed with Steel, Vibranium and an unknown catalyst, True Adamantium is created through the mixing of certain chemical resins whose exact composition is a closely guarded government secret, and exists in a mould-able state for 8 minutes before hardening no matter the temperature. The only known substance able to pierce Adamantium is the compound known as Antarctic Vibranium, also called "anti-metal".
There also exists Secondary Adamantium, which is far weaker and can be broken by those with Superhuman strength.
Due to the prohibitive cost of Adamantium’s creation, many parties have resorted to the use of a somewhat weaker compound named Secondary Adamantium, which was once used to coat the sentient computer named F.A.U.S.T..
Adamantium - Marvel.com
Adamantium Beta, which formed after True Adamantium merged with Wolverine's bones and was changed on the molecular level by his Healing Factor:
Wolverine's mutant healing factor allowed him to survive the process and induced a molecular change in the metal, transforming it into a wholly new metal, named Adamantium Beta that does not inhibit the biological processes of bone.
There also exists various variations for Adamantium for different Earths, such as Earth-1610 (Ultimate Earth), which isn't as strong as that of Earth-616, etc. Finally, as noted in the above link, the Soviet Union attempted to create Adamantium but weren't exactly successful:
Attempts by the former Solviet Union to reproduce the metal resulted in the creation of Carbonadium, a weaker yet far more malleable form that was used to create retractable coils wielded by the Russian super-soldier Omega Red.
While Wolverine is almost certainly the most frequently-seen character with adamantium, we've generally seen few stories where people are explicitly trying to damage his adamantium bones.
The character who's probably the best bet for magical attacks against adamantium explicitly would be Ultron, as his body has (almost) always been made from adamantium, and as he's fought so many characters over the years.
Ultron has an internal molecular rearranger inside him that allows him to change the shape of his adamantium shell. The Scarlet Witch's probability-altering powers have forced this to malfunction on at least one occasion, allowing Ultron to be beaten. I don't believe she's ever used magic to directly destroy his adamantium body.
(Note: it's not clear if the rearranger merely allows larger alterations, like when Ultron has altered portions of his body into a non-humanoid shape; or, if the rearranger is what allows him to move his body at all (as opposed to being an adamantium statue).
Outside of the Scarlet Witch, Ultron has rarely encountered magic-users. Technically, of course, Thor might count as a magical being - but his attacks on Ultron have been physical (or electrical), not strictly speaking magical. I can't recall notable encounters between Ultron and the likes of Doctor Strange or Juggernaut (or even the Black Knight's sword), for instance.
Therefore, I don't believe there's definitive proof as to whether adamantium could be destroyed by magical means. As magic generally stretches outside the laws of physics, I'd think it would be possible, but there's no definitive proof one way or the other.
Wakandan vibranium's special abilities come from its ability to absorb vibration; any advantages it has in durability are tied to that. While I can't recall if magic has ever been explicitly used to destroy something made of vibranium, there's no reason to suppose that magic that could affect tin, gold or aluminum couldn't affect vibranium as well (leaving off iron/steel, as some versions of magic have an explicit issue with those).
The same holds true for Antarctic vibranium (aka anti-metal, since it destroys other metals).