This anime I remember a scene where two normal people can summon a warrior from ancient times and during a battle some girl summons Joan of Arc and some guy summons a character who's ability is to use any sword ever created mythical or real and he fights Joan with the Excalibur
You are probably thinking of Fate/stay night (or its prequel, Fate/Zero), with a number of details misremembered.
The character who you think is Joan of Arc is actually a genderswapped King Arthur (but until her identity is revealed, people commonly guess that she's Joan of Arc). As such, it is she who uses Excalibur, not her opponents. She is typically called "Saber".
There are two characters whose ability could be construed as being able to use any sword ever created. One of them (Gilgamesh) literally owns a copy of every weapon in existence and has the ability to summon them using the so-called "Gate of Babylon". The other (Archer) has the ability to create a copy of (almost) any sword he witnesses.
What you recall as "pulling swords out of a wall" is probably the Gate of Babylon, which is really more of a "pulling swords out of thin air" sort of thing, but they all tend to emerge from the same vertical plane, so I can see why you'd recall it being a wall.
While Senshin's answer ID's both Fate/Stay Night and Fate/Zero the answer focuses on Fate/Stay Night by mentioning Archer and using images of him and Gilgamesh from Fate/Stay Night
I would say however it is actually Fate/Zero
First off Senshin is correct in IDing Gilgamesh and Arturia (The Female King Arthur) however there is are 2 parts in the question which don't match the answer being Fate/Stay Night
The first thing is
some girl summons Joan of Arc
Senshin is correct about the mistaken identity of Arturia being Joan of Arc but this was only in Fate/Zero (in Fate/Stay Night, to my recollection, Joan of Arc is never mentioned). another person who was summoned, Gilles de Rais, who fought with Joan in the 100 Year War mistook Arturia for being Joan
Also, Arturia wasn't summoned by a girl. she was summon by Kiritsugu who is a man (and is summoned by his son Shirou in Fate/Stay Night). however as apart of his plan for fighting in the Holy Grail War Kiritsugu had his wife Irisviel von Einzbern act as if she had summoned Arturia.
The second thing is that while Gilgamesh appears in Fate/Stay Night, he was summoned in Fate/Zero and survived until his appearance in Fate/Stay Night (which takes place several years after Fate/Zero).
the person who Gilgamesh follows in Fate/Stay Night, Kirei Kotomine wasn't the one who summoned him but it was Kotomine's "Teacher" Tokiomi, but later in Fate/Zero Gilgamesh sides with Kotomine