Gotham City isn't in the same universe as Lovecraft's writings, but Lovecraftian monsters appeared in Batman: The Doom that Came to Gotham in the year 2000. It is an Elseworlds story which isn't part of the main canon.
Arkham Asylum was named for its founder, Dr. Amadeus Arkham. He first appeared in Arkham Asylum: A Serious House on Serious Earth, in 1989 (according to the DC Comics Database). I don't know that the asylum had much backstory before that. It's only a coincidence that Dr. Arkham shares his name with the fictional town in H. P. Lovecraft's writings.
Lovecraft was one of the inspirations for the stories of Batman, at least by the 1960s.
In the Batman TV series, which first aired in 1966, there was a villain named the Outsider, whose name was taken from a short story by Lovecraft. (See Batman: The Complete History, page 102.)
Dennis O'Neil, one of the writers of Batman comics, stated that one of his influences was Lovecraft. From Batman: The Complete History, page 138, emphasis mine:
This story, for Detective Comics #395 (January 1970), presented Batman as a fish out of water, caught up in a Mexican horror tale about a wealthy couple who seek eternal life but end up crumbling into dust. This was a long way from Batgirl and the Batmobile, but O'Neil said, "I'm sure we didn't give that a second's thought. I just wanted to make it Gothic and spooky. I was being influenced by writers like Lovecraft and Poe, and I didn't think about Gotham City."
Arkham Asylum first appeared in 1974, inspired by Lovecraft. From former DC Comics editor Jack C. Harris, emphasis mine:
One summer, I immersed myself in the dark works of H. P. Lovecraft... It was during a conversation with writer Denny O'Neil, before I had even graduated from college, that I suggested that Batman villains such as Two-Face and Joker should never be housed in a common prison; they should be locked away in an insane asylum. And what better asylum could there be for such maniacs than Arkham, the dark dwelling of the tormented souls from Lovecraft's horrific tales? Denny agreed and used the idea, making Arkham my first contribution to the foundations of the Dark Age of Comics. (It's all there, in Batman #258, September [should be October] 1974, the true first appearance of Arkham Asylum in DC Comics...)
—Foreword to The Dark Age: Grim, Great & Gimmicky Post-Modern Comics, page 5
The summer mentioned in that quotation may have been before 1970, which could explain O'Neil incorporating elements of Lovecraft into Batman in 1970; I couldn't find a date for when Harris graduated college.
Before 1966, I couldn't find any explicit mention of Batman being inspired by Lovecraft. Lovecraft published The Picture in the House, his first story to feature the town of Arkham, Massachusetts, in 1921; the first Batman story was published in 1939. It's possible that Lovecraft had an influence over Batman from the beginning, but Lovecraft's writings were still fairly new when Batman debuted, and the genre hadn't had time to develop fully. On Wikipedia, the earliest work not written by Lovecraft to feature the town of Arkham was published in 1937.