Most of the existing answers cover this. In general though, there are a few similarities across versions:
There was already an existing "bat's cave" on the Wayne manor grounds. Originally, this was just a space underneath the manor where Bruce could operate in secret. Since Wayne Manor was on the side of a hill on several dozen acres of private property, this made sense. All that was needed to do was basically outfit the existing space, which took time. In some versions, this was undertaken by Alfred in the years Bruce was away training; in other versions, like Mask of the Phantasm, Bruce and Alfred did the initial work on the cave after finding it when a 20-something Bruce was out walking with his then-girlfriend Andrea.

Based on the Animated Series logic, the cave isn't that expansive, comparatively (save for one cliff on the inside); it basically houses the massive computer, his armory, the batmobile and bat boat, a gym/training area, a relatively small crime lab, and a space for resting. It seemed to grow in scale [with some details changing] as the series progressed.

The cave in the Nolan films was relatively small, as well, compared to the grounds of the house; biggest things we saw in there were the Batmobiile and the BAT wing. Aside from the power source and the computer, and that odd raising platform, there really wasn't very much in there.

The 1989 Bat cave was massive by comparison; it had so much space, one would think it took a team of workmen months to construct all of the various security features and moving parts.

By the time Batman Forever came around, the size of the cave had been scaled back a bit; the space was smaller, but it was more tech heavy than previously shown. It was more than likely a stylistic choice on the part of the new director to differentiate from Burton's more "dark, Gothic" version... I mean, that would explain all the needless neon lighting.... but in-story, it could have just been Bruce and Alfred updating the cave with then-current, more "cutting edge" tech; the redone scale would be due to them "locking off" the unneeded spaces.

Especially in the cases where there is so much obvious construction, I too find it hard to believe Bruce and and Alfred did it all on their own; even with the cave's existing structure, some help likely had to b brought in to do some foundation work, reinforce existing elements to create entrances and exists, take care of power, air flow, wiring and to install security protocols. How this is done is never really alluded to, at least not in any story I've read, and after hearing about a seemingly well funded vigilante crime-fighter in the same city, an astute architect or workman who helped in the construction might put 2 & 2 together, causing a possible security leak.
However, IF the Cave facility ITSELF was built some years BEFORE Batman's debut and the hired help was paid well, kept in the dark about its exact location, only there when necessary and given no knowledge of what they were doing beyond completing a "special security facility" like a big as Panic Room for some "Eccentric wealthy client" or "private entity"... I could see this happening.
Using a lot of skilled foreign labor, Bruce could have had this done within a few months, and did the fine tuning himself; the guy has 12 master degrees and was taught by one of the worlds top tinkerers, so he is a mechanical genius in his own right. After that, up-keeping the cave would probably be a full time job for Alfred; it's not like Wayne Manor itself is used for much, so Al probably would spend a lot of time in there keeping the place running when he didn't serve as Batman's "man in the chair," as he did in the Batman V Superman film.
The size of the cave is actually an important factor; the relative size in each iteration of Batman is important, as it would inform the amount of work/manpower that went into developing and building the cave over time. I'm willing to believe that Bruce and Alfred more or less put things together by themselves in the Nolan Franchise, but most other version would require some serious manpower to get everything in there, and yet still keep it secret.