The official Star Trek website is incorrect. Kirk became a starship captain when he was YOUNGER than 34.
According to the official, but not necessarily canon or correct, Star Trek Chronology: The History of the Future Kirk is born in 2233 and becomes captain of the Enterprise in 2264, the year he turns 31.
In the SECOND SEASON episode "The Deadly Years" Kirk is rapidly aging and the computer at his competency hearing says that Kirk's biological age is between 60 and 70. And Kirk then says:
KIRK: No, I'm thirty four. I'm thirty four years old.
Nobody says that Kirk's statement is inaccurate and proof of his failing memory. So at that moment Kirk's age must have been between 34 and 35 years of some unspecified type that would be proper to use in such a situation.
In the FIRST SEASON episode "Shore Leave" that probably happened before "The Deadly Years" Kirk hasn't seen Ruth or Finnegan for "fifteen years" (about 14 to 16 years?). Finnegan tormented Kirk in Kirk's first year at Starfleet Academy (possibly also in later years) so Kirk should have graduated from Starfleet Academy about 11 to 13 years before "Shore Leave" and thus when aged about 21 to 24 - or younger allowing for the time between "Shore Leave" and "The Deadly Years" and for Finnegan possibly tormenting Kirk in later years at the Academy and for Kirk possibly graduating earlier than usual.
In the SECOND SEASON episode "Obsession" it is revealed that Lieutenant Kirk served on the Starship Farragut under Captain Garrovik eleven years (11 to 12) earlier, and thus when aged about 22 to 24.
So Kirk became a starship captain when aged somewhere between about 22 and 34.
I believe that Kirk is said to have been the youngest starship captain in the writer's guide for TOS.
A similar statement is made in The Making of Star Trek (1968) Part II: An official Biography of the Ship and its Crew, Chapter 4 "The Ship's Captain", page 216, that says "Kirk has been in command of the Enterprise for more than four years and was the youngest Academy graduate ever to have been assigned as a Starship Command Captain". The page number may vary in later editions.
Note it doesn't say that Kirk was either the youngest person to command a starship or the youngest person to be promoted captain, but "the youngest Academy graduate ever to have been assigned as a starship command captain".
So one's opinion about Kirk's record depends on how canonical one considers those sources to be.
Possible rivals to the title include
1) Garth of Izar who looked rather young in "Whom Gods destroy" to have already been the hero of Starfleet when Kirk was in the Academy. He might have become a starship captain very young.
2) Captain Christopher Pike. In the FIRST SEASON episode "Menagerie", Commodore Mendez, who looked significantly older than Kirk, said that Christoper Pike was about Kirk's age (from Mendez's point of view of course). Pike had been the skipper and commanding officer of Enterprise "thirteen years" (about 12 to 14) before "Menagerie", and thus might have become a starship captain when aged much younger than Kirk was in "Menagerie" and thus might possibly be a rival for Kirk's record.
3) Willard Decker. Actor Stephen Collins was born 1 October 1947. Filming of Star Trek: The Motion Picture began 7 August 1978 and ended 26 January 1979. Collins was aged almost 31 to 31.25 during filming. If Willard Decker was the same age as Collins, he was a captain age 30 or 31.
4) Jean-Luc Picard who commanded the Stargazer, perhaps with the rank of captain already, 31 years before the first season of STNG (for 22 years according to the TNG writers guide, I think, and ending 9 years before the first season according to "The Battle").
5) Captain Tryla Scott in the STNG "Conspiracy": "It's said you made Captain faster than anyone in Starfleet history, present company included" (possibly due to the alien parasites). Note her ship the Renegade is described as a frigate, thus she may not be a starship captain. And it is not specified if she became captain fastest after birth (and thus youngest), or fastest after entering Starfleet.
Please note the precise wording and the possible meanings of how Kirk's record is described in The Making of Star Trek (1968) and how the other captains are described before trying to reconcile them and decide who has the record.