No, G'Kar doesn't go to Z'ha'dum personally.
G'Kar and other Narns go out to investigate the possibility of the Shadows returning, but they only find indicative hints and rumours. As he says to Na'Toth:
When you told me about the destruction of our base at quadrant 37, I knew that only a major power could attempt an assault of that magnitude. But none of the governments here could've done it. Which left only two possibilites: a new race, or an old race. A very old race.
His own investigations lead him to think the Shadows are stirring again, but he doesn't go in person to Z'ha'dum, instead convincing the Narn government to send a reconnaissance ship:
G'kar address the council about what he encountered at the rim. "...which leads me to believe that the ancient race described in our holy books may have returned. If true, this holds grave danger for all of us." Sheridan suggests that the dead worlds where he found life may have been colonized by another race, but G'kar will not accept this, as he says that the ships that fired upon him looked very much like the drawings made by G'Quan. But, because the rulers of the Narn homeworld have reservations about what he has found, no official confirmation of his findings have been given. G'kar tells the council that he has convinced the rulers of his planet to send a ship "to the heart of the enemy's own domain, located at the rim of known space...a dark and terrible place known as Z'ha'dum. It has been dead for a thousand years. No one goes there...no one. Our ship will arrive in twelve hours, coming out of hyperspace as close to the enemy's homeworld as possible. They will scan the planet for signs of life and return to hyperspace before anyone can attack."
Source: Midwinter synopsis page
If he'd been there himself and escaped, then surely he wouldn't need to send another ship to Z'ha'dum to scan it. The planets he was investigating must have been other worlds influenced by the Shadows, perhaps places where Shadow vessels had been hibernating since the last great war, but not the Shadow homeworld itself.
Also, there's a lot of talk later in the series about how nobody goes to Z'ha'dum and survives, or nobody comes back from Z'ha'dum.
Jesus Sheridan, of course.
If G'Kar himself had been there, surely it would've been mentioned at that point.