According to the film's official novelisation he used the planet's magnetic interference to disguise his approach, then simply hid his shuttle in amongst a small convoy bringing parts and technical crew to the
Death Star USS Vengeance. The sheer volume of ships, confusion over their numbers and call-signs and a general air of complacency was sufficient to get him into the ship's vicinity.
Wait a minute, he told himself. They weren’t talking to me. Whoever
they were. In fact, from the gist of what he had overheard, “they”
weren’t even aware of his presence. Not surprising. This close in to
the gas giant’s powerful magnetosphere, there was all kinds of
distortion on the spectrum, and plenty of upper hybrid resonance
instability. Communicating inside the Io plasma torus was difficult
enough, and scanning more so. Subject to such powerful external
influences, instruments didn’t behave the way engineers wanted them
As each of the arriving craft gave their name and detailed their
cargo, it occurred to Scott that he might just possibly be able to
lose himself in the confusion. Bringing himself down and around, he
slipped easily into the cargo fleet’s scattered formation. There was
plenty of room for Scott to maneuver among the other shuttlecraft. No
one questioned his presence. After all, what would another single,
small shuttlecraft be doing in Io’s vicinity? All current scientific
work was performed by automated spacecraft and instrumentation.
Precisely how he got on board isn't described, but it's noted that he recognises the ship as Federation-designed so presumably he'd have knowledge of how to use the various emergency entrances and exits without setting off alarms.