In the TOS Episode By Any Other Name, the Kelvans make enhancements to the Enterprise, allowing it to travel at far greater speeds than it normally could (it passed warp 11 from memory). I was wondering then is there any further reference to these modifications being removed from the Enterprise/studied by Starfleet (I don't mind if these are referred to in TOS or onwards, although I'd expect it'd only be discussed in TOS), or was this just ignored as at the next episode?
They were forgotten by the next episode and never mentioned again. I would assume that the in-universe explanation would be that the Kelvans took them out again at the end of the episode, which would explain the lack of reference to them in the future, but this is just conjecture. TOS had very poor continuity.
OFFICIALLY: No record of what was done with the Kelvan technology is ever made. But I doubt the technology was lost. More than likely what could be salvaged was taken for study by Federation scientists.
A bold statement but one confirmed by an episode of ST:TNG called "Where No One Has Gone Before"
Kirk mentions that an intergalactic voyage by a 23rd century starship would take "thousands of years" to reach the Andromeda Galaxy. For the Kelvans, intergalactic travel is a three-century journey.
In the 24th century, as seen in TNG: "Where No One Has Gone Before", Federation technology has apparently matched the Kelvans, perhaps due to this encounter, when it is discussed that a return trip to the Milky Way from the Triangulum Galaxy would take three hundred years at maximum warp.