As is perhaps usual in Doctor Who, we are not given much of an explanation as to how the transformation works. Ood Sigma simply refers to the concoction as :
SIGMA: Ood-graft suspended in a biological compound, sir.
It seems likely that “Ood-graft” is Ood tissue of some sort (like a skin graft). Whatever the biological compound is must be designed to facilitate the slow replacement of human tissue with Ood tissue. So the DNA of the human is probably not being changed (which would in any case be insufficient). Rather, they are being replaced, cell by cell, with Ood.
The process is clearly slow (as one might expect from gradually replacing someone’s organs):
DOCTOR: Oh, they’ve been preparing you for a very long time. And now
you’re standing next to the Ood Brain. Mr Halpen, can you hear it?
Replacing most bodily tissue is not really a problem. The main problem is with brain tissue. But if we assume that Ood neural tissue can substitute somewhat for human neural tissue (particularly plausible given their psychic abilities), then the neurons storing all of the target’s memories could easily have been replaced over time.
Further, the process seems to work from the inside-out. By the time of the previous quote, the target of the potion was already mostly transformed.
As the previous quote indicates, it would seem that standing next to the Ood brain dramatically accelerates the action of the Ood tissue.
As to the origin of the Ood, it can be presumed (due to their generally humanoid form) that, as with humans, there was some Time Lord meddling in the past. The two species are likely genetically pretty distinct, nonetheless.