According to the (officially sanctioned) Serenity Blueprints Reference Pack, Serenity's main propulsion is provided by an radion accelerator core:
The pack itself does not seem to be available for purchase, but you can browse some of the pages on the site I linked above.
River says the same thing in "The Train Job":
Midbulk transport. Standard radion accelerator core class-code 03-K64. Firefly.
River quotes on BrownCoats.com
Radion, in physics, is a
hypothetical particle that emerges as an excitation of the metric tensor (i.e. gravitational field) but whose physical properties are virtually indistinguishable from a scalar in four dimensions, as shown in Kaluza–Klein theory.
Which may be the [pseudo-] science explanation for Serenity's propulsion in-universe. Also, to quote this answer:
Firefly class ships used a fusion explosion to propel it and that glow is the explosion causing the ship to gain a burst of speed.
On the other hand, there is also Radon (symbol Rn, atomic number 86), which is a noble gas, which, according to this answer could be used in real life for ion propulsion engines. The reason it's not used is because it's radioactive, but the Firefly 'Verse is able to shield radioactivity (except Reavers).
Maybe (speculation here!) the name of the engine was supposed to be a reference to this, as the Firefly 'Verse is not that far away in terms of propulsion (no FTL drives, for instance). One argument against that is the real-life ion drives are very slow - they provide constant but minuscule acceleration, as opposed to one huge burst we see in the series.