In the intro to the show it says/suggests the planets to be in one star system. But in episode 9 (Ariel) the intro suggests that they are spread across the galaxy. The specific quote is:
...And terraformed a whole new Galaxy of Earths
Which is it or is it not really discussed? Given that there doesn't seem to be any form of hyperspace/FTL I assume it's just one star system, but at some occasion it was mentioned the Reavers had been to the edge of the galaxy.
As Travis pointed out on Wikipedia the system in Firefly/Serenity is
actually a cluster of five stars and several brown dwarfs
More precisely the system is laid out as such:
One central A0 class star (where the "central" planets are located) that is orbited by a G5 and G0 (home to the "border" planets), and finally a F5 and a F0 (location of the "rim" planets). There are also a number of protostars scattered in there which help to maintain the habitability of the more distant planets.
The system has five stars and seven brown dwarf/protostars. All the stars orbit the White Sun, (also called 34 Tauri (2020) A, or Bai Hu (White Tiger) by the Chinese), which is a class A0 giant, 2.5 times bigger than Sol and 80 times as bright. The "Central Planets" are the terrestrial worlds that orbit the White Sun. Farther out in this system the Red Sun (a class G5, also called 34 Tauri (2020) B, or Zhu Que (Red Phoenix)) and Georgia (class G0, also called 34 Tauri (2020) C, Huang Long (Yellow Dragon)) share an orbit and worlds orbiting these stars are referred to as "Border Planets". The fourth star Kalidasa (class F5, also called 34 Tauri (2020) D, or Xuan Wu (Black Tortoise)), and its planets orbit next. After that, the fifth and last star Blue Sun, (class F0, also called 34 Tauri (2020) E, or Qing Long (Blue Dragon)), and its planets orbit. Worlds orbiting these last two stars comprise "The Rim". Planets too far away from the biozones of the main stars are sustained for habitability by a series of protostars. The protostars Qin Shi Huang and Lux orbit the White Sun. Murphy orbits Georgia. Himmbjørg and Heinlein orbit the Red Sun, Penglai orbits Kalidasa, and Burnham orbits the Blue Sun.
Although never mentioned in the aired episodes, a deleted scene from Our Mrs. Reynolds has Malcolm mention
that there are "more than 70 earths spinning in this galaxy"
This signifies that there are a great number of planets in this system of stars, many of which are habitable.
As Phillip mentions the TV Series never made it clear, it wasn't until Serenity that the one giant solar system was confirmed (Wikipedia).
To get an idea of the scale of the solar system of the Verse a "Map of the Verse" was created by the team at Firefly Ship Works.
It is stated in Serenity (the movie) that humanity found a new star system.
This system had many planets (conjecture - young system?).
They terraformed planets to make more of them earth like. Some of the terraforming did not take.
But one system.
Both the approved RPGs, the Firefly board game, as well as the QMX maps, and the onscreen maps in both Firefly and Serenity, all show a multiple star system.
The official size (as evidenced in The 'Verse In Numbers (TVIN, which is the data derived from the QMX maps and approved by Universal), star orbits are shown to 180 AU distance (1497 LM, or 24.95 light hours). For comparison, Pluto varies between 29.6 and 48.8 AU distance. Sedna is calculated to run between 76 and 937 AU. So, bodies we know to be gravitationally bound can reach further than the distances cited in the official and semi-offical sources.
Note that with the other stars all orbiting White Star (a white star, which, even if a dwarf as is Sol, it would still be more massive than Sol), according to TVIN, two are at opposition at 68 AU (Red Sun and Georgia), then one is at 121 AU (Kalidasa), and the fourth at 180 AU (Blue Sun). So, despite being 5 stars and their planets, they are in fact within gravitational reach of the central star (and would be even if it weren't inherently significantly more massive than Sol), so it's correct to refer to this as a single system.
According to Wikipedia, which refers to an official document, The Verse is a complex solar system made up of five starts, several brown dwarf proto-stars, and at least 20 planets.
The "one system" idea may be supported by a lot of sources, but there's a line directly from the show that seems to conflict.
The episodes Safe, Ariel, War Stories, and Heart of Gold all begin with this intro (in Mal's voice):
Here's how it is: The Earth got used up, so we moved out and terraformed a whole new galaxy of Earths.
