From the Star Wars Databank article on kyber/lightsaber crystals (emphasis mine):
At the heart of every Jedi lightsaber is a kyber crystal found on several planets, most notably the icebound caves of Ilum. This crystal is attuned to the Force, and connected to a Jedi Knight on a deeply personal level. In this way, a lightsaber is an extension of a Jedi's Force awareness. Because Jedi let the Force guide their selection of the crystal, the vibration that the crystal creates in the lightsaber blade helps Jedi center themselves and find balance in the Force. In this way, a Jedi can center his or her attention beyond the distractions of combat. A lightsaber crystal is colorless until first attuned and connected to a Jedi -- at which times it glows either blue or green, or in some rare instances, another shade. From that point on, it retains that hue.
It should also be noted that Jedi lightsabers come in all colours of the rainbow until Ilum became the primary - and after the Ruusan Reformation, sole - source of lightsaber crystals gathered, at which point the colours are limited almost exclusively to blue or green only. Quoting from Wookieepedia's article on lightsaber crystals (Note: No source cited, although the following are known to be generally true):
Lightsaber crystals which Jedi harvested are at first colorless. When, however, a Jedi harvests and attunes to the crystal it gains its color, though by discernment and the guidance of the force the Jedi most likely already knows the color the crystal is to be.
...most Jedi used blue or green crystals from Ilum Caves based on personal choice, and then the use of crystals coming from various source worlds became marginal. Purple, yellow, and orange crystals were still used by some Jedi until the fall of the Jedi Order, but they were exceedingly rare, and often passed down through generations.
This shows that although the Jedi attuning him/herself to the crystal is the one who decides - by the will of the Force - its colour, the source planet has some influence one way or another too.
As opposed to naturally occurring crystals, the colour of synthetic crystals depend purely on the nature of its creator. Before the dark times, before the Empire, the Jedi and/or the Republic are in control of all known crystal sources, so the Sith Order are forced to rely almost exclusively on self-made synthetic crystals:
The synthetic crystals, favored by users of the dark side for their offensive properties, were impregnated with the negative energies of their creators which gave to the crystals their red color; the majority of the Sith and Dark Jedi wielded crimson-bladed lightsabers, though colors similar in hue to red, such as orange and magenta, were not unheard of.
During the Imperial era, the tables are turned and it is now the Jedi who are forced to rely on synthetic crystals. Compare the Sith's use of synthetic crystals to Luke's formation of his own synthetic (Wookieepedia lightsaber article - Shadows of the Empire):
When Luke Skywalker set about constructing his lightsaber, he based the design on schematics supplied by Obi-Wan Kenobi that he found in a journal. Notably, he was forced to use a synth-crystal for his weapon, as he lacked any knowledge of the sites where Jedi traditionally gathered crystals, most of which having been razed during the Purge. The resulting crystal possessed a green coloration, due to Skywalker's meditations upon it during its formation.
During the rise of the New Jedi Order (Jedi vs. Sith: The Essential Guide to the Force),
When Skywalker's reformed Jedi Order arose, many of the original crystal sites would be reclaimed, but early apprentices would rely on synth-crystals.
Also note that
No longer having Ilum crystals at their disposal, Jedi of the New Jedi Order favored the use of various crystals and gems, creating lightsaber blades of multiple colors.
The loss of Ilum to the New Jedi Order, despite the reclamation of other crystal sites as well as the Jedi Temple of Coruscant, may have been due to the fact that Ilum has been kept secret by the Jedi from even the Republic:
Ilum was first discovered by the wider galaxy in 22,800 BBY when a Jedi, navigating hyperspace in the Force, felt a strong pull from deep in the Unknown Regions
The real wealth of Ilum was not discovered until one survey crew was chased into a mountainous ravine by local predators and discovered the mother lode of kyber crystals
When the High Council learned of this discovery, they dispatched artisans, scholars, and other Jedi to protect the caves and research how deep they went. Instead of reporting the discovery to the Galactic Senate per custom, the High Council deleted all records of Ilum from their ships and removed mention of it from all reports. A team of surveyors within the Order mapped out a complex route through hyperspace from Metellos that would take a Jedi team directly to Ilum. In order to navigate the course however, one would need the prowess in the Force that only a fully trained Master would possess
Some time before the Cold War, the Sith Empire attacked the planet with a force of Sith and shock troopers, killing the Jedi present there
Eventually, the Order reclaimed Ilum and once again hid its location, removing records of the battles there
Because its existence was not known generally to non-Jedi, Ilum was virtually forgotten after the destruction of the Jedi and Palpatine's information purges. This gave the remaining Jedi a safe harbor.
Ilum's secrecy was not to last. After scouring the Temple Archives on Coruscant, the Empire swiftly moved into seize the planet.
Either due to the difficulties involved in reaching Ilum, or because it is no longer a secret, there may be no point for the New Jedi Order to use Ilum exclusively anymore, especially when the new order was not built on a millennia-long tradition. As such, New Jedi Order lightsabers are not dominated by blue and green colours due to the prevailing practices.
It should be noted that synthetic crystal colours can be modified with the right technique, further proving that a synthetic's colour is all about the nature of the Force energies imbued within:
The Jedi Order had an advanced technique that allowed, in case of need, the cleaning of synthetic crystals made by Dark Side users from the hate of their former owners. In 41 ABY, Jaden Korr, who had lost his lightsaber, used the Force to banish the Dark Side from the red crystal of a Dark Jedi he had killed. Through meditation, he was able to remove all traces of the Dark Side and imbue the crystal with the Light Side, turning the Dark Jedi's lightsaber into a yellow-bladed Jedi weapon.