It seems that life in the Federation is less perfect than we are led to believe. The Federation actually maintains several penal colonies,
Elba II, for example, housed the criminally insane. This included, notably, Garth of Izar who was jailed for attempted genocide during the 23rd century.
Those who were deemed curable in the 23rd century were sent to Tantalus V.
Seemingly, insanity was not the crime de jure of the 24th century. For example:
Ro Laren was imprisoned on Jaros II for disobeying a direct order and The New Zealand settlement you mentioned was home to Richard Bashir and Tom Paris.
Bashir had been involved in the illegal genetic enhancement of his son while Paris was convicted of treason after joining the Maquis.
So to answer your question, in the 23rd century, penal colonies were seemingly reserved for the insane (curable or not). By the 24th century, "crime" seemed to be considered anything that destabilized or threatened to destgabilize the government of the Federation. Paris had been responsible for the deaths of three other officers in a piloting accident and was only discharged from Starfleet. When he joined the Maquis, he went to jail. Bashir was jailed because his actions could have created another Khan Noonien Singh who would have made war on (and against) Earth.
Crime existed well into the 24th century. The Maquis were traitors and/or terrorists depending on who was answering the question. The New Essentialists tried to sabotage Risa. The Orion Syndicate originally only consisted of Orions but seemed to have members from many species by the 24th century (O'Brien infiltrated despite being human).
Why is there a penal colony on Earth?
Richard Bashir described it as a minimum security facility when he was sentenced there. That implies that this was for nonviolent criminals or those who presented a relatively low threat. Paris had been convicted of treason on his first mission with the Maquis. It's possible he was sent to minimum security because (A) he was not seen as dangerous and/or (B) his father (Admiral Paris) pulled some strings to keep him out of a nastier place.