In season 1 of Supergirl all the aliens were criminals from Fort Rozz but now, in season 2, there are loads of civilian aliens in the world just living amongst the humans in peace (mostly). So I guess I missed something but where did all the non-Fort Rozz aliens come from?
They were previously in hiding. At the start of Season 2 the President signed the "Alien Amnesty Act", allowing those aliens that were hiding on Earth to come out of the shadows and live ordinary, peaceful lives.
Supergirl: Wait, the President is coming here?
Alex: Yeah, she wants to tour the DEO while she's in National City to sign the Alien Amnesty Act.
Supergirl: It's amazing, an executive order allowing aliens on Earth to come out of the shadows and live as full citizens.
As to how they got there, the short answer is that faster-than-light travel is readily available in the Superman universe. The mere existence of the DEO prior to Fort Rozz crashing shows that alien arrivals have been happening for decades before the start of the show.