We are shown 2 new examples of this in the Season 2 finale:
Zoom travels back in time several seconds, causing 2 Zooms to exist at the same time. The older Zoom - the one that traveled back in time - kills the younger Zoom when Barry won't do it, thus leaving one Zoom left.
And:
At the end of the episode, Barry travels back in time several seconds and convinces his younger version to sacrifice himself to stop Zoom's machine from destroying all of the alternate Earths.
It makes a little more sense after seeing in action 2 more times. Time, apparently, isn't perfectly rigid and when a speedster travels back in time and alters events, the out-of-time speedster continues to exist despite the new order of events that prevent them from traveling back in time and completing the loop.
Zoom used this to his advantage. He traveled back in time and created his grand plot with the help of himself. So, all the appearances of Zoom as the villain were one version of Zoom and all the appearances of Jay Garrick were the other version of Zoom. Thus, they were allowed to exist simultaneously.
When Zoom pulled Garrick through the breach and killed him, he was killing himself - his time remnant. It did not affect him because the timeline had changed.
The link posted in the comments helps explain the time remnant situation a little better:
https://movies.stackexchange.com/questions/52100/the-flash-timeline-remnant-theory
