It's never explicitly stated that Smith was with the Nazis prior to the surrender of the Allies
I'm not sure which particular conversation from episode six you're talking about. I have found only one in which they discuss the war (00:33:07):
Wegener says:
Nobody talks about the camps. Nobody talks about how many we exterminated. We got commendations for it. But nobody talks about it.
By "we" here, Wegener could simply be referring to himself and Nazis in general.
However, Smith responds:
What's there to say, Rudolph? It was necessary work. We did it.
Which implies that Smith also took part in such atrocities. This however, leaves three options.
Smith joined the Nazis early, perhaps traveling to Germany (As in the answer of @NKCampbell or something similar.) He thus took part in atrocities in Europe.
After the US surrendered, Smith joined the Nazis as white americans were encouraged to do under the new regime. The camps and exterminations Smith took part in were thus all after the US had surrendered. Smith may refer to this as part of the "war" as he was dealing with a (probably initially large) American civilian resistance, etc, which was probably almost like a minor civil war initially. Also,Another reason he wasrefers to it as a "war" could be the fact that he is trying to justify his actions. (Wegener, who brought it up, had not used the word "war" while talking about the atrocities.)
Throughout the series, Smith is often talking about the individual being unimportant and needing to devote itself to the greater cause of Nazism. Thus, when he says "We did it", he may simply be talking about the Nazis as a whole, seen as a giant entity which he considers himself a part of. Though, of course, we also know that Smith has done plenty of terrible things himself during his career in the SS.
Regarding option #1, one thing to note is that Smith's German is limited. In episode 8, Heydrich asks Smith whether his children speak German. Smith replies
Better than me, certainly.
Heydrich then says
Ah. I seem to recall.
This might be inconsistent with Smith staying several years in Germany surrounded by German speakers? Though not necessarily. It's possible he lived there without learning German too well. I just figured it was worth noting.