This is a series my father read when he was younger (he was born in the 60s) and had me read as a kid, the book cover had the look and feel of classic fantasy novels such as the Wheel of Time series.
It is set in a medieval kingdom, the premise of the book was that in the past a portal had opened up to another world and an alien race tried to invade, they were beaten back and the portal was shut.
From what I remember as the series progresses another portal opens up and a war starts with said alien race.
At one point in the series one of the main characters ends up going through the portal and ends up as a slave to an alien race. Where he was enslaved was akin to a plantation in a swamp (I believe).
There is also a very vague memory of female assassin like character in a castle on an peninsula. Full disclosure, that detail may be from a different series/not very helpful.