As far as I know (having just quickly scanned through the compiled Words of Brandon), Brandon has been extremely vague on this topic, but yes, the idea of cross-Cosmere fighting has come up at least twice.
First, someone asked him almost the exact question you did, and he deftly avoiding giving a real answer. Instead, we got a teaser:
Q: Will you ever have characters use the different systems of magic openly in front of each other?
A: I'll warn you this, the third Mistborn trilogy is a space opera, which will involve travel between planets.
He has already said more than once that the third Mistborn trilogy will be the place where the Cosmere as an entity takes the forefront, much more significantly that what we've seen even in The Stormlight Archives. So, it seems likely that the various world hoppers we've seen will come into conflict at some point during that series. (On a probably related note, he's made comments that imply that Sazed is going to be very important in the Cosmere end-game, because he's the only know dual Shardholder.)
However, he has also briefly talked about what would happen in someone with one kind of magic left their home world, and it's not as obvious as you'd thing:
Q: If an Elantrian bonded to a Seon and traveled to Roshar, would that act as a Nahel bond?
A: It would act very very similarly, yes. But it would be like… it wouldn't necesarily do the exact same things. It would be treated the exact same way, but wouldn't grant the same powers.
The thing to note here is that, once the person left Sel for Roshar, the bond they had with their Seon would "work similarly" and "be treated the exact same way" as the bond a Roshar native would have with a spren. In order words, the person would still have some kind of investiture, but it won't be the same as what they are used to back home. This lines up pretty well with what we know about magic in the Cosmere: magic on the Shard worlds is caused by the interaction of the Shards with each other and their environment. Remove any aspect of that setup (say, leave the vicinity of the Shard) and the magic changes.
Based on these answers, it seems highly likely that we're going to see two magic users from two different worlds duke it out eventually, though probably not for a few more novels. But it's also likely that such a fight isn't going to be a straight up Allomancy vs. AonDor fight that you're imagining -- it will be much weirder.