Against the odds, browsing in a (different) library a decade later, I managed to find the story again. It is entitled "I'm a stranger here myself", a short story by Mack Reynolds, first published in Amazing Stories, December 1960.
As I recalled in recounts the meeting of two men, Paul and Rupert, meeting by chance at a bar in Tangier. This was presumably during the existence of the Tangier International Zone, when the city was a haven of tolerance and bohemianism. The International Zone ended in 1956 when Tangier was restored to Morocco, so the story was already a little dated by the time it was published. As the story notes:
"It's the one town in the world where anything goes. Nobody gives a
damn about you or your affairs. For instance, I've known you a year or
more now, and I haven't the slightest idea of how you make your
living."
"That's right," Paul admitted. "In this town you seldom even ask a man
where's he's from. He can be British, a White Russian, a Basque or a
Sikh and nobody could care less.
They chat, and one of the topics that come up is whether Earth is being visited by UFOs. Paul asks Rupert where he is from, and when he replies "California", he replies:
"No, you're not ... "I felt your mind probe back a few minutes ago when
I was talking about Scotland Yard or the F.B.I. possibly flushing an
alien. Telepathy is a sense not trained by the humanoids. If they had
it, your job—and mine—would be considerably more difficult. Let's face
it, in spite of these human bodies we're disguised in, neither of us
is humanoid. Where are you really from, Rupert?"
"Aldebaran," I said. "How about you?"
"Deneb," he told me, shaking.
We had a laugh and ordered another beer.
As I recalled, Paul (the Denebian) was on Earth "Researching for one of our meat trusts. We're protein eaters. Humanoid flesh is considered quite a delicacy", while Rupert was there "Scouting the place for thrill tourists. My job is to go around to these backward cultures and help stir up inter-tribal, or international, conflicts—all according to how advanced they are. Then our tourists come in—well shielded, of course—and get their kicks watching it." The story concludes with Paul reproving this practice:
"That sort of practice could spoil an awful lot of good meat."
The full story is available on Project Gutenberg.