There are several blatantly antagonistic alien races in The Original Series that are indistinguishable from humans.
In TOS "Bread and Circuses", Kirk, Spock, and McCoy are abducted by a species known as the Magna Roma:
The trio are made to fight trained gladiators and are intended to die for public entertainment.
In TOS "Patterns of Force", we meet the Ekosians:
They are identical to humans and, due to inappropriate meddling by a human historian, they have becomes Nazis. Their first action was to fire a nuclear warhead at the Enterprise. Decidedly not friendly.
In TOS "Wink of an Eye", we are introduced to the Scalosians:
Their first action is to hijack the Enterprise, with the end goal of harvesting the male members of the crew as genetic stock.
After The Original Series, the issue becomes less clear cut. The El-Aurelians and the Sona are possible examples, but they are problematic at best. I'll say a little about these first, with caveats.
I would argue that we have only ever really encountered three El-Aurians: Guinan, Dr. Tolian Soran, and Martus Mazur. The second was the principal antagonist in Star Trek: Generations. He was very willing to commit genocide on a planetary scale.
That being said, Soran's mind was warped by the Nexus and so we may not consider him to be inherently evil. On top of that, he seemed to be "good" at first, and so Soran doesn't really count towards what the OP is asking for.
Mazur was featured in the DS9 episode "Rivals". This El-Aurian is a practiced con artist, and he is revealed to be such from the very beginning of the episode. That being said, he could be considered to be quite affable — the kind of chap you might have a beer with over at Quark's.
The El-Aurian issue is murky at best. Discussions with Guinan in TNG suggest that El-Aurians are a generally peaceful people and a race of "listeners". Guinan, however, might be giving us an idealistic view — one possibly inspired by a time before most of her people were assimilated by the Borg. There's too little evidence either way to judge the general disposition of the El-Aurians (unlike Betazoids, for instance).
Seeing as the El-Aurians are one of the few post-TOS races that are indistinguishable from humans, I felt it was important to try to assess them as completely as possible, within canon at least.
The Sona are the baddies in Star Trek: Insurrection. Before time did this to them,
they looked like this:
Baku / Sona society seems to be very similar to human society: some are friendly, some are decidedly not.
Despite the joint Federation-Sona project in the Briar Patch, very little is done to make the Sona seem friendly at any point in Insurrection. As mentioned by Troi and Riker fairly early in the film, the Sona were widely known to supply goods to enemies of the Federation — notably, they manufactured Ketracel-white within the Alpha Quadrant for the Dominion. As the film goes on, we learn more about the full scope of their activities, including
the attempted forced migration of their kin, the Baku.
However, because the Sona themselves do not look "human" and we only find out their connection to the Baku later in the film, this example tends to play into the human-looking = good / alien-looking = evil stereotype rather than work against it.
A more promising example is the following:
In TNG "Too Short a Season", we meet the Mordanites of Mordan IV, in particular Karnas:
Any initial friendliness on the part of Karnas is only a very thin veneer. We learn fairly quickly that the Federation has already had a troubled history with Karnas: he had captured a Federation starliner several decades earlier in order to bargain for weapons to vanquish his enemies. We subsequently discover that Karnas has taken hostages again. He then threatens to execute one hostage every 15 minutes until he is given access to Admiral Mark Jameson (the man who had originally negotiated the release of the starliner's passengers). Granted, Karnas is one man, but Mordan IV is described to be a very troubled world with blood feuds, violent reprisals, and constant aggression. Not a friendly place.
Post-TOS, which is what the OP is mainly seeking, I would have to say that the Mordanites of Mordan IV are the best example of a species that looks identical to humans and which are unfriendly / antagonistic. This is embodied by Karnas, as described above.