In season 4, episode 9 (Hard Time), Miles O'Brien gets annoyed with Quark's slow service and grabs hold of his hand into some sort of joint lock.
Aikido, Hapkido, or Jujitsu
This seems to be a standard "wristlock", perhaps a supinating one:
This is a common move in the disciplines I listed above — see here.
(Note: I'm not a martial arts expert. If someone can pin down the exact move that O'Brien is using, or can find an official reference about the level of improvisation in the scene, I'd be happy to defer to that.)
In Jujutsu/Aikido it is called the Kote Gaeshi.