As presented in the various series ("The Trouble with Tribbles", TOS, "More Tribbles, More Troubles", TAS, and "Trials and Tribble-ations", DS9), Tribbles are cute, fuzzy equivalents of house flies - all the metabolic energy that could have gone into making bigger (or at least smarter) brains goes into making as many baby tribbles as possible.
As Dr. McCoy explains throughout the episode:
KIRK: Both. How many of these did Uhura give you?
MCCOY: Just one.
KIRK: But you've got, er, eleven.
MCCOY: You noticed that, huh? Here. This ought to take care of it.
KIRK: How do they? How do they?
MCCOY: I haven't figured that out yet, but I can tell you this much. Almost fifty percent of the creature's metabolism is geared for reproduction. Do you know what you get if you feed a tribble too much?
KIRK: A fat tribble.
MCCOY: No. You get a bunch of hungry little tribbles.
Then later in the same episode:
KIRK: Doctor McCoy.
MCCOY: Yes? Did you want to see me, Jim? Don't look at me. It's the tribbles who are breeding. If we don't get them off this ship, we're going to be hip deep in them.
KIRK: Explain that.
MCCOY: The nearest thing I can figure out is they're born pregnant, which seems to be quite a time saver.
KIRK: I know, but really
MCCOY: And from my observations, it seems they're bisexual, reproducing at will. And, brother, have they got a lot of will.
MCCOY: Jim, I think I've got it. All we have to do is quit feeding them. We quit feeding them, they stop breeding.
And as Spock points out:
SPOCK: Surely you must have realised what would happen if you removed the tribbles from their predator-filled environment into an environment where their natural multiplicative proclivities would have no restraining factors.
JONES: Of course. What did you say?
SPOCK: By removing the tribbles from their natural habitat, you have, so to speak, removed the cork from the bottle and allowed the genie to escape.
JONES: If by that, you mean do they breed quickly? Of course, that's how I maintain my stock. Breeding animals is not against regulations, only breeding dangerous ones. And tribbles are not dangerous.
KIRK: Just incredibly prolific.
Based on these descriptions, it's clear that tribbles occupy roughly the same ecological niche as small mammals like mice and rabbits on Earth - herbivorous prey animals who rely on crazy high reproductive rates rather than intelligence to survive. They instinctively recognize danger and signal an alarm, but they're not sentient as such.
Also based on these descriptions, they're the ultimate r-strategists, spending no energy nurturing offspring after birth; they just pump out as many as they can as quickly as they can. That implies little to no social structure - there's no bonding, there's no hierarchy. It'd be like a bunch of house flies, just a great mass of critters all lumped together, not an orderly society.
Klingons obviously have something in common with their natural predator(s), whether it's a scent or something else, so they instinctively react.