Short answer: Nothing stops the teachers from exploiting the point system as they wish.
The purpose of the point system is to introduce a sense of community between students and evoke motivation to learn and behave well. As Profesor McGonagall explained in the first book:
While you are at Hogwarts, your triumphs will earn your house points,
while any rulebreaking will lose house points. At the end of the year,
the house with the most points is awarded the house cup, a great
honor. I hope each of you will be a credit to whichever house becomes
The point system is basically the tool for the teachers and prefects to create social pressure on the students. The main idea behind is that one would be respected and praised by her/his peers whenever they do good and are awarded points. Similarly, they would be disliked and subject to ostracism if they misbehave and lose points. That was exactly what happened to Harry, Hermione, and Neville after they lost 150 points the night they left Norbert to Charlie's friends:
From being one of the most popular and admired people at the school,
Harry was suddenly the most hated. Even Ravenclaws and Hufflepuffs
turned on him, because everyone had been longing to see Slytherin lose
the house cup. Everywhere Harry went, people pointed and didn’t
trouble to lower their voices as they insulted him. Slytherins, on the
other hand, clapped as he walked past them, whistling and cheering,
“Thanks Potter, we owe you one!”
Hermione and Neville were
suffering, too. They didn’t have as bad a time as Harry, because they
weren’t as well-known, but nobody would speak to them, either.
Hermione had stopped drawing attention to herself in class, keeping
her head down and working in silence.
The system works great for that purpose. In general, the greater effect the teacher wants to invoke, the greater number of points they award or withhold. But no particular limitation for teachers is mentioned in the books. Professor McGonagall said once that Umbridge as a teacher had every right to give detention and I assume that was also true for the house points:
“Every evening this week!” Harry repeated, horrified. “But, Professor,
couldn’t you — ?”
“No, I couldn’t,” said Professor McGonagall flatly.
“She is your teacher and has every right to give you
Throughout the books we can find a lot of examples of teachers using arbitrary numbers, mostly Snape:
Ron finally cracked and flung a large, slippery crocodile heart at
Malfoy, which hit him in the face and caused Snape to take fifty
points from Gryffindor.
But other teachers also have no problem in bending the rules, like Professor Sprout:
The teachers were, of course, forbidden from mentioning the interview
by Educational Decree Number Twenty-six, but they found ways to
express their feelings about it all the same. Professor Sprout awarded
Gryffindor twenty points when Harry passed her a watering can;
The teachers report to Headmaster which, I guess, should act whenever a teacher is manifestly unjust. But we know that Dumbledore also likes to play with the numbers for his own benefit:
The din was deafening. Those who could add up while yelling themselves
hoarse knew that Gryffindor now had four hundred and seventy-two
points — exactly the same as Slytherin. They had tied for the house
cup — if only Dumbledore had given Harry just one more point.
Dumbledore raised his hand. The room gradually fell silent.
all kinds of courage,” said Dumbledore, smiling. “It takes a great
deal of bravery to stand up to our enemies, but just as much to stand
up to our friends. I therefore award ten points to Mr. Neville