The story is "The Nonesuch" by Larry Niven, which appears in his short story collection Convergent Series (1979).
From memory, it starts with a paragraph or two of backstory saying that a human colony on another planet had appeared to be doomed. A deadly, invisible predator killed one or two members of every party that ventured out of their base, and a colony cannot go on forever cowering behind barricades. Then, it says without elaboration, "a solution was found". The rest of the story describes in a much lighter vein how that solution worked in practice.
The story then jumps forward a decade or two. The next scene opens with a teenage girl walking out in the countryside of this planet going to visit someone. Evidently the colonists can now go for walks in the countryside. The viewpoint switches to that of the intelligent (but not very) telepathic predator who spies this juicy morsel. It attacks. The girl sees it coming towards her for a moment - I think it is described as looking like an ugly but somehow vaguely comic parody of a human yokel - then she shakes her head in disbelief and laughs at herself. For a moment something or other had looked like that old wives' tale of "the Nonesuch" but everyone knows they don't exist.
The poor old predator had been zapped into temporary nonexistence by telepathic feedback. The effect wore off after a while, but it decided not to attack any more humans.