In Fantastic Beasts: The Crimes of Grindelwald, when Ministry officials asked Dumbledore to fight Grindelwald, Dumbledore refused saying he couldn't. The result of this conversation was that Dumbledore got a surveillance spell put on him and he lost his post of Defense Against the Dark Arts teacher.
Why didn't Dumbledore simply tell the truth why he couldn't fight Grindelwald? It could be one full sentence: "I can't because of a blood pact."
Dumbledore seems to be somewhat ashamed of what happened between him and Grindelwald and those years of his life. It's enough that people know of their close relationship but to admit they made a Blood Pact is even worse. He just isn't ready to admit it publicly, least of all to someone like Travers who he doesn't like and seems to have a bit of history with.
DUMBLEDORE is looking at the pictures. These memories are agony. He is full of remorse but, almost worse: nostalgia for the only time in his life he felt fully understood.
TRAVERS: Will you fight him?
DUMBLEDORE: (pained) I can’t.
Fantastic Beasts: The Crimes of Grindelwald - The Official Screenplay
Because admitting such a pact existed instantly makes him seem even more of a threat. It leads to questions about his loyalties: to perform a blood pact with someone indicates a deep and trusting relationship.
Is he a secret spy, working with Grindlewald?
Does he still have the same feelings that he had when he formed the pact?
Even if the pact was ended there would be suspicion that he and Grindalwald had worked together to insert him as a double agent. Remember there are huge issues here around trust and an understanding already that Grindelwald is manipulative and runs schemes inside schemes.
