Where did Dumbledore go during Order of the Phoenix?

Question

Following his eviction (of sorts) from Hogwarts it isn't clear where Dumbledore heads off to. The only place he rules out is Grimmauld Place.

"Where will you go, Dumbledore?" whispered Professor McGonagall. "Grimmauld Place?"
"Oh no," said Dumbledore, with a grim smile. "I am not leaving to go into hiding. Fudge will soon wish he'd never dislodged me from Hogwarts, I promise you."
_{(Order of the Phoenix, Chapter 27, The Centaur and the Sneak)}

Yet we don't see Dumbledore again until he turns up at the Ministry at the book's end. There are various places he could have gone to escape the Ministry but few that would be in keeping with his intention not to go into hiding.

I found a discussion thread that mooted the concept that Dumbledore went Horcrux-hunting, which would at least explain how he was able to destroy the Peverell ring so quickly that summer. However, that would not present any sort of problems for Fudge. Dumbledore explicitly says that he will set out to make life difficult for Fudge. So presumably Dumbledore would go somewhere in keeping with that aim.

So is there any confirmation on where Dumbledore went? Indeed, is there any indication that he actually did anything which made Fudge regret dislodging him from Hogwarts?

Answer 1

We don't really know

We know that the Ministry had no idea where Dumbledore was, as Umbridge told Professor Marchbanks near the end of the year:

“Journey was fine, journey was fine, we’ve made it plenty of times before!” she said impatiently. “Now, I haven’t heard from Dumbledore lately!” she added, peering around the hall as though hopeful he might suddenly emerge from a broom cupboard. “No idea where he is, I suppose?”

“None at all,” said Umbridge, shooting a malevolent look at Harry, Ron, and Hermione, who were now dawdling around the foot of the stairs as Ron pretended to do up his shoelace. “But I daresay the Ministry of Magic will track him down soon enough . . .”

—Harry Potter and the Order of the Phoenix

This suggests that whatever his method of making Fudge regret expelling him from Hogwarts, it was nothing too public, or the Ministry would have had some idea of where he was. Further, Harry, Hermione, and Ron would presumably have heard of it.

We also know that, although Dumbledore was not hiding out at Grimmauld Place, he occasionally showed up there, likely continuing to use it as his base:

Dumbledore heaved a great sigh and then said, “Alastor Moody, Nymphadora Tonks, Kingsley Shacklebolt, and Remus Lupin were at headquarters when he made contact. All agreed to go to your aid at once. Professor Snape requested that Sirius remain behind, as he needed somebody to remain at headquarters to tell me what had happened, for I was due there at any moment. In the meantime he, Professor Snape, intended to search the forest for you.

—Harry Potter and the Order of the Phoenix

That said, even though we have little information to go on, I think we can take a decent guess. If Dumbledore was not publicly working against Fudge, as the first quote indicates, he was probably working full-time to expose Voldemort. The end result of this would have been to humiliate Fudge and cast him from office (as indeed happened), a task to which Dumbledore would not have been able to devote much time had he been consumed in the day-to-day affairs of Hogwarts.

Answer 2

I think he did go Horcrux-hunting. The fact he shows up at the end of the book when Voldemort had been proven to be alive was enough to make Fudge wish he had not "dislodged" him from Hogwarts.