Miles is, at this point in his life, something of an adrenaline junkie. He's fiercely proud of all the work he's done as Admiral Naismith with the Dendarii. Up to this point in the series, he's a very hands-on leader, and always in the thick of the action. It's won him numerous accolades, albeit classified ones. As he tells Haroche when he goes to him as Imperial Auditor:
I didn't clean out my father's desk drawer, if that's what you're asking. Everything here is accounted for, in my classified files. You may be one of the few men on the planet who doesn't have to take my word for it.
Earlier, after his ice bath, Miles tells Ivan and Galeni:
You can see why I was afraid it would be a one-way ticket to a medical discharge, at worst. A desk job at best, and no more Dendarii Mercenaries, no more field work.
A little later in the conversation:
Miles's brows rose in startlement. "Impossible. Being a desk driver requires very different qualities than being a field agent. A diametrically opposed attitude to the rules, for starters. I'm not... wasn't nearly ready for Illyan's job."
"Chief of ImpSec at age thirty-five. Huh. God be praised, I'm saved from that at least."
To be fair, this was all before Miles's voyage of self-discovery that takes place over the course of the novel. I'm quoting it here because I think it's still relevant. Just because Miles comes to a new understanding of who he is doesn't mean he's fundamentally changed.
So when he tells the emperor:
[Miles:] "I'm not playing hard-to-get. It's a desk job stuffed with the most tedious routine, in between the terror-weeks, and the chief of ImpSec not only almost never gets off-planet further than Komarr, he scarcely ever gets out of Cockro— out of ImpSec HQ. I would hate it."
[Gregor:] "I think you could do it."
"I think I could do almost anything I had to do, if you ordered it, Gregor."
I believe him. He's not ready to settle down in one spot yet. He would have the terror and adrenaline rush when ImpSec was dealing with a crisis, but it would, in many respects, be a remarkably helpless sort of terror. He would be stuck behind a desk, directing divisions of analysts and agents, and waiting for what came back. He would not and could not be in the thick of the action like he was as the Little Admiral. The rest of the time, he'd be looking over reports from various departments, not commanding a mercenary fleet looking for action.
He wants to serve the emperor, yes, but he knows that there are people better suited to the job than he is. Because he knows he would hate it, he probably would not be able to do the job to the best of his ability, and therefore he would not truly be serving the emperor. Allegre is much better suited and prepared for the job, and Miles wholeheartedly recommends him for the position.
Contrast this to the position of Imperial Auditor. Every case is different, and auditors are used when normal means of problem solving have failed. This allows an individual auditor to get stuck into the problem, and tackle it head on, just like Miles did as Admiral Naismith, and allows him to serve the emperor by solving some of the thorniest Imperial problems.