It’s likely Scrimgeour was killed by Imperiused Ministry officials.
It’s most likely that the way Scrimgeour was killed was by the Ministry officials who were Imperiused. Yaxley implies this is the plan while talking to the Dark Lord.
“Yes – my Lord, that is true – but you know, as Head of the Department of Magical Law Enforcement, Thicknesse has regular contact not only with the Minister himself, but also with the Heads of all the other Ministry departments. It will, I think, be easy, now that we have such a high-ranking official under our control, to subjugate the others, and then they can all work together to bring Scrimgeour down.”
- Harry Potter and the Deathly Hallows, Chapter 1 (The Dark Lord Ascending)
This is also what the Dark Lord himself was planning, since he says that Imperiusing enough people to ensure the Minister is surrounded by their people before they try to kill him was crucial.
“It is a start,’ said Voldemort. ‘But Thicknesse is only one man. Scrimgeour must be surrounded by our people before I act. One failed attempt on the Minister’s life will set me back a long way.”
- Harry Potter and the Deathly Hallows, Chapter 1 (The Dark Lord Ascending)
Their plan was to use the Imperiused officials to kill Scrimgeour, which is likely what they did.
It was done silently, so wouldn’t be an open Death Eater invasion.
Lupin, a member of the Order who would have been informed of what happened, described the takeover of the Ministry as happening virtually silently. This would mean that the Dark Lord and the Death Eaters wouldn’t have gone to the Ministry building to overthrow the Minister.
“But surely people realise what’s going on?’
‘The coup has been smooth and virtually silent,’ said Lupin. ‘The official version of Scrimgeour’s murder is that he resigned; he has been replaced by Pius Thicknesse, who is under the Imperius Curse.”
- Harry Potter and the Deathly Hallows, Chapter 11 (The Bribe)
Though some people suspected, due to the changes in Ministry policies, that the Dark Lord was behind it, he stayed hidden and did nothing to openly prove that he took over the Ministry.
“However, that is the point: they whisper. They daren’t confide in each other, not knowing whom to trust; they are scared to speak out, in case their suspicions are true and their families are targeted. Yes, Voldemort is playing a very clever game. Declaring himself might have provoked open rebellion: remaining masked has created confusion, uncertainty and fear.”
- Harry Potter and the Deathly Hallows, Chapter 11 (The Bribe)
Since they were claiming Scrimgeour resigned, and didn’t want to show they’d taken over the Ministry, they couldn’t overthrow him in any way that’d make it obvious that they had. Therefore, Scrimgeour wouldn’t have been killed by the Dark Lord or the Death Eaters showing up at the Ministry to kill the Minister since then it’d be obvious he was killed and didn’t really resign. There wouldn’t have been a duel, at least not one where many people at the Ministry could see it.
Kingsley may have been at the Ministry at the time when it fell.
It’s not known how many, if any, Ministry officials tried to defend Scrimgeour, but the possibility exists that Kingsley could have been at the Ministry when it fell. He knew right after it fell and sent a warning, so it’s likely he was somewhere he’d be in a position to know - possibly the Ministry itself.
“Then the Patronus’s mouth opened wide and it spoke in the loud, deep, slow voice of Kingsley Shacklebolt.
‘The Ministry has fallen. Scrimgeour is dead. They are coming.”
- Harry Potter and the Deathly Hallows, Chapter 8 (The Wedding)
He was an Auror for the Ministry, so he worked there at the time, and would have a reasonable excuse for his continued presence at the Ministry. Since he was also in the Order, and knew the Dark Lord and the Death Eaters were trying to take over the Ministry, it’s possible he was trying to protect Scrimgeour when the Ministry fell, which could be how he knew so quickly that it did.
Scrimgeour was rumored to have been tortured before his death.
It’s possible that Scrimgeour was tortured for information on where Harry was before he was killed. Though it’s never said whether it’s true, Arthur Weasley heard it as a rumor.
“Arthur heard a rumour that they tried to torture your whereabouts out of Scrimgeour before they killed him; if it’s true, he didn’t give you away.”
- Harry Potter and the Deathly Hallows, Chapter 11 (The Bribe)
Since the Death Eaters didn’t know where Harry was, either Scrimgeour didn’t tell them despite being tortured, or the rumor wasn’t true - but it’s unclear which it is.
Some people at the Ministry remained not-Imperiused after it fell.
There were at least some people who worked at the Ministry who remained not under the Imperius Curse when it fell. Kingsley wasn’t Imperiused, though it’s unclear if he stayed at the Ministry after it fell, as the next time he’s mentioned he’s on Potterwatch and possibly in hiding. Arthur Weasley, who continued to work at the Ministry after it fell, also wasn’t under the Imperius Curse.
“They had agreed that it was far too dangerous to try to communicate with Mr Weasley while he walked in and out of the Ministry, because he was always surrounded by other Ministry workers. It was, however, reassuring to catch these glimpses of him, even if he did look very strained and anxious.”
- Harry Potter and the Deathly Hallows, Chapter 12 (Magic is Might)
Arthur clearly wasn’t Imperiused, since he resists the new Ministry policies the Dark Lord created.
“So, Dirk Cresswell is ten times the wizard you are,’ said Mr Weasley quietly, as the lift sank ever lower. ‘And if he survives Azkaban, you’ll have to answer to him, not to mention his wife, his sons and his friends –”
- Harry Potter and the Deathly Hallows, Chapter 13 (The Muggle-Born Registration Commission)
How many other officials weren’t Imperiused isn’t clear, but there was at least one who was both not under the Imperius Curse and still working at the Ministry after it fell.