The spell was cast without words, simply a flick of the Death Eater's wand, and a 'streak of purple flame' that lanced across her chest. It was later mentioned that the spell was weakened because of the lack of a verbal component, but still 'did enough damage as it was'. She was in the infirmary healing from the damage for some time. What spell was this?
The spell that Dolohov hit Hermione with was actually purple:
But the Death Eater Hermione had just struck dumb made a sudden slashing movement with his wand; a streak of what looked like purple flame passed right across Hermione’s chest. She gave a tiny ‘Oh!’ as though of surprise and crumpled on to the floor, where she lay motionless.
Order of the Phoenix - page 698 - Bloomsbury - chapter 35, Beyond the Veil
I looked through the spells at the Harry Potter Lexicon and the one spell that seemed possible is a revulsion jinx. The revulsion jinx repels and emits purple light when cast. This is the spell Hermione used on Yaxley in Deathly Hallows to shake him off the trio, when he had grabbed hold of Hermione as the trio attempted to Apparate from the Ministry of Magic to No twelve Grimmauld Place after they infiltrated the Ministry and retrieved the Slytherin locket from Umbridge (Deathly Hallows, chapter 13, The Muggle-born Registration Commission). A revulsion jinx could be considered an offensive spell, probably as well as a defensive one.
It's interesting that Order of the Phoenix indicates a non-verbal spell might be less effective than a verbal spell. Snape doesn't mention this when he teaches non-verbal spells in Half-Blood Prince.
My best guess? A compression curse or crushing curse. Hermione's ribs were painful, it may be designed to inflict internal damage but fortunately it was diminished thanks to the silencing charm.
"She winced slightly and put a hand to her ribs. The curse Dolohov had used on her, though less effective than it would have been had he been able to say the incantation aloud, had nevertheless caused, in Madam Pomfrey's words, "quite enough damage to be going on with."
Still it must have been powerful enough to severely strike her and knock her out. It was further blunted by Harry's Shield Charm, yet it managed to knock him off his feet. Definitely a curse meant to strike very violently at the victim, perhaps violently enough to kill on the spot.
It is a spell probably created by : 1) Voldemort , because in the "Harry Potter & the Philosophers stone" there was the Snapes task and there was a purple flame which you could cross after drinking the right potion and since Dolohov, Snape , Barty CrouchJR and the Lestranges were probably the most loyal followers he couls've taught it to them; 2)Antonin Dolohov bc he is a good spell creator so he could mix the relashio and fiednfyre, incendio or confringo