This is Kristin Cashore's Fire.
a girl who has special hair when men see it they want her
In the land of the Dells, there are "monsters." Impossibly beautiful animals of unnatural colors, that can entrance people with their appearance and control them with their minds. Lady Fire is the last human monster in existence.
I know it was a fantasy story in a medieval like context.
Definitely: castles and kingdoms and so on.
I remember she lived first in a hold with her childhood friend. Who was the son of the lord of the hold.
She grows up in a small corner of the kingdom together with Archer, the son of Brocker, the wheelchair-bound lord of the estate.
But then she needs to go on on a quest. Where she meets the king of the land and his brother who is the commander of his army.
With both Nax and Cansrel now dead, the new king Nash and his brother Brigan hope to return the country to order and stop civil war breaking out. [...] Brigan calls upon her, saying Nash has requested she come to the capital and use her powers to interrogate prisoners for useful information.
She falls in love with his brother.
Fire arrives at a Fort Flood which is under attack, and meets Brigan. The two confess they are in love with each other.
I also think she was an archer but I am not so sure about that.
Not quite: the archer wasn't her but her childhood friend and one-time lover.