First, if you are thinking that this question is a joke or troll, I'd like to welcome you to the 21st century when the mankind is discussing legality of gay marriages.
Now, if you look at the sexuality (I'll keep it clean) of all the Avengers minus Captain America:
Iron Man: Well, he is a billionaire playboy who can bring hot female journalists to bed with a snap of a finger (Iron Man movie). Otherwise, he has a girlfriend who is CEO of a multi-billion dollar company.
Thor: He has a Nobel prize winning girlfriend who doesn't belong to his species.
Hulk: Dr. Banner had a girlfriend, but then the gamma accident happened which could turn him into a green monster upon arousal (The Incredible Hulk movie). It's still unclear whether he can now have sex or not (after he controlled himself with meditation) and his girlfriend is still with him or not, but he isn't a virgin and nobody knows what happened after Age of Ultron. It's possible that Black Widow successfully seduced him after he was found again.
Black Widow: Although, she is single and she has been sterilized, it's difficult to believe that she is a virgin and she doesn't have sex regularly because she is seen kissing around. And, she was trying to seduce Dr. Banner (Avengers: Age of Ultron movie). The point is: She is sexually active.
Hawkeye: He is married (Avengers: Age of Ultron movie).
Now, look at Captain America:
When Peggy kissed him in Captain America movie, it was their first kiss and then he got iced.
Fast forward to 21st century: Although his age is 80+, he is still young. Yet, he doesn't have a girlfriend and he doesn't show any interest in girls. Even when Black Widow kissed him, he didn't try to drag her to bed.
Talking about low libido (a medical condition which arises due to stress and other factors), it's impossible because his super soldier serum should keep him healthy in all way. From the functional biology point of view, low libido is actually caused by imbalance of hormones. Stress suppresses rest-and-digest autonomous nervous system and activates fight-or-flight autonomous nervous system (that's why stress, depression etc damage health). In case of Captain America, yes, he got negative emotions after he was defrosted, but his super soldier serum should take care of his biological health. So, one of the other possibilities: He is gay (which has been proved to be a natural condition, not some kind of abnormality).
To prove that he is gay, let's look at his emotional responses: In 21st century, when he met Peggy, he didn't become emotional. But, when he saw Bucky, he became emotional.
I am a straight male and I am creative enough to emulate the situation well. If I had met my first love in such situation, I would have become emotional and it'd actually hurt to know that she is old and I am young. Talking about a dead best friend who is somehow alive and attacking me, my natural response would start with a slang. By no means, Captain America's emotional response looks straight man's response and it looks a gay's response.
Is Captain America really gay? Am I overthinking or producers are really hinting something to throw later? These days it's common in media for awareness. For example, Flash (2014) TV show has a gay police officer.
A more general question: Has sexual orientation of Captain America been disclosed? If he isn't gay, is he straight (there are lots of sexual orientations e.g. Bisexuality)? What about comics?