In BSG (2005), season 4, episode 13, Baltar calls Felix in the CIC, and mentions "our little secret, sealed with a special pen." Similarly, when Felix visits Baltar in his cell (I can't remember the episode number), Baltar whispers a secret in Felix's ear. Does anyone know what the secret(s) they're referring too?
You have to watch this webisode The Face of the Enemy to learn what the secret is.
If you want the SPOILER read below
Baltar was aware of Gaeta's secret relationship with a dishonest copy of 8 which lead to a lot of civilian deaths.
he whispered: "I know what your Eight did"
The Battlestar Galactica "Face of the Enemy" webisodes (which are pretty much impossible to find anywhere these days - and if I'm wrong on this, I'd love to know where to get them - ETA: I'm dumb. They're all on Hulu.com as it turns out) are definitely meant to be the answer to this question. In the commentary to the webisodes, writer Jane Espenson said that they wanted to resolve this unanswered question from the Season 3 episode "Taking a Break From All Your Worries" (the one you refer to where Baltar whispers a secret to Gaeta and Gaeta stabs him), about what Baltar said to Felix. However, the webisodes are not the originally-intended answer to your question.
Originally, according to DVD commentary by the show's creator Ronald Moore, Baltar and Gaeta's secret was meant to be part of a larger storyline in Season 3 that was cut. In Moore's commentary, he explains that a story was going to come out during Baltar's trial of pre-Cylon Occupation atrocities that happened on New Caprica involving Saggitarons being massacred (Saggitarons, as may or may not know, are sometimes referred to as a racial group in the BSG 'verse that are frequently marginalized and discriminated against for their religious-based beliefs in pacifism and the avoidance of using modern medicine).
The backstory that was supposed to come out during Baltar's trial is not well explained by Moore, but he mentions that the atrocities were somehow going to involve Zarek being culpable and possibly Baltar too, or at least Baltar was going to have not done anything about it at the time. The secret that Baltar was intended to be whispering in Gaeta's ear in that episode was that he was going to falsely pin responsibility for the Saggitaron murders onto Gaeta, who had had nothing to do with it. However, Moore eventually determined that the Saggitaron storyline was so terrible, that they should cut it out, even though some of the episodes featuring it had already been filmed. So they cut out all references to this storyline that they could in whatever had already been filmed, but they could not manage to cut out this bit with Baltar and Gaeta's secret without compromising the integrity of this particular episode, so they had to leave it in.
(My pet theory is that this is also one of the reasons why around this time, we have these random scenes of Zarek repeatedly urging Roslin against public trials for Baltar and any other people accused of having committed atrocities on New Caprica, and even recommending that Roslin declare martial law, which is somewhat out of character for him. I think the idea was supposed to be that he didn't want this info to come out in public trials. But I have no proof for this.)
Anyway, hope that helps.