TL;DR
To address your actual question: He was not able to read Leia's thoughts because she was not in the same state of emotional turmoil, AND not emotionally suffering because of specific information about specific person - which is what enabled him to read Luke's mind.
Details:
One does not simply walk into Mordor read someone's mind with the Force. There's no generic "Force Mind Read" ability I'm aware of in canon (at least pre-TFA).
Instead, you can get a glimpse, in very specific circumstances.
ROTJ novelization by James Kahn explains what happened when Vader sensed about Leia, and it was not nearly as straightforward as "read Luke's mind and found out he has a sister": he was sensing things associated with strong feelings Luke was experiencing at the time. Extremely amplified by Leia being shot at just that time, which Luke sensed:
He was picking up additional feelings from Luke, now, new crosscurrents of doubt. Remorse, regret, abandonment. Shades of pain. But somehow not directly related to Vader. To others, to … Endor. Ah, that was it—the Sanctuary Moon where his friends would soon die. ...
Note that he reads emotions, not thoughts, and then logically deduces which thoughts they came from based on circumstances. Then:
“Give yourself to the dark side, Luke,” he entreated. “It is the only way you can save your friends. Yes, your thoughts betray you, son. Your feelings for them are strong, especially for — ”
But... we just saw in the last paragraph, he did not sense Luke's thoughts. Just feelings. Vader simply misspoke.
Vader stopped. He sensed something.
Luke withdrew further into shadow. He tried to hide, but there was no way to hide what was in his mind—Leia was in pain. Her agony cried to him now, and his spirit cried with her. He tried to shut it out, to shut it up, but the cry was loud, and he couldn’t stifle it, couldn’t leave it alone, had to cradle it openly, to give it solace.
Vader’s consciousness invaded that private place.
“No!” screamed Luke.
Vader was incredulous. “Sister? Sister!” he bellowed. “Your feelings have now betrayed her, too … Twins!” he roared triumphantly. “Obi-Wan was wise to hide her, but now his failure is complete.” His smile was clear to Luke, through the mask, through the shadows, through all the realms of Darkness. “If you will not turn to the Dark Side, perhaps she will.”
