The very start of the latest trailer for the upcoming Ender's Game movie features Mazer Rackham and Colonel Graff sitting across the table from one another:
Rackham: He's not ready.
Graff: You're never ready. You go when you're ready enough.
I don't recall the two of them ever having a conversation in either Ender's Game or Ender's Shadow. Or am I misremembering?
There is at least one - after the final battle, both Graff and Mazer speak to Ender briefly. They explain (among other things) why the commander had to be a child, kept in the dark about the fact that the ships and men were real. After Ender turns away and tries to go to sleep, Graff and Mazer argue about whether they pushed him too far, and Mazer ends the conversation by essentially stating "I did what I did, and we won. You can criticize my methods, but you can't prove I did anything I didn't have to do." (This defense is later used by Graff in his court-martial.)
There is a different conversation - just before the final battle. While he's sleeping, he hears two voices:
During the night he thought he felt hands touching him gently. Hands with affection in them, and gentleness. He dreamed he heard voices.
"You haven't been kind to him.”
"That wasn't the assignment.”
"How long can he go on? He's breaking down.”
"Long enough. It's nearly finished.”
"So soon?”
"A few days, and then he's through.”
"How will he do, when he's already like this?”
"Fine. Even today, he fought better than ever.”
In his dream, the voices sounded like Colonel Graff and Mazer Rackham. But that was the way dreams were, the craziest things could happen, because he dreamed he heard one of the voices saying, "I can't bear to see what this is doing to him." And the other voice answered, "I know. I love him too." And then they changed into Valentine and Alai, and in his dream they were burying him, only a hill grew up where they laid his body down, and he dried out and became a home for buggers, like the Giant was.
It's uncertain who the voices are (though I always believed they actually were Mazer and Graff).
For completeness, they actually first spoke when Mazer was still in a courier ship flying at relativistic speeds away from Earth, and Graff was a green young Lieutenant assigned to find the next commander for Third Invasion. Pretty much entire short story "Mazer in Prison" is of how they first communicated to each other. It's implied that they were in regular communication since then since Mazer was Graff's main trusted "consultant" on what to look for in a commander and how to train him.
@Adam's excellent answer lists all the conversations in Ender's Game proper.
Graff and Mazer were certainly in contact throughout the short story entitled "Mazer in Prison" which is where (and when) they first met.
