As far as I know there is no official word on this but I don't think we need official confirmation to come to the conclusion that yes he was trying to poison her.
In the Inside the Episode D.B. Weiss says the following which hints at a poisoning attempt, how else would he "overthrow" her without an army? Of course it could be argued he's talking about Varys having secret conversations with Jon and sending the letters but there's more to it than that.
D.B. Weiss: I think Varys knew that it was unlikely that he would survive the attempt to overthrow Dany in favour of Jon. And he also knew that ethically in his mind he had no choice but to try to do that anyway.
Game of Thrones, Season 8 Episode 5, "The Bells" - Inside the Episode
In Episode 4, Tyrion's reaction to seeing Varys' face after asking what will happen to her is enough evidence on its own, in that moment both characters know Varys will attempt to kill her. And it's shown enough that we the viewers should have picked up on it.
Varys: You know where my loyalty stands. You know I will never betray the realm.
Tyrion: What is the realm? A vast continent, home to millions of people, most of whom don't care who sits on the Iron Throne.
Varys: Millions of people, many of whom will die if the wrong person sits on that throne. We don't know their names, but they're just as real as you and I. They deserve to live. They deserve food for their children. I will act in their interest, no matter the personal cost.
Tyrion: So, what happens to her? Please. Don't.
Varys: I've spoken as honestly as I can. Each of us has a choice to make. I pray we choose wisely.
Game of Thrones, Season 8 Episode 4, "The Last of the Starks"
Then lets look at the Martha conversation again, they're "trying again" at supper which isn't necessarily odd in itself but then Martha is worried that the soldiers are watching her. Why would she be worried if all they are doing is bringing her food? Varys says it's their job and again he probably would word that differently if it wasn't for the poisoning attempt.
Lastly, examine "The greater the risk, the greater the reward." Why would there be a risk involved in simply bringing her food if there wasn't more to it? It isn't just implied it is strongly implied that Varys is trying to poison her through the use of Martha. In fact it's all but spelled out for us.
Varys: And? Nothing?
Martha: She won't eat.
Varys: We'll try again at supper.
Martha: I think they're watching me.
Varys: Who?
Martha: Her soldiers.
Varys: Of course they are. That's their job. What have I told you, Martha?
Martha: The greater the risk, the greater the reward.
Varys: Go on. They'll be missing you in the kitchen.
Game of Thrones, Season 8 Episode 5, "The Bells"