It was an anime where the lead character has a mecha suit that's composed of 3 robots, one bigger than the previous, the smaller one fits into the medium one and this fits into the bigger one. I think each one had different abilities.
This is the 1979 anime TV series Gordian. The protagonist of the anime pilots the robot Protesser, which nests itself inside the larger robot Delinger, which finally enters the largest robot, Garbin, in the same way, forming Gordian. Various cutaway illustrations showing how the three robots move together are an iconic representation of the series.
Among anime robots, this particular gimmick is actually almost unique to Gordian. One of the only other similar robots is Baikanfu, from the 1986 anime Machine Robo: The Revenge of Cronos. However, Baikanfu does not consist of three robots in the same way as Gordian, nor does it have a human pilot in the center. The cast of Machine Robo are mainly living robots, and the protagonist of the series, who is an android, takes the place of the Protesser robot from Gordian. So the image seen in the Revenge of Cronos anime is of the protagonist merging with the giant robot Kenryu, which then merges into the larger robot Baikanfu.
Since the series you saw had a pilot who uses three robots, and because each robot's unique abilities were shown off, it's almost definitely Gordian.