It's actually wrong to say that Star Wars followed in the footsteps of preceding movies.
Droidspeak, a.k.a. Binary, was in fact an original invention for Star Wars, the result of using an analogue synthesizer and mixing it with Ben Burtt's own voice. This was a deliberate move on the part of George Lucas, who wanted to differentiate Star Wars from Sci-Fi films of the preceding decades, much of whose sound effects were distinctly electronic, and who made Burtt go through several iterations until he came up with a sound for R2-D2 that was "organic" enough.
The fad of having electronic soundtracks for Sci-Fi movies has as its archetype Forbidden Planet, of course; although as noted Robby itself clatters mechanically.
It perhaps seems that there is a correlation simply because of fashion. Robby was very popular, and appeared in a lot of films and television programmes, so it may seem that that era depicted robots in "Robby-like" fashion simply because of the ubiquity of Robby itself.
Similarly, Ben Burtt spent 28 years at LucasFilm, and did a lot of sound design and editing, including famously two of the robots in WALL-E. So a lot of sound design may be seen to be "Burtt-like" simply because he did it. And of course, R2-D2 was an inspiration it its turn to others, just as it was inspired by Huey, Louie, and Dewey in Silent Running.
They, of course, were mute; and the simple truth is that there is no single characteristic "robot sound" in the history of Sci-Fi cinema, and no single origin for the many that there are. There are a lot of sounds: the mutes like Gort from The Day The Earth Stood Still, the mechanical noised, the human voices processed beyond recognition (Vincent in The Black Hole was a voice-controlled Korg synthesizer, and designed to sound like a servo-motor operating), the electronically synthesized, the human voices processed but still comprehensible like RoboCop, other things electronically processed (the voice of Max in The Black Hole was the sound of a growling panther run through a Vocoder, for example), plain old human voices (such as Robin Williams in Bicentennial Man), and so on.
Interestingly, there are, here and there, real robots, outwith fiction, whose design has been influenced by Sci-Fi robots; and in fact far from fictional robots being based upon real sounds (rather than, as is far more prosaically the case, whatever the sounds effects people could come up with and decided fit the film, from vocoded panthers to a Disney workshop bandsaw breaking) in some respects real robots have been modelled on the fictional ones. The designers of MIT's RoCo, for example, explicitly imitated the sound style of R2-D2.
- J. W. Rinzler (2010). The Sounds of Star Wars. Chronicle Books. ISBN 9780811875462.
- Thomas S. Hischak (2011). "Burtt, Ben". Disney Voice Actors: A Biographical Dictionary. McFarland. ISBN 9780786486946. p. 33.
- Paul M. Sammon (1980). Inside The Black Hole. Cinefantastique. Volume 9 Issues 3–4. pp. 4–63.
- Cynthia Breazeal and Rosalind Picard (2008). "The rôle of emotional-inspired abilities in relational robots". In Raja Parasuraman and Matthew Rizzo. Neuroergonomics. Human-Technology Interaction Series. Oxford University Press. ISBN 9780195368659.