The question AT-ST at the Battle of Hoth in the original version of Episode V? makes it clear that the AT-ST, or "chicken walker", was indeed in the original release of The Empire Strikes Back. My question is, why - why was there a teeny-tiny little cameo of the AT-ST in Episode V when we don't get properly acquainted with them until Episode VI? It seems weird to me; you either feature them properly in the battle, or wait for the next movie, you know? Has Lucas or anyone commented on the reasoning behind the decision to include the AT-ST this way?
Crafters at ILM had made an AT-ST, and George Lucas found the design as 'neat' and included it The Empire Strikes Back. This required ILM to change it to add in metal work for stop-motion animation. Probably the reason that this was made so small is that it wasn't intended to be in the movie. Between movies they look slightly different (though we could just say that was a change for different climates).
While there is no other explanation on 'why' outside of this, it does show off that the Empire has lots of various units for a ground war. From a military standpoint, it makes sense to include the AT-ST, as it is better at anti-infantry and can move faster. It can help take out the smaller targets faster, and protect the larger AT-ATs.
The original model for the AT-ST was created by members of the Industrial Light & Magic team, and George Lucas thought it was "neat" and decided to include it in The Empire Strikes Back alongside the AT-AT, so the ILM model shop disassembled it and added a metal stop-motion armature.
The AT-ST model received some cosmetic changes for its extensive appearance in Return of the Jedi, thus explaining the minor differences in the models used for the two films.
"It seems weird to me; you either feature them properly in the battle, or wait for the next movie, you know?"
No. To me, it seems more natural to include more things than are featured, to establish a setting that has details that aren't focused on, but are still part of the setting, just as the real world has many things present that aren't the focus of a story that would be set there, but would be weird if they weren't there.
It also makes sense to me that a mechanized force would have some variety in its equipment.
As loose on realism as George Lucas may be in some respects, one of the features of his films is incidental details and things making some sort of sense, and trying to portray settings that have some reality and interesting incidental detail.
For examples, consider these minor details from Lucas' first three Star Wars films, and realize there are more:
- All sorts of weird droids who look intriguing but are incidental.
- All sorts of weird characters in Mos Eisley who are incidental.
- Space chess.
- Y-Wing fighters.
- Various rebel weapons in the defensive trench.
- Various bounty hunters, most of which barely do anything, even Boba Fett.
- Cloud City police ships.
- Various incidental characters at Jabba's palace, and their equipment/race/etc.
- Assorted varieties of spaceships and fighters on both sides of the space battle.
- Assorted miscellaneous unexplained classes of Imperial guards and advisors.