I've been watching the old Star Trek again for the first time in 20 years or more. I am really impressed with the appearance of the ship in space, but more so I am impressed with the appearance of planets from space. They have a nice 3d appearance to them, the mountains and clouds have shadows, etc. How was this done in a special effects sense?
The original Star Trek episodes typically did not have that kind of detail on planets seen from space.
It's likely you're seeing the versions of episodes with the special effects re-done using CGI.
See this article, for example.
A typical view of the Enterprise in orbit around a planet from the original series can be seen here.
And here's a side-by-side (well, top-by-bottom) comparison of a shot of the Enterprise in Earth orbit.