After reading Valorum's answer and the film's script, I'd like to post an alternative analysis:
Yes, they are flying. The same original script that Valorum posted says it explicitly:
Just then Buzz banks under some power lines and soars upward
again. Woody takes a peek.
WOODY: Hey, Buzz!! You're flying!!
Because he was indeed flying, he then answered
BUZZ: This isn't flying. This is falling -- with style!
This presented the viewers with the opposite situation to what happened when Buzz made his "flying" demonstration in Andy's room. Then, he didn't fly at all, which is why Woody said
WOODY: That wasn't flying! That was falling with style!
Obviously, Buzz's flight is an element of fantasy that is only shown in that critical, final scene, as Buzz can't fly in any other moment in any movie.
In reality, Buzz wouldn't be able to fly, but he probably wouldn't be able to glide either. In reality, they would probably just plummet to the ground. If the movie is giving Buzz a fantastic ability in that scene, Why simply make him "glide" instead of "fly" given that a critical aspect of the script is Buzz thinking he can fly?
Also: I'm not a native English speaker, so I'm not sure about how is the word "soar" used in the US, but Oxford Dictionary says that a definition of "soar" is "fly or rise high in the air". I don't think they used "soar" with the other meaning (gliding).