It looks like in the movie they are flying with the "handle" to the rear. But I've seen illustrations where it looks like they are flying with the "handle" forward. So which is it?
The image you've posted is actually a (previously) unpublished illustration from Potter cover artist Mary GrandPré. In other images the keys vary between bow-first and pin-first as well as upright. There seems to be no consistent decision on which direction they face or what the wings should look like.
HP:PS Concept Art
Pottermore moment - Chapter 16
Illustrated Harry Potter
Interestingly, in the Harry Potter and the Philosopher's Stone video game, the keys fly bow first, with the bit facing upwards. Weird.