Whosoever holds this hammer, if he be worthy, shall possess the power of Thor
which means that you have to be "worthy" in the eye(s) of Odin to use the hammer..
You can't wield Mjolnir if you're not worthy yourself, but Is it possible to stop Thor from wielding his hammer?^. i.e. Can you stop the hammer from going to Thor?
# Give or take an instance or two where he's unworthy, and then proves that he's worthy after all.
^ I don't mean making Thor unworthy, by doing something like, corrupting him or something, but can someone physically stop Thor from grabbing his hammer, without holding his arms or anything?