That depends on what you mean by "ward off" zombies.
In season 1, while Rick is talking to Andrea in the department store, they establish that walkers track potential prey by three means: motion, sound and smell. Apparently, walkers can distinguish living people from others of their kind by those three senses.
For Michonne, dragging around her two ex-friends worked because it masked her (living) scent among their death stench, and camouflaged her sounds and movements as part of the crowd. As long as she stayed relatively quiet and didn't break out into a run or something, other walkers nearby wouldn't notice her among the other two dead-heads.
She took off their jaws - and their arms, BTW - in order to prevent the two of them from attacking her. That part had no effect on warding away other walkers around them, it was just to prevent these two from biting or scratching her while she was distracted by something else.
(Sweet action figure replica to provide reference without being too gory for younger readers)
If you put walkers around a building, would that ward off other walkers?
It might help to deter other walkers from going inside, but the noise these ones make could also attract other walkers to meander about the perimeter, so it wouldn't be a perfect defense. These walkers would be in "wander" moe rather than "close and attack" mode, but any loud noises inside or sudden movements near a window might draw their attention and turn things sour. Also, with you inside the building and the "guards" outside, you wouldn't pick up their smell, so you wouldn't have that defense if/when you need to leave the premesis.
So yes, it can be a valid strategy for home defense, but it definitely should not be your ONLY strategy.
In fact, we've seen a few variations of this "wall of zombies" actually being used in the show:
From Morgan's hideout in Whatever City That Was, where he had all the zombies skewered on various poles and wooden traps, to the current season where we see Neagan's group forcing the "points-system" crew to chain up zombies in various positions around the perimeter of their HQ (assuming that was their HQ - it may not be.)
Also worth noting that putting up a string of "disarmed" walkers around your chosen hidey hole won't do much to protect you against the REAL threat out there - other people. In fact, it may even draw attention from other people scouting the area, so you would be well advised to have other defenses in place as well.
Would you have to cut off their jaws for it to work, is that the magic trick?
Cutting off the jaws and arms of your "guards" is not required, no.But it might still be wise, for the same reason Michonne did it. It's not "magic," just common sense. In case some of your "guards" wander inside, or try to lash out at you as you're moving in and out, removing their arms and jaws would make it harder for them to injure or kill you. It would also make them less aggressive, as the Governor's assistant noted while examining Michonne's friends in season 3 (I think it was.)
On the other hand, leaving your "guards" intact would make them a credible threat to other humans trying to get inside. So, I suppose you could go either way depending on how concerned you are about getting in and out through the guards yourself.