They're found near the Temple and have lightsabers but that doesn't really answer my question, which is: Can Jedi Temple Guards use the Force?
According to the Star Wars Databank article on Jedi Temple Guards, the answer is a very firm "yes".
An ancient, honored heritage within the hallowed halls of the Jedi Temple was the role of the Jedi Temple Guard. Reporting to Cin Drallig, head of security, the Temple Guards were anonymous sentinels plucked from the ranks of the Jedi as part of a Knight's ongoing commitment to the Order. A Jedi Temple Guard would disappear inside the formal robes and identity-concealing masks, the ultimate expression of emotional detachment required in service to a higher calling. Temple Guards carried imposing lightsaber pikes -- thick, double-bladed weapons that produced a rare and distinctive yellow blade.
Temple Guardians were trained Jedi who were selected for their dedication and skill. They shouldn't, however be confused with the Temple Security Force whose ranks included recruits who were non-Force-wielding.
The Temple Security Force was the security police of the Jedi Temple and the Temple Precinct surrounding it on Coruscant. Though the name was traditionally and generally applied to the non-Force-sensitive soldiers and police officers, Temple Security also included the Jedi Peacekeepers and the Jedi Temple Guards that were stationed on Coruscant
Yes, in fact the Grand Inquisitor
used to be a Temple Guard, and we know that he could use The Force.
Additionally, from their Wookieepedia page:
The Jedi Temple Guards were part of the Jedi Order. They wore golden-brown robes with Jedi patterns on them to signify their loyalty to the order and carried special lightsaber pikes which produced yellow blades.
You have to be able to use The Force to be a Jedi.