The phrase "May the force be with you" does not exclusively belong to the Jedi. But members of the dark side seem to be reluctant in saying "May the force be with you/us" to each other. Why? Don't they want the force to be with them?
You can read the following on Wookieepedia
"May the Force be with you" was a phrase used to wish an individual or group good luck or good will, one that expressed the speaker's wish that the Force work in the favor of the addressee. The phrase was often used as individuals parted ways or in the face of an impending challenge.
Jedi / Light side
"Wish that The Force work in the favor of the addressee"
shows how the relation between that power and a Jedi works. The easy answer would be that for the Jedi, the Force is a power that does not belong to him. It's an energy that flows through everything, big and small, far and near, an energy that bind and guide him through his destiny. So the sentence is wishing for someone to have a good future planned by the force.
It would be like petting a cat, the Jedi way would be to sit and wait for the cat to come to him because he wants the light side user to pet it.
Sith / Dark side
On the other end of the spectrum however, the Sith has a different relationship with the Force. He does not want to follow what the Force has planned for him. Self-centered as he is, he wants to possess and control everything, be it the power, the resources of all the galaxy or his own destiny. He doesn't want to follow the will of the Force, he wants to tame it, to bend it to his will.
If he wanted to pet the cat from earlier, the Sith way would be to Force-Pull the animal to him and then throw it in a cage for later petting.
So the reason the Sith will never, NEVER use that sentence is because it goes against everything he believes in. It would be like acknowledging that there is something controlling him, something he has no power over and that is not a fact his pride and arrogance would ever accept.
Any answer is going to be opinion based
With that being Jedi see the force as a living being, the force is in everything, the force makes things happen for its own reasons.
The Sith ultimately see the force as a tool to use so saying "may the force be with you" would be like saying "may the screwdriver be with you"
see this answer Why do the Jedi say "May the Force be with you"?
This question is not quite a duplicate but its kinda close