"Inside John Barth", a novelette by William W. Stuart, first published in Galaxy Magazine, June 1960, available at the Internet Archive. The story is also available at Project Gutenberg. You may have read it in the 1961 anthology The Fifth Galaxy Reader edited by H. L. Gold.
It starts when he's on a hunting trip:
We were lounging by the campfire after supper and a few good snorts. Uncle John was entertaining himself with a review of some of his nearer, more thrilling misses. I, to tell the truth, was sort of dozing off.
Then, all of a sudden, there was a bright flash of blue-green light and a loud sort of a "zoop-zing" sound. And a sharp, stinging sensation in my thighs.
I hollered. I jumped to my feet I looked down, and my pants were peppered with about a dozen little holes like buckshot. I didn't have to drop my pants to know my legs were too. I could feel it. And blood started to ooze.
He's used to shaving with a safety razor, not a straight razor, but even that is too risky for the microscopic colonizers:
I always had used an ordinary safety razor — nicked myself not more than average. It seemed OK to me. Never cared too much for electric razors; it didn't seem to me they shaved as close. But — I took to using an electric razor now, because I had to.
One workday morning I dragged myself to the bathroom of my bachelor apartment to wash and shave. Getting started in the morning was never a pleasure to me. But this time seemed somehow tougher than usual. I lathered my face and put a fresh blade in my old razor.
For some reason, I could barely force myself to start. "Come on, Johnny boy!" I told myself. "Let's go!" I made myself take a first stroke with the razor. Man! It burned, like fire. I started another stroke and the burning came before the razor even touched my face. I had to give up. I went down to the office without a shave.
The alien microbes communicate with their host:
"Now, now, noble Fatherland. Please, do not become upset and unreasonable. We honor you greatly as our home and country. Surely we who were born and raised here have our rights. True, our forefathers who made the great voyage through space settled first here in a frightful wilderness some four generations back. But we are neither pioneers nor immigrants. We are citizens born."
His health improves:
It was the first winter since I could remember that I never caught a cold. A cold? I never once sniffled. My health was perfect; never even so much as a pimple. My dandruff and athlete's foot disappeared. I had a wonderful appetite — which was lucky, since I didn't have much other recreation left. And I didn't even gain weight!
He gets drunk when the aliens let up their guard:
"You are in the South Side Hospital, Mr. Barth. You will be all right — which is a wonder, considering. Remarkable stamina! Please tell me, Mr. Barth, what kind of lunatic suicide pact was that?"
"Suicide pact?"
"Yes, Mr. Barth. Why couldn't you have settled for just one simple poison, hm-m? The lab has been swearing at you all day."
"Uh?"
"Yes. At what we pumped from your stomach. And found in the girl's. Liquor, lots of that — but then, why aspirin? Barbiturates we expect. Roach pellets are not unusual. But aureomycin? Tranquilizers? Bufferin? Vitamin B complex, vitamin C — and, finally, half a dozen highly questionable contraceptive pills? Good Lord, man!"
He killed off all the colonists:
Now what? What should I do? I turned, as always, inward for advice and instructions. "Folks! Why didn't you stop me? Why did you let me do it? And now — what shall I do? Answer me, I say. Answer!"
There was only an emptiness. It was a hollow, aching sensation. It seemed to me I could hear my questions echoing inside me with a lonely sound.
I was alone. For the first time in nearly ten years, I was truly alone, with no one to turn to.
They were gone! At last, after all these years, they were gone. I was free again, truly free. It was glorious to be free — wasn't it?
He wants them back. They have established a secondary colony in his Aunt Helga:
"Nurse!" I shouted at the top of my voice. "Nurse, come here. I want to send a wira Rush. Urgent. To my aunt, Mrs. Helga Barth, the address is in my wallet. Say, 'Helga. Am desperately ill, repeat, ill. Please come at once. I must have help — from you.' "
She’ll come. I know she will. They’ve got to let her. It was an accident, I swear, and I’m not too old. I’m still in wonderful shape, beautifully kept up.