John Varley's Hugo-winning 1984 novella "Press Enter ■" wasn't written by Asimov, but it was published in Isaac Asimov's Science Fiction Magazine (May 1984 issue, available at the Internet Archive); could that be what you were thinking of?
The story sort of fits the meager description you gave. As I seem to recall from reading it 30 years ago, it's not exactly the phone exchanges per se that become sentient, it's all the computers in the world, communicating over the telephone lines. Or something like that.
I couldn't find a useful review, but I found my copy of the story (in Varley's collection Blue Champagne), so I'll copy out a few quotations. You should be able to tell from them if this is the story you're looking for.
"This is a recording. Please do not hang up until—"
I slammed the phone down so hard it fell onto the floor. Then I stood there, dripping wet and shaking with anger. Eventually, the phone started to make that buzzing noise they make when a receiver is off the hook. It's twenty times as loud as any sound a phone can normally make, and I always wondered why. As though it was such a terrible disaster: "Emergency! Your telephone is off the hook!!!"
Phone answering machines are one of the small annoyances of life. Confess, do you really like to talk to a machine? But what had just happened to me was more than a petty irritation. I had just been called by an automatic dialing machine.
Some plot explanation:
"The connections. Again, it's different, but the concept of networking is the same. A neuron is connected to a lot of others. There are trillions of them, and the way messages pulse through them determine what we are and what we think and what we remember. And with that computer I can reach a million others. It's bigger than the human brain, really, because the information in that network is more than all humanity could cope with in a million years. It reaches from Pioneer Ten, out beyond the orbit of Pluto, right into every living room that has a telephone in it. With that computer you can tap tons of data that has been collected but nobody's even had the time to look at it.
"That's what Kluge was interested in. The old 'critical mass computer' idea, the computer that becomes aware, but with a new angle. Maybe it wouldn't be the size of the computer, but the number of computers. There used to be thousands of them. Now there's millions. They're putting them in cars. In wristwatches. Every home has several, from the simple timer on a microwave oven up to a video game or home terminal. Kluge was trying to find out if critical mass could be reached that way.
I live by candlelight, and kerosene lamp. I grow most of what I eat.
It took a long time to taper off the Tranxene and the Dilantin, but I did it, and now take the seizures as they come. I've usually got bruises to show for it.
In the middle of a vast city I have cut myself off. I am not part of the network growing faster than I can conceive. I don't even know if it's dangerous to ordinary people. It noticed me, and Kluge, and Osborne. And Lisa. It brushed against our minds like I would brush away a mosquito, never noticing I had crushed it. Only I survived.
But I wonder.
It would be very hard . . . Lisa told me how it can get in through the wiring. There's something called a carrier wave that can move over wires carrying household current. That's why the electricity had to go.
I need water for my garden. There's just not enough rain here in southern California, and I don't now how else I could get the water.
Do you think it could come through the pipes?