I remember reading a story about some guy who makes a phone call and is connected to someone he doesn't know. They agree to meet, but can't find each other at the agreed meeting place. When they speak on the phone again, it turns out they are living in different time lines, one in which there has been a war, and another where there hasn't. If memory serves in the end they are reluctant to hang up for fear of never getting to speak to each other again.
The story is "Out of this World" by Alfred Bester.
This is the collection I read it in originally.
Your plot summary is spot on.
Summary from Science Fiction Runinations:
Man fantasizes about women who call in with the wrong number. A particular women piques his interest enough to call her back. They set up a meeting. But, both miss each other. The phone line is due to be repaired. They try to meet again — and again, miss each other. Oh, crossed phone lines leads to a call to an alternate world… Silly.