Or does it happen always - when a Muggle finds out something - poof! - memory erased?
Parents of Muggle-borns are allowed to retain their memories, and we see Hermione's parents in Diagon Alley.
"So you don't think I'm a match for Lucius Malfoy?" said Mr. Weasley indignantly, but he was distracted almost at once by sight of Hermione's parents, who were standing nervously by the counter that ran along the great marble Hall, waiting for Hermione to introduce them.
Harry Potter and the Chamber of Secrets, chapter 4
And we know that the Dursleys also have memories of the Wizarding world, and Petunia knew all about magic when Lily was in school:
"...I'm sorry, Tuney, I'm sorry! Listen -" She caught her sister's hand and held tight to it, even though Petunia tried to pull away. "Maybe once I'm there - no, listen, Tuney! Maybe once I'm there, I'll be able to go to Professor Dumbledore and persuade him to change his mind!"
Harry Potter and the Deathly Hallows, chapter 33
"Knew?" shrieked Aunt Petunia suddenly. "Knew! Of course we knew! How could you not be, my dratted sister being what she was? Oh, she got a letter just like that and disappeared off to that - that school - and came home every vacation with her pockets full of frog spawn, turning teacups into rats. I was the only one who saw her for what she was - a freak! But for my mother and father, oh no, it was Lily this and Lily that, they were proud of having a witch in the family!"
Harry Potter and the Sorcerer's Stone, chapter 4
See also @Colin__s's answer, which has another excellent example.
As we learn from Chapter 1. The Other Minister in the Half Blood Prince, the muggle prime minister of the day is told of the ministry, and magic, with nothing to suggest they're oblivated when they leave office.
In the sixth book they prove, that he was NOT obliviated:
It was precisely this sort of behavior that made him dislike Fudge's visits so much. He was, after all, the Prime Minister and did not appreciate being made to feel like an ignorant schoolboy. But of course, it had been like this from his very first meeting with Fudge on his very first evening as Prime Minister. He remembered it as though it were yesterday and knew it would haunt him until his dying day.
Harry Potter and the Half-Blood Prince,chapter 1
An obvious exception are the families of Muggle born Witches and Wizards. Lots of Muggle borns are mentioned in the books such as Hermione, Dean, Ted Tonks etc. We see that Hermione's parents were exposed to the Wizarding Community, visited Diagon Alley and even met the Weasleys.
Another instance we see is that of the Dursleys. We don't know how informed Vernon was about Magic before Harry ended up at their doorstep but he knew enough to refer to the Potters as "their lot".
Comment: Are those the only exceptions? Could a Muggle do something important that they'd be seen as worthy of keeping their memories? – Elisa Elisija
Reply: Yes. The only instance I can think of is... Becoming the Prime Minister. From the interaction observed between Cornelius Fudge and the British Prime Minister we see that though former PM knew about the Wizarding Community, he was not obliviated when his term ended in the office. The assumption that no one would believe a word about magic is considered a sufficient safeguard for this