I know I read it more than 40 years ago, in some collection of fantasy (and maybe also SF), but whether in English or French translation, I am not sure. It was certainly not a french original.
The "Through the Looking Glass" in the title of my question is just a joke, to describe the situation, but the author made no direct allusion to Lewis Carroll.
In the living-room there is a fireplace with a mantel above which there is a large mirror. The children (I think there are two of them, I think the younger is a boy, the older I'm not sure, and possibly more than two) are told never to stand on the mantel, it is narrow, they might fall and hurt themselves. One day the parents must go out, and they cannot get anyone to stay. So they just warn the children again.
But the boy says he has seen on various occasions shapes in the "room behind the mirror" that do not match things in "their" room, and he is sure they hide just behind the mantel "across"? By standing on "their" mantel he thinks he'll have a good vantage to see them. Against their parents order, he does climb. Form there he can indeed see one (or more) monster(s). Realising their secret is discovered the monster(s) cross the mirror and attack the children.
When the parents come home, there is no sign of breaking and entering but they find the horribly dismembered bodies of their children.
Because it is fantasy, and the time I read it, it is possible (but not at all certain) that this story is in the same collection as the one I asked some time ago about