Sounds like the Night Visions episode, "A View Through the Window" (2001).
Night Visions is an American television horror anthology series, with each episode comprising two half-hour stories dealing with themes of the supernatural or simply the dark side of human nature. It was produced by Warner Bros. Television for the Fox network, originally airing from 2001 to 2002. Musician Henry Rollins was the uncredited host of the show.
In "A View Through the Window", Major Ben Darnell is a brilliant physicist and a grieving father, whose estranged wife blames him for the tragic accidental death of their little son. He's called up to a top secret location, where the military is investigating a strange supernatural phenomenon. A portal to another place has somehow appeared in the middle of the desert and an invisible barrier is blocking it. The soldiers can see the people on the other side - a seemingly happy pilgrimesque family that's living in a picturesque farmhouse surrounded by a luscious green field, but the family can't see the portal or what's on the other side, so they're living their idyllic everyday life like normal. Darnell is stricken by them, since they represent everything he wishes for in his tragic life. He eventually realizes that the barrier disappears for a few seconds each day and decides to risk everything to cross over to the other side despite his commanding officer's direct order not to try anything until they learn more about these people and their world.
You can view the scene with the ice cubes at around the 9:37 mark in the video below. You can view the scene where one of the soldiers goes through the portal and is attacked by the people on the other side at around the 18:15 mark.
As per this answer, this story was apparently based on a 1980 short story by Bob Leman, titled "Window".