I have read the short story you are describing. A bit of research suggests that it is probably one of the Marionettes stories by Adam-Troy Castro, probably The Funeral March of the Marionettes, or its sequel The Tangled Strings of the Marionettes.
What little I can remember: I think the dance of the alien species was intended as communication of some kind, and that the species was unable to communicate with humans in any way other than via dance, and even when they did, the messages were so garbled as to be almost useless. Humans that learned to speak the alien dance-language to any substantial degree seemed to think they were starting to understand it, and yet also become increasingly incomprehensible to ordinary humans.
The big ritual dance that you refer to was described by the aliens as being somehow crucially important for the universe, but they could never explain why in a way that humans could understand, and whether the dance was somehow artistically/aesthetically important or important in some other more tangible way. And I seem to remember that participants in the dance died afterwards.
(Note: According to here, The Funeral March of the Marionettes was nominated for both the Hugo and Nebula awards in the Best Novella category in 1998, as well as taking 16th place in a Locus poll. The Tangled Strings of the Marionettes was nominated for the Nebula awards for the Best Novella category in 2004, as well as being shortlisted for the Theodore Sturgeon Memorial Awards.)
(For the record, when searching for the story title, Google availed me naught, but I finally found a brief description and the story name on TVTropes for "Starfish Language" in the "Literature" section.)