The only organised physical activity in the Harry Potter books seems to be the Quidditch Cup. The teams for that are very small (even assuming a reserve for every position, there only 14 players per house times 4 houses - that's not a lot of students). Aside from flying lessons with Madam Hooch, which seem to only happen in first year (To what extent are flying lessons provided at Hogwarts?) - is there any other activity for students who don't make it onto the Quidditch team? Do they just give up? Are training sessions provided by Madam Hooch to help people train for try-outs? Is there a B-league that's never shown? Do they play any other sports, or have PE lessons, or is running away from fellow students' curses considered sufficient exercise?
I've read the books and never found any reference to non-Quidditch-Cup exercise, but perhaps there is something from an interview or Pottermore that touches on this - any suggestions?