Helping out on a Twitter request, original here.
YA book. A family visiting a historic castle while on a trip gets separated and falls through time into King Arthur times. The mom doesn't realize they've gone back, keeps being like “I was told there would be a cream tea” and literally never figures it out.
The cream tea is a VITAL detail, unquestionably present.
Twitter request states she's been looking for this book for ~20 years. And that:
DISCLAIMER: you will want to say “Half Magic” by Edward Eager, but it is not.