I read this as a kid in the 70s or 80s. It had a man (Census Taker, Reporter, Author?) visiting an unusual family. I seem to recall they were VERY advanced for time frame the story was set in. They might even have had supernatural abilities.
The only other thing I can remember is one of the youngest children was chanting (singing?) what the man thought was a nonsense verse. When he was leaving however, he remarked that the chant sounded like the theory of relativity ie.. E=MC2 etc... (ee em see to...)
It was NOT the Hogben series, they were more a typical suburban family.
This has bugged me for years! OCD can suck sometimes. :-) I have done numerous net searches to no avail, hopefully someone out there can shed some light.