When I was in my teens, so it must have been in the seventies, I read a science fiction story (I do not remember whether it was a novel or a short story) that contained a displayed mathematical formula. I remember it as looking like an infinite sum of a series of fractions, but when I showed it to my math teacher he pointed out that that did not make sense (the result of the formula interpreted as a series could not have been rational? or maybe it would be divergent? something like that), and that it was supposedly an infinite continued fraction that the publisher did not typeset properly.
I think it might have been a formula by Ramanujan, but I do not remember whether the story named Ramanujan explicitly, or that we recognised the formula as something like that ourselves (I would guess the latter).