This question already has an answer here:
I'm trying to find the name of a novel (or was it a short story) where a group of humans travel to a distant planet and along the way (through genetics?) become more and more intelligent. I seem to recall that at some point they send back one of their who's a little less bright. Also, they send communications coded with some mathematical factorization that takes humans a long time to decrypt.