Was it because he didn't want to be cursed by Mandos, or he didn't wish to endure the ice desert aka the Helcaraxe? And why did all his kids leave to go with Fingolfin instead of staying with him? Was it his conscience that made him stay in Valinor?
According to Tolkien Gateway:
Finarfin, dismayed by the prophecy and already contemplating return because of the tragedy of the Kinslaying of his wife's people at Alqualondë, returned to Valinor with a small group of his people; his sons, though, would not forsake the sons of Fingolfin and went on.
This is sourced to the Silmarillion, "Quenta Silmarillion: Of the Flight of the Noldor".
Interestingly it seems this story wasn't always told in the same way:
In the Silmarillion, after the Kinslaying at Alqualonde and the Prophecy of the North, Finarfin and many of his people returned to Valinor and were pardoned by the Valar (p.88); but [in an earlier version of Tolkien's notes] those few who went back found there was no welcome for them, or else "Mandos has them" (p.168).