We know from the 2nd book that it is not the sword Caesura he received from the Adem for it looks completely different and I believe he even said so. I am just hoping I did not miss anything related to it.
My best guess is that it is an inside joke for Kvothe. The most poignant use of the word folly in my opinion is what Abenthy inscribed in the book "Rhetoric and Logic"
"Kvothe, Defend yourself well at the University. Make me proud. Remember your father's song. Be wary of folly. Your friend, Abenthy."