It seems to me that your player came with some, let's say non-standard character for you that may be certainly valid if you fill in some blanks. Let's break that PC down on elements and find out if his backstory fits.
Has the soldier background and is a fighter
Your PC was a soldier that quite reasonably knew how to wield a sword or other weapon at the time he was serving in army.
Character that is a farmer
He has later on retired to rustic livestyle. That still seems valid as a whole.
He also wants to be a charismatic historian
Which doesn't make any sense considering he's a farmer.
Here is where you are disagreeing with your player, but as far as I look there is no contradiction in what he currently does as occupation and his hobbies present or past. If his character pumped up intelligence maybe he was not merely a frontline soldier, but had commanding position. It not only required wits, but also charisma to be trusted by his fellows and be promoted. He may have spent his time studying history in his free time as officer, damn even as a farmer he could find it interesting to read books of past times when not working on a field.
I think that you should try to broaden your horizons and just come with sensible story with your player. Do not dismiss his ideas, but ask questions. When did he read history books? What position he held at army? What does he mean be being charismatic, are you likeable person or the one other won't ignore (be it good perception or bad)?