I think History is sort of a confusing term for this stat, because -- as you note -- it can go down, and how do you get less history with someone? What it really means, as per page 103, is how well you know someone. So maybe I met you yesterday, but it was the most important life-changing day you've ever had and I spent the whole thing with you. I could easily know you very well, and accordingly have a high History. Or I could have known you my whole life, but you're just that guy who lives in the next shack over, and I never knew you were an axe murderer. Low History.
So if I change a lot, you don't know me as well.
This doesn't entirely satisfy me as an explanation, given that my History stat with someone can go from +4 to +1 at the end of a session, and there are mechanical incentives to make that happen. On the one hand, I can decide that I don't want to make that move because I don't know how to play it out in the fiction. On the other hand, then I'm passing up advancement.
In play, I'd want to link that change to whatever advancement I was thinking about. I know when I change in real life, my opinions on people change, and sometimes I look at them in ways that make it harder to empathize. Hey, I just got a gang -- now I'm thinking about them more than I'm thinking about you, and I've kind of lost touch with you. I've lost my knowledge of you, because I'm not exercising it.
Short form: "It's not you, it's me." Which is a cliche but there's something in it.
All that said, +4 to +1 isn't always directly linked to an advancement, since it doesn't always generate the fifth circle of experience. I think this still works as a method of foreshadowing your expected changes, though.