Reasoning
The existing answers show us that the game rules don't address the question. One way to handle this is to consider the abstractions that the rules use, and think about how those relate to our own mental processes.
We all have memories of understanding things that we've now forgotten. That seems like a reasonable model for having had a temporary proficiency, but not having it any more. It's also clear that you can write things down while you have a temporary proficiency, but it's not clear what the writings do once the proficiency has gone.
Handling it
The way I'd deal with this case is to consider the information that was written down: not the actual words, which deal with things that aren't real and we can't describe in any detail as players, but its purpose.
What did you write down, and how clearly did you write it? If you just wanted to know a simple specific fact, then sure, you recorded it and can access it later. This is no different from a player taking notes of things that happened in the game.
As for recording a lot of information, that gets harder. The more you try to write down, the more you'll have to use jargon that you understand while you have the proficiency, but probably not afterwards. Thinking you understand the jargon when you don't will likely lead you into utterly wrong information.
If a player wants to exploit this, I'd let them copy from the potion descriptions, or similar rules, writing by hand. I'd limit them to five minutes, because modern paper and pens are a lot easier to use than the typical ones available in the game setting. That will get them descriptions of the things they know about, but it won't get them subtler stuff, like how that potion from this maker generally works fine, but don't eat cabbage after taking it, unless you want to spend the evening throwing up.
Learning
If you can use a temporary proficiency frequently, without anyone getting annoyed with you about it and stopping you, then you can plausibly use that to teach yourself the proficiency. Of course, the DM needs to be happy about that, since it's an extrapolation of the brief rules under "Training" on p. 187 of the 5e Players Handbook, which only cover languages and sets of tools.
It's hard to suggest a general rule about the time and cost, since it will depend on how much of the time you can use your temporary proficiency. If you only have it for ten minutes a day, self-training is going to take longer than if you can use it for ten minutes an hour. You don't have to pay someone to train you, but if using the proficiency requires materials, you'll have to buy those.