That's entirely a matter of writing your background story, and working with the DM so that it fits the world. I would go little further than to choose or roll the specialty, and the traits/ideals/bind/flaw, and talk to the DM about the rest.
The specialties have no mechanical function; they only give some info about what aspects of military life your character would know best. In designing your character, I would co-ordinate the specialty with the characters skills, either by selecting a specialty that matches the skills you chose, or by selecting skills to match the specialty you chose or rolled. e.g. If you were a "Healer" in the military, you should probably have the "Medicine" skill; if you were a "Scout", you should probably have proficiency in "Stealth"; if you were "Cavalry", you probably should be proficient in "Animal Handling"; etc.
Any other effects will be about roleplaying and what the DM allows. For example, if you are surveying a grounds that will soon be a battlefield, you might be asking the DM where the cavalry should station themselves and charge from. If you were in the Cavalry in your background, you will have a better chance of knowing or finding out the answer to that question.
But as to what military organization you might have been in, the only person who can tell you that is your DM.
To take an example from my experience, in one of my campaigns a fighter chose the soldier background, with the officer specialty. The problem for me was that the character is a Dragonborn. My campaign takes place on a sort of alt-Australia, and Dragonborn aren't even native to the continent -- they come from alt-NZ. She went fishing and got caught in a storm and blown to alt-Australia. So the player and I had to work out the story. We decided that she beached near a large port city, and due to her size and intimidating presence, she was hired by the Constabulary, and quickly became an officer due to her intelligence and tactical ability (she selected the battlemaster archetype when it was time for that). So her experience, and insignia, and military rank, and ability to requisition things, all are most relevant when dealing with local town or city constabularies, and not so much with armies or factions.
There's no way that player could have predicted any of that, or pre-made a character story that would fit in my world.