There are no rules for worshiping gods, but likely a lawful good god
Aasimar are found in the Volo's Guide to Monsters and the relevant part of their description is below:
Aasimar bear within their souls the light of the heavens. They are descended from humans with a touch of the power of Mount Celestia, the divine realm of many lawful good deities. Aasimar are born to serve as champions of the gods, their births hailed as blessed events.
The description is "champions of the gods", gods in general with no specific reference made to a particular deity. The only guidance we are given is that Mount Celestia is home to mostly lawful good deities. The next section also assumes this:
Aasimar are placed in the world to serve as guardians of law and good. Their patrons expect them to strike at evil, lead by example, and further the cause of justice.
Although this is descriptive text the assumption is that most Aasimar will follow a lawful good deity. This doesn't have to be the case however, since the description goes on to mention their conflicted souls:
Despite its celestial origin, an aasimar is mortal and possesses free will. Most aasimar follow their ordained path, but some grow to see their abilities as a curse. These disaffected aasimar are typically content to turn away from the world, but a few become agents of evil. In their minds, their exposure to celestial powers amounted to little more than brainwashing.
Remember however, that this is only guidance and there is no restriction on the alignment, deity or class of any race in 5th edition. What gods exist in your world is up to your DM and you can work with them to decide which suits your character best. If you choose to be an archetypical aasimar that would likely be a lawful good god but you aren't required to follow this.
As for celestials, they are a creature type not a race. Therefore guidance on what god they are likely to follow is hard to find. The stat entries for most celestials are lawful good, though some (Pegasus) are chaotic good.
Generally celestials are good aligned and would worship a good or lawful good god if they worship one at all. Despite being basically angels themselves celestials are no more or less required to worship than any other creature type.