As with all Paladins, mine has a holy symbol. What is its purpose? Does it being inscribed on my sword hilt have any bearing?
Your holy symbol is your spellcasting focus
In addition to being, you know, a symbol of your beliefs and tenets which define your character as a Paladin.
The last subfeature of your Spellcasting feature reads:
You can use a holy symbol (see chapter 5, "Equipment") as a spellcasting focus for your paladin spells.
Said spellcasting focus can be used to provide non-costed, non-consumed material components for your spells. (See Chapter 10 of the PHB; Spellcasting > Casting a Spell.)
Excatly how it will work being engraved on your sword hilt you'll have to ask you DM about (I personally would expect "yeah, that's fine". The issue is that the two options discussed by the rules are discrete items, or on a shield, but not on a weapon. The relevant rule for the DM:
To use the symbol [as a spellcasting focus], the caster must hold it in hand, wear it visibly, or bear it on a shield.
— Player's Handbook, p. 153
If that's valid as holding in your hand, or as a simple flavour mod from having it on your shield, it's a no problem spellcasting focus for you.