Just an out-of-the box suggestion, disregarding magic items :
Disguise self does trick vision, but not the other senses.
To start rumors, you could use :
- Dogs intrigued by the specific scent, coming to sniff at your character - and running in fear once they touch you.
- A local blind man, intrigued by either the unusual scent, or the loud steps heard on the pavement - asking around if an ogre just arrived in the town.
- This testimony could even be approved by a local huntsman, who found a disproportionate footprint in a muddy street.
Once rumors have spread, someone curious with high investigation and the "observant" feat should be enough to pierce your disguise consistently.