If the party earns a Dragon Slayer Sword, it can be a type of weapon used by one of the party members but it says it must be a sword, so would a Glaive count?
No, glaives aren't swords, and halberds aren't axes. Putting a blade on the end of a pole makes it fundamentally a different weapon. After all, a spear is not a dagger!
That said, if you're the DM (and you're not playing in Adventurers League), you can make substitutions in published adventures' treasure as you see fit. If you want to have the party find a Dragon Slaying Glaive instead of a Dragon Slaying Sword, do it. That sounds pretty cool.
By strict RAW, a glaive can't be Dragon Slayer. On pg. 206 of the DMG are Sword of Life Stealing and Sword of Sharpness; Life Stealing has the (any sword) tag, while Sharpness (any sword that deals slashing damage). So, a shortsword could be Life Stealing, but not Sharpness. That said, there is nothing stopping the Game Master from tweaking the RAW to accommodate Rule of Cool. Also, it's a bit of an oversight that there aren't more clear definitions of the weapons like in 4e (i.e. does a scimitar or rapier count as a (sword) even though sword is not in their name or properties)