Usually, this would just be a matter of description, not rules
3.5 doesn’t generally get into the details of the style and construction of weapons. A single weapon covers a large number of real-world weapons with the same stats. A longsword could be any number of one-handed straight blades. A rapier covers just about all of the thrusting swords. A bastard sword, notably, is also explicitly a katana. This is just a longsword with a distinctive style. I wouldn’t generally expect it to have special rules. This is what I recommend.
Dungeon Master’s Guide II agrees
Dungeon Master’s Guide II introduces the idea of “signature traits” which are purely-cosmetic effects applied to items when made by a crafter who knows that particular signature trait. One of these traits is
Barbed: The item has jagged hooks or spines.
This explicitly has no effect on the item.
But if you insist that the sword should be mechanically special somehow...
If it’s magical, it might have wounding
Wounding is a +2 special weapon property, and wounding weapons are described as being covered in barbs and the like. If this sword is valuable enough that it might count as a +3-equivalent weapon, it might be a +1 wounding longsword. Wounding is quite good, adding 1 Constitution damage to your attacks with it.
But I didn’t gather from your description that this was something valuable enough to be a +3-equivalent weapon. What you could do, however, is say the sword has an affinity for wounding. This concept, introduced in Dungeon Master’s Guide II, applies a 10% discount on the cost of raw materials necessary to add the given special property with an Item Creation feat. So wounding would cost a crafter 10% less to add to this sword. The rules do not suggest that you’d see any discount when hiring someone to add wounding to the sword, but hopefully your DM is willing to work with you on that.
As for mundane effects, though...
There are no rules for a barbed longsword
They have never published such a weapon, or rules for generically modifying a given mundane weapon to have barbs.
There are rules for a barbed dagger
Complete Adventurer has a “Dagger, Barbed” entry on page 115:
Barbed daggers, as their name implies, look like normal daggers with long barbs covering the blade.
(Despite this description, the illustration has barbs on the crossguard, not the blade.)
It grants you a +2 damage bonus as long as you have 5 ranks in Sleight of Hand (to twist the knife). It is also an exotic weapon, and despite otherwise being identical to a dagger, it has lost the ability to be thrown or to do slashing damage.
This is, basically, worthless. Exotic Weapon Proficiency is very expensive, and the ranks in Sleight of Hand are very problematic for most characters. And +2 to damage is just not that valuable. The dagger going from piercing-or-slashing damage to piercing only is also quite awkward for a longsword, which only has slashing as an option (inane though that is).
But the piercing/slashing issue does offer a mote of inspiration: piercing-and-slashing damage is a nice thing, overcoming a fair amount of DR. Still not worth a feat (a morningstar has nearly identical stats with bludgeoning-and-piercing damage, and it’s simple), but still, nice. Maybe the barbs make the sword deal piercing-and-slashing damage. Toss in the +2 damage bonus for the 5 ranks in Sleight of Hand if you like; I doubt you have them and if you do, you deserve the pittance of damage.
Final recommendation
Barbed signature trait, affinity for wounding, piercing-and-slashing damage, and +2 to damage if you have 5 ranks in Sleight of Hand. Clearly better than a regular longsword, but even if you have the ranks its expected damage is 0.5 less than the executioner’s mace (which can be bludgeoning-and-piercing or bludgeoning-and-slashing, which is way better), so this is still a reasonable martial weapon. Figure these upgrades make the weapon worth a few hundred gold pieces more than a typical masterwork longsword; call it an even 1,000 gp instead of 315 gp?