You only apply the modifier
Spiritual Weapon states the damage as (per PHB p. 278):
1d8 + spellcasting ability modifier
The spellcasting ability modifier is simply the ability score modifier for that attack - in this case, Wisdom, at your stated +3.
You can increase the damage dice by casting this at a higher spell slot.
When you cast this spell using a spell slot of 3rd level or higher, the damage increases by 1d8 for every two slot levels above the 2nd.
This is different than how you determine a successful attack
The use of proficiency bonuses never applies to damage - only to attack rolls.
Determining a successful attack involves rolling a d20 and adding both your ability modifier and proficiency bonus. Rules around this can be found on page 193 of the PHB.