Yes! You can use a superior weapon without a proficiency feat.
The +2 or +3 proficiency bonus is what you get for being proficient with that weapon. So if you use a weapon without proficiency you don't apply the bonus.
From Weapons MM283
Characters can pick up and use a monster’s weapons unless otherwise noted. If the weapon is listed in the Player’s Handbook or it it is a larger version of a PH weapon, use the rules for it found there. Otherwise, the characters lack proficiency with it, and the weapon deals its listed damage but gains none of the other effects that a monster gains from it.
So why is proficiency important:
4e operates on a pretty narrow margin. The difference in being proficient and not being proficient is pretty significant. it's 10-15% less of a chance to hit every swing. That adds up, if you're dealing on average, 20 damage on average and monsters have 80HP, and your chance to hit is 80%, reduce that to 65% and you're now dealing 16.25 and killing a monster in 5 rounds instead of 4. That means your taking one more hit every battle and dying that much quicker.
Note that implement proficiency works differently, you can't use an implement if you're not proficient in it. This is a bit of a difference and comes from the fact that implement attacks primarily target Non-AC-Defenses (NADs). Instead of giving them a bonus, monster defenses are lowered (NADs are typically 12+level of monster where AC is typically 14+level of monster). The math works out about the same here with heavy and light blades (typically +3) getting a one up on everyone in exchange for a bit of damage.