The Quick Answer: Check the item’s type and description
There are two ways to know if a magic item counts as a magic weapon:
- It is a magic item of the type weapon
- Its ability to attack as a magic item is specified in the description.
Magic Items of Type: Weapon
From Sage Advice Compendium 2016 reads:
Every magic weapon can bypass resistances and immunities to damage from nonmagical attacks, but only certain magic weapons are more accurate and damaging than their non- magical counterparts.
That is, the mechanic from some earlier editions, where a magic item needed a particular bonus to damage a creature, is gone. Now, you just have to confirm whether an item is a magic weapon.
The type appears immediately below the name of the magic item in the block. (See Magic Item Categories, DMG p. 139) Swords, bows, and other weapons appearing in the weapons table of the Player’s Handbook are typically magic weapons.
Usually, but not always, the weapon will also be specifically called out as “this magic sword” or “this magic weapon.” (Oathbow is an example where the magic weapon is referred to simply as “this weapon.”)
Other Items that can be wielded as magic weapons
If the item's description specifies so, the item can be used to attack as a magic item. For example, the Staff of Power (DMG, page 203) can.
This staff can be wielded as a magic quarterstaff...
Some Enchanted Weapons are NOT magic weapons
In your example of a sword that has a magical effect, this may or may not be a magic weapon. If the item is from published Wizard's material, check the above criteria.
If it’s an custom item invented by your DM then you just need to ask (or if you are the DM, just decide). It would be fine to have “wondrous item” that happens to be a weapon. For example, a non-magic axe might have a magic handle (fashioned from wood of the rubber tree plant) that allows it to be changed into a hand axe, battle axe, or halberd.
Magic Items as Improvised Weapons
A magic item that is used as an improvised weapon, such as a wand being used as a club, functions no differently than any other improvised weapon as far as dealing damage. They are magic items but not magic weapons.
An item's Magic Item Resilience (DMG, p.141) may make an otherwise-delicate item usable as an improvised weapon.
(Regarding the specific example of Gauntlets of Ogre Power, I have to say, worn gauntlets don't feel like an improvised weapon to me, and I would use the unarmed attack rules, maybe making ruling to give a +1 to damage if I felt generous. Wearing gloves is not very improvisational. Gauntlets that are taken off and used to slap might be.)
Spells on Weapons
One final note, there are spells that can be cast on weapons that do not turn them into magic weapons for damage immunity considerations. Continual Flame (PH, p. 227) is an example.
If a spell makes a weapon a magic weapon, it says so. See Magic Weapon, PH p. 257.