It depends on what you want to classify as "magic"
The rules distinguish between two types of magic. There is the "background magic" that allows for ghosts and fire-breathing dragons, and there is "focused magic" that creates spell effects.
If we assume that either of the above definitions is magic then only Force, Necrotic, Radiant and Psychic damage are clearly magical, where as most of the other damage types could be produced via nonmagical means as described here:
Acid damage can be caused by acid (obviously). A flask of acid is the most obvious means of achieving this (even though such concoctions are typically described as 'alchemical', its easy to assume this is nonmagical).
Cold damage can be dealt by extreme cold environments.
Fire by fire (as you said)
Lightning damage could conceivably be dealt by getting hit by, well, lightning. Although the chances of this happening without magical means in a typical campaign setting is negligible. Although you may argue that getting hit by lightning is still nonmagical even if it occurs due to being in a more fantastical setting (like a Plane of Eternal Storms or something).
Poison is also generally considered nonmagical, although magical poisons do exist. Many monsters have poisonous stings or bites, and this is usually considered nonmagical.
Thunder damage, caused by a 'sonic attack', is another one that is conceivably nonmagical but it is hard to imagine any nonmagical means of producing such a thing in a fantasy setting.
For the other types:
Force is inherently magical.
Necrotic and radiant being, in a sense, 'unholy' or 'holy' energy is also magical in nature as far as I'm aware.
And lastly, Psychic is also clearly magical (or 'psionic' if you are differentiating but still magic to all intents and purposes).