Cantrips scale automatically with character level. Weapon attacks only scale with ability score and any extra attack features you might get.
How could the damage of weapon attacks keep up?
It is cantrips that can not keep up, if you get Extra Attack
Weapon attacks alone are not competitive, a 14th level Wizard does more damage with a cantrip than with a dagger1. Class features like Fighting Style and Extra Attack make weapon attacks superior.
First I will measure the Wizard against the Fighter, and later show how other classes compare. For simplicity assume every class has the same hit chance at the same level, and every class increases its attack stat at every possible chance.
The weapon used for the comparison is Rapier, Fighting Style is Duelling, if applicable.
Wizard vs Fighter
The best damaging Wizard cantrip2 (Firebolt) does 1d10 damage at the beginning. Compare this to 1d8+3 of a Rapier. This is a 36% advantage, with Duelist this goes up to 72%.
On level 4 the gap is increased again, as the ASI provides a hit bonus to both, but a damage bonus to weapons only.
At level 5 both values double, so the relative difference does not change.
At level 11 both the Wizard and the Fighter get a 50% improvement, it is more complicated for Paladins and Rangers, but the ratio remains about the same.
From level 17, Firebolt does 4d10 damage, that is 22 on average, while a Rapier in the hands of a Duelist Fighter is 3x(1d8+5+2)=34.5 (46 from level 20) damage, almost double.
They do not get Fighting Styles, and no other feature to support weapon attacks after Extra Attack on level 6. There the expected damage is 2 x (1d8 + 4) = 17, which increases to 19 when the attack stat reaches 20 (mostly level 8).
This is more than what a 16th level Wizard does (3d10 = 16.5).
Fighing Style and Extra Attack mean that even on level 5 they do more damage than a top level Wizard with cantrips, and Improved Divine Smite increases this even further. 8th level is 2 x (1d8 + 5 + 2) = 23, 11th level 2 x (1d8 + 5 + 2 + 1d8) = 32.
They are famous for using two weapons, so I will use it in the calculations. 5th level is 3 x (1d6 + 4) = 22.5, already better than a top level Wizard. Depending on Archetype, it increases a bit more, still the Ranger has lesser damage output than a Fighter or a Paladin.
Getting Sneak Attack damage is quite easy, unless all party members are ranged. At level 5 they can expect 1d8 + 4 + 3d6 = 19, which is already better than 16th level Wizards, and it keeps increasing without any cap, unlike Rangers or Paladins.
This specialist Wizard gets to add his Int modifier to damage rolls with Evocation cantrips from level 10. Firebolt now does 16 damage, still behind a 6th level Valor Bard. Next level it increases to 21.5, still behind a 5th level Fighter, Paladin or Ranger. At level 17, the damage is 27, less than a 11th level Fighter or Paladin, but better than a Ranger.
Same as Evoker, just the bonus damage comes on level 6 instead of 10. Still, this is only enough to beat the Valor Bard from level 11.
The big exception. They get to add their Charisma to each beam of Eldritch Blast, making it the most damaging cantrip in the game. It is only a bit behind a Fighter, gaining the same amount of attacks, but doing 1d10 + 5 per beam against 1d8 + 5 + 2 per strike. With Hex they even come out ahead, but a Fighter could similarly take a feat to compensate, so it would not be a fair comparison.
A Longbow does the same base damage as a Rapier, but the Fighting Style adds +2 to attack instead of damage. In most cases, this means an even higher DPR.
Many monsters are resistant to damage from non-magical attacks, taking half damage. A Fighter still beats a Firebolt, but a Ranger cannot. Paladins should just cast Magic Weapon.
This is one of the reasons why Magic Items have such a big influence on game balance.
Even the Valor Bard, the weakest of the primary weapon users is better with a Rapier than most casters with cantrips until level 17.
After level 11 blaster casters like an Evoker or a Dragon Sorcerer can beat this, but are still far behind any class that gets a Fighting Style.
This is clearly visible in this damage(DPR)-level chart .
Firebolt is in the graph, so I will compare the new cantrips to that.
Toll the Dead: It is great, at least better than Firebolt, unless you are fighting necrotic resistent creatures, and those are usually easy to recognize. Most casters tend to open with the big guns, and change later to cantrips, so I consider it a simple 1d12 damage, wich is 18% more than Firebolt, still not enough to catch up with the Ranger, the least damaging martial class.
Evokers with Potent cantrip really like this one, along with Clerics, who just received their first decent combat related cantrip.
Sword Burst (also Word of Radiance): 1d6 damage is really small, and with diminishing returns, you need to stand next to a lot of enemies to be competitive. Which is generally a bad idea.
If we calculate with full damage to the first target, half to the second, and so on, you need 4 targets to get ahead of Toll the Dead, which can be used from a safe distance.
Celestial Warlocks add their Charisma to the damage with Word of Radiance at 6th level, but it is still not enough to catch up with Eldritch Blast, especially if they have Agonizing Blast.
Green-Flame Blade: You can rely on this one (as opposed to Booming Blade). The problem is that while it requires an attack roll, it is not an Attack action, so will not enable for example Two-Weapon Fighting. The extra damage is small, about 1.5-2 on levels 1-4. On level 5, you deal 1d8 additonal damage to the primary target, and 1d8 + SAM (Spellcasting Ability Modifier), about 8.25, while the Fighter deals 10.5 twice.
Green-Flame Blade is only a good idea if you do not get Extra Attack, but are likely to hit with a weapon, like a Cleric.
Booming Blade: It is a trap. DnD5 is one of the most static games tactically3, so the extra damage will rarely trigger. The base damage is identical to Green-Flame Blade.
1) Until level 5 even casters are better off with weapons.
A usual Wizard with Int 16 and Dex 14 does on average 3.3 damage against AC14 with Fire Bolt, but 3,575 with a Light Crossbow.
2) Poison Spray does more damage, but it has horrible range, and the most commonly resisted damage type. Even a 20th level Wizard knows only 5 cantrips, and Poison Spray is just not good enough to be in the first 5 out of 23.
3) Unless you use the Flanking optional rule (and I have never seen it used), there is no incentive to move once you are in melee reach, but a lot of incentive to stay put, because of Opportunity Attacks
Cantrips Can't Keep Up
The flaw in your assumption is that it's only raw damage that matters. There are more factors involved, specifically the more attacks you make the more likely you are to hit.
For this edition of Maths Make My Head Hurt, we'll compare four class' DPR at the various break points, all spec'd for max damage: fighter, rogue, warlock, and wizard.
- Any class feature that increase base damage of the weapon/spell attack is taken as soon as possible
- Starting attack stat of 16
- No magic items
- No feats
- ASI goes into attack ability
- Attacks will be made against average AC of a CR = character level, as seen on DMG274
- The above 2 bullet points gives us a steady 65% to-hit chance
- Ignoring critical hits for simplicity
- DPR calculations: [hit%] x [avg damage] x [number of attacks]
Level 1 (AC 13)
- Fighter takes a longsword and the dueling fighting style. That gives them 1d8+5 (9.5 average) damage. DPR of 6.175
- Rogue takes a rapier 1d8+1d6+3 (11 average) damage (assuming sneak attack). DPR 7.15
- Warlock takes eldritch blast, giving 1d10 (5.5 average) damage. DPR of 3.575
- Wizard takes fire bolt, giving 1d10 (5.5 average) damage. DPR of 3.575
Level 5 (AC 15)
- Fighter adds action surge (double DPR when used) and extra attack. 1d8+6 (10.5 average) damage, twice. DPR of 13.65
- Rogue adds cunning action, so we can assume advantage due to hiding as a bonus action, which makes their accuracy 87.75%. 1d8+3d6+4 (15 average) damage (assuming sneak attack). DPR 13.162
- Warlock adds the agonizing blast invocation, granting +Cha to damage of eldritch blast, and can attack twice with it, giving a 1d10+4 (9.5 average) damage. DPR of 12.35
- Wizard adds the school of evocation, which doesn't do anything right now. 2d10 (10 average) damage. DPR of 6.5
Level 11 (AC 17)
ASI increases our to-hit ability score to 20(+5). This keeps our accuracy at 65% (87.75% for our rogue with advantage)
- Fighter adds and additional extra attack. 1d8+7 (11.5 average) damage, thrice. DPR of 22.425
- Rogue doesn't get any offensive-specific features. 1d8+6d6+5 (30.5 average) damage (assuming sneak attack). DPR 26.76
- Warlock can now attack thrice with eldritch blast, giving 1d10+5 (10.5 average) damage. DPR of 20.475
- Wizard gains empowered evocation, granting +Int to the damage. 3d10+5 (21.5 average) damage. DPR of 13.975
Cantrips Lag Significantly
As you can see, even without going all the way to level 20 cantrips start to lag considerably once the martial characters get the extra attack feature—or right away when considering rogue's with sneak attack.
Of course, this is just for cantrips. Higher level wizards can use their spell slots to cast more powerful spells that do a lot more damage. But fighters can action surge for more multi-attack actions, and paladin and rangers get some serious burst damage spells. And…and…and…ad infinitum
Warlocks keep pace pretty steadily, but that's because eldritch blast gets more attacks instead of more damage. Most likely this is intentional due to the rather limited number of spell slots that they get compared to other magic-using classes.