Vancian magic is memorization-based
A Vancian magic system is one where a wizard memorizes a spell and then loses that memorization from his brain after a single use. It's termed "Vancian" magic because it's directly modeled on the depiction of magic in the works of Jack Vance, like The Dying Earth (1950). From the first two short stories in that book:
These were volumes compiled by many wizards of the past, untidy folios collected by the Sage, leather-bound librams setting forth the syllables of a hundred powerful spells, so cogent that Turjan's brain could know but four at a time.
Turjan found a musty portfolio, turned the heavy pages to the spell the Sage had shown him, the Call to the Violent Cloud.
Then he sat down and from a journal chose the spells he would take with him. What dangers he might meet he could not know, so he selected three spells of general application: the Excellent Prismatic Spray, Phandaal's Mantle of Stealth, and the Spell of the Slow Hour.
For all Mazirian's magic he was helpless. The mesmeric spell had been expended, and he had none other in his brain. In any event he could not have uttered the space-twisting syllables with that mindless clutch at his throat.
The 14th appendix of the AD&D Dungeon Master's Guide (1979), Appendix N, directly credited Vance as an influence.
The most immediate influences upon AD&D were probably de Camp &
Pratt, REH, Fritz Leiber, Jack Vance, HPL, and A. Merritt; but all of the above
authors, as well as many not listed, certainly helped to shape the form of
the game. For this reason, and for the hours of reading enjoyment, I
heartily recommend the works of these fine authors to you.
Gygax described D&D "Vancian" magic as early as 1976
Even this might not be the true first and oldest use of the term, but Gygax himself used it in TSR's The Strategic Review Vol. II No. 2 (April 1976):
The Dungeons & Dragons Magic System
by Gary Gygax
Because there are many legendary and authored systems of magic, many
questions about the system of magic used in D & D are continually raised. Magic
in CHAINMAIL was fairly brief, and because it was limited to the concept of table
top miniatures battles, there was no problem in devising and handling this new
and very potent factor in the game. The same cannot be said of D & D. While
miniatures battles on the table top were conceived as a part of the overall game
system, the major factor was always envisioned as the underworld adventure, while
the wilderness trek assumed a secondary role, various other aspects took a third
place, and only then were miniatures battles considered. So a somewhat different
concept of magic had to be devised to employ with the D & D campaign in order to
make it all work.
The four cardinal types of magic are those systems which require long conjuration with much paraphernalia as an adjunct (as used by Shakespeare in MACBETH or as typically written about by Robert E. Howard in his “Conan” yarns),
the relatively short spoken spell (as in Finnish mythology or as found in the superb
fantasy of Jack Vance), ultra-powerful (if not always correct) magic (typical of
deCamp & Pratt in their classic “Harold Shea” stories), and the generally weak
and relatively ineffectual magic (as found in J.R.R. Tolkien’s work). Now the use
of magic in the game was one of the most appealing aspects, and given the game
system it was fairly obvious that its employment could not be on the complicated
and time consuming plane, any more than it could be made as a rather weak and
ineffectual adjunct to swordplay if magic-users were to become a class of player-character.
The basic assumption, then, was that D & D magic worked on a “Vancian”
system and if used correctly would be a highly powerful and effective force. There
are also four basic parts to magic: The verbal or uttered spell, the somatic or
physical movement required for the conjuration, the psychic or mental attitude
necessary to cast the spell, and the material adjuncts by which the spell, can be
completed (to cite an obvious example, water to raise a water elemental). It was
assumed that the D & D spell would be primarily verbal, although in some instances the spell would require some somatic component also (a fire ball being an
outstanding example). The psychic per se would play little part in the basic magic
system, but a corollary, mnemonics, would.
All spells were assumed to have a verbal component. Each and every
spell (not found on a scroll or otherwise contained in, or on, some magical device)
would be absolutely mnemonic, magic-users would have to memorize the spells
they wished to have available, and when a particular spell was recalled and its
other parts enacted, then the memory would be gone and the spell no longer
available until it was re-memorized (thus the magic-users’ spell books!).
Writing for the Sorcerer's Scroll magazine column in The Dragon #16 (July 1978), Gygax criticized certain houserules, like adding a “critical hit” to D&D, or entirely replacing his "Vancian" magic system with a spell points system.
Role-Playing: Realism vs. Game Logic; Spell Points, Vanity Press and Rip-offs
by E. Gary Gygax
With the popularity of DUNGEONS & DRAGONS increasing so
dramatically, I fervently desire to put the matter of variants, particularly “realistic” variants, to rest once and for all, so as to get on to
other more important things, but it keeps springing up every time a
sound stroke is dealt to it. Additions to and augmentations of certain
parts of the D&D rules are fine. Variants which change the rules so as
to imbalance the game or change it are most certainly not. These sorts
of tinkering fall into the realm of creation of a new game, not development of the existing system, and as I stated earlier, those who wish to
make those kind of changes should go and design their own game. In
order to make this clear, a few examples of destructive variants are
given below.
Spell point systems are also currently in vogue amongst the fringe
group which haunt the pages of “Amateur Press Association” publications. Now APAs are generally beneath contempt, for they typify the
lowest form of vanity press. There one finds pages and pages of banal
chatter and inept writing from persons incapable of creating anything
which is publishable elsewhere. Therefore, they pay money to tout their
sophomoric ideas, criticise those who are able to write and design, and
generally make themselves obnoxious. * While there are notable exceptions, they are far too few to give any merit to the vehicles they appear
in. From this morass rose the notion that a spell point system should be
inserted into D&D. Strangely enough, “realism” was used as one of the
principal reasons for use of spell points. These mutterings are not as
widespread as the few proponents of such a system imagine. The D&D
magic system is drawn directly from CHAINMAIL. It, in turn, was inspired by the superb writing of Jack Vance. This “Vancian” magic system works splendidly in the game. If it has any fault, it is towards
making characters who are magic-users too powerful. This sort of fault
is better corrected within the existing framework of the game — by requiring more time to cast spells, by making magic-users progress more
slowly in experience levels. Spell points add nothing to D&D except
more complication, more record keeping, more wasted time, and a
precept which is totally foreign to the rest of the game.
A letter to the editor from Marc Jacobs touched on the topic in The Dragon #28 (August 1979):
I HAVE HAD ENOUGH OF FORGETTING
SPELLS!!!!!!!
You reject spell points out of hand because one
author uses a system whereby the spell is forgotten
after casting. No other author uses such a system.
To allow a mage to repeatedly cast a spell you
make powerful implements such as wands, staves
and rings. This makes two different magic systems
(forgetting and storing) into one dubious believability and playability. I have seen dungeons where a
mage walks in with a wand of magic missiles containing 20 charges that the DM used as a treasure
because once the spell is used it is forgotten. For
want of spell points one multiplies MU implement
use incredibly.
Unfortunately for you, D&D was based, the
magic aspect of it, upon Jack Vance’s concepts of
magic in his “Dying Earth” books. That is the way it
is designed. If you don’t like it, play something else,
or design your own variant. Most people make
mages far too powerful, and badly unbalance the
game in so doing. As to the magic items extant in
any given campaign, that is solely the province of
the DM.
Gygax later elaborated on his plans to nerf spellcasters, by adding requirements for verbal, somatic, and material components to spells, in the Sorcerer's Scroll column in The Dragon #33 (January 1980), which later got reprinted under the title "Make-believe magic" in Best of Dragon Vol. II (November 1981).
AD&D’s Magic System: How and Why It Works
©Gary Gygax
Working up rules about make-believe can be difficult. Magic,
AD&D magic, is most certainly make-believe. If there are “Black Arts”
and “Occult Sciences” which deal with real, working magic spells, I
have yet to see them.
Mildly put, I do not have any faith in the powers of magic, nor have I
ever seen anyone who could perform anything approaching a mere
first-level AD&D spell without props. Yet heroic fantasy has long
been one of my favorite subjects, and while I do not believe in invincible superheroes, wicked magicians, fire-breathing dragons, and the
stuff of fairie, I love it all nonetheless! Being able to not only read about
heroic adventures of this sort, but also to play them as a game form,
increased the prospects of this enjoyment of imaginary worlds. So
magic and dragons and superheroes and all such things were added to
Chainmail.
This left me with two major areas to select from. The internal power,
or manna, system where each spell-caster uses energy from within to
effect magic, requires assigning a total point value to each such character’s manna, and a cost in points to each spell. It is tedious to keep track
of, difficult to police, and allows Magic-Users far too much freedom
where a broad range of spells are given.
Having read widely in the fantasy genre since 1950, I opted instead
for the oft-used system which assumes that magic comes from power
locked within certain words and phrases which are uttered to release
the force. This mnemonic power system was exceedingly well articulated by Jack Vance in his superb The Eyes of The Overworld and
Dying Earth novels, as well as in various short stories. In memorizing
the magical words, the brain of the would-be spell-caster is taxed by the
charged force of these syllables. To increase capacity, the spell-caster
must undergo training, study, and mental discipline.
This is not to say that he or she ever understands the words, but the
capacity to hold them in the memory and to speak them correctly
increases thus. The magic words, in turn, trigger energy which causes
the spell to work.
The AD&D magic system was therefore predicated on the concept
that there were three power-trigger keys—the cryptic utterances, hypnotic gestures, and special substances—the verbal, somatic, and
material components, possible in various combinations, which are
needed to effect magic. This aspect is less “Vancian,” if you will, but at
the same time the system overall is more so, for reasons you will see
later.
In order to expand
mnemonic capacity, spell-users must do further study and be trained.
Thus, the system is in some ways more “Vancian,” as such information
and studies are indicated, if not necessarily detailed, in the works of that
author. It might also be said that the system takes on “Lovecraftian”
overtones, harkening to tomes of arcane and dread lore.
Although the word 'Vancian' doesn't appear in the rules of D&D, the memorization-based Vancian magic system was still in use in the 2e Player's Handbook (1989). From page 81:
Ultimately, it is the memorization that is
important. To draw on magical energy, the
wizard must shape specific mental patterns
in his mind. He uses his spell books to force
his mind through mental exercises, preparing it to hold the final, twisted patterns.
These patterns are very complicated and
alien to normal thought, so they don't register in the mind as normal learning. To shape
these patterns, the wizard must spend time
memorizing the spell, twisting his thoughts
and recasting the energy patterns each time
to account for subtle changes—planetary
motions, seasons, time of day, and more.
Once a wizard memorizes a spell, it
remains in his memory (as potential energy)
until he uses the prescribed components to
trigger the release of the energy patterns.
The mental patterns apparently release the
energy while the components shape and
guide it. Upon casting, the energy of the
spell is spent, wiped clean from the wizard's
mind. The mental patterns are lost until the
wizard studies and memorizes that spell
again.
The number of spells a wizard can memorize is given by his level (see Table 21); he
can memorize the same spell more than
once, but each memorization counts as one
spell toward his daily memorization limit.
Part of a wizard's intelligence can be seen in
the careful selection of spells he has memorized.
3rd edition reflavored 'memorization' as 'preparation'
The 3e Player's Handbook (2000) still used Vancian game rules for wizards, paladins, and most other spellcasters, but distanced those rules from the traditional terms of Vancian magic. In 3e a wizard didn't 'memorize' spells, a wizard 'prepared' spells in 'spell slots,' and along with the terminology shift, an entirely different, new fluff explanation was given for the reflavored but virtually unchanged game mechanic:
WHAT IS A SPELL?
A spell is a one-time magical effect. Most spellcasting characters—wizards, clerics, druids, paladins, and rangers—prepare their spells in
advance and use them when the time is right. Preparing a spell requires
careful reading from a spellbook (for wizards) or devout prayers or
meditation (for divine spellcasters). In either case, preparing a spell
means casting the first and lengthiest part of it. Only the very end of the
spell, its trigger, remains to be activated. After preparing a spell, the
character carries it, nearly cast, in his or her mind, ready for use. To use a
spell, the character completes casting it. Spellcasting might require a few
special words, specific gestures, a specific item, or any combination of
the three. Even though most of the spell was essentially cast ahead of
time during preparation, this final action is known as “casting” the spell.
This different, non-Vancian, explanation, which doesn't involve a spellcaster instantly forgetting something upon casting, is just as limiting in practice, but probably resulted in significantly fewer frustrated reader letters like the one from Marc Jacobs!
3e not only combined that less-Vancian flavor with the still-Vancian-shaped spellcasting rules for wizards and most other spellcasters, it also added something new and truly non-Vancian: what it termed 'spontaneous casting,' done by 3e sorcerers, bards, and even sometimes (to a limited extent) clerics.
Sorcerers and bards cast arcane spells, but they do not have
spellbooks and do not prepare spells. A sorcerer’s or bard’s level
limits the number of spells he can cast (see these class descriptions
in Chapter 3: Classes).
Spontaneous Casting: Good clerics (and neutral clerics of good
deities) can channel stored spell energy into healing spells that they
haven’t prepared ahead of time. The cleric can “lose” a prepared spell
in order to cast any cure spell of the same level or lower (a cure spell is
any spell with “cure” in its name).
The divine energy of
the spell that the cure or inflict spell substitutes for is converted into
the cure or inflict spell as if that spell had been prepared all along.
4th edition mentions wizards forgetting some spells, but not all
4e classes with an Arcane power source, like the wizard and the warlock, are described as forgetting their daily powers as the flavor reason for why e.g. a warlock can't simply cast Armor of Agathys a second time in a single day. There's no mention of a warlock doing any kind of extensive memorization, or even daily preparations, in the first place, though.
An encounter power can be used once per encounter. [...] If
you’re an arcane or divine character, these are spells or
prayers of such power that they take time to re-form in
your mind after you unleash their magic energy.
A daily power can be used once per day. Daily powers
are the most powerful effects you can produce, and using
one takes a significant toll on your physical and mental
resources. [...] If you’re an arcane magic-user, you’re reciting
a spell of such complexity that your mind can only hold it
in place for so long, and once it’s recited, it’s wiped from
your memory.
The 4e wizard's Spellbook class feature says:
After an extended rest, you can prepare a number of
daily and utility spells according to what you can cast
per day for your level. You can’t prepare the same spell
twice.
A spell like Burning Hands is an encounter power, never gets written down in a wizard's spellbook, requires no daily memorization routine and inflicts only temporary memory loss upon casting.
5th edition has minimal fluff but has clearly non-Vancian rules
Colloquially speaking, in 3e terms, everyone is a sorcerer: every 5e spellcaster is a spontaneous caster. Nothing of the Vancian system remains but the continued use of the term 'spell slot,' which now, unlike 3e, is no longer a literal slot where a caster stores a prepared spell.
And 5e doesn't even bother giving long detailed in-universe explanations for the game mechanic limits on spell castings per day, there's only the game rules, with no particular why or how beyond a vague mention of it being 'physically and mentally taxing.' And at this point, 5e spellcasting rules are closer to the 'spell point system' that Gygax disliked than they are to Vancian magic - and the 5e wizard class hints at something more akin to an 'internal power, or manna, system' at work than anything mnemonic:
Arcane Recovery
You have learned to regain some of your magical energy by studying your spellbook. Once per day when you finish a short rest, you can choose expended spell slots to recover.
The 5e Dungeon Master's Guide, in the section on Creating New Character Options, even offers using a pool of points for spellcasting.
Variant: Spell Points
Instead of gaining a number of spell slots to cast your spells from the Spellcasting feature, you gain a pool of spell points instead. You expend a number of spell points to create a spell slot of a given level, and then use that slot to cast a spell.