The term 'chaotic' is part of the alignment system in D&D. Within the alignment system, your personality and decision making is rated on two scales. One from good to evil, and the other from lawful to chaotic.From the D&D Player's Handbook (5e):
Lawful good (LG) creatures can be counted on to do
the right thing as expected by society. Gold dragons,
paladins, and most dwarves are lawful good.
Neutral good (NG) folk do the best they can to help
others according to their needs. Many celestials, some
cloud giants, and most gnomes are neutral good.
Chaotic good (CG) creatures act as their conscience
directs, with little regard for what others expect. Copper
dragons, many elves, and unicorns are chaotic good.
Lawful neutral (LN) individuals act in accordance
with law, tradition, or personal codes. Many monks and
some wizards are lawful neutral.
Neutral (N) is the alignment of those who prefer to
steer clear of moral questions and don’t take sides,
doing what seems best at the time. Lizardfolk, most
druids, and many humans are neutral.
Chaotic neutral (CN) creatures follow their whims,
holding their personal freedom above all else.
Many barbarians and rogues, and some bards, are
Lawful evil (LE) creatures methodically take what they
want, within the limits of a code of tradition, loyalty, or
order. Devils, blue dragons, and hobgoblins are lawful evil.
Neutral evil (NE) is the alignment of those who do
whatever they can get away with, without compassion or
qualms. Many drow, some cloud giants, and yugoloths
are neutral evil.
Chaotic evil (CE) creatures act with arbitrary
violence, spurred by their greed, hatred, or bloodlust.
Demons, red dragons, and orcs are chaotic evil.
Good and evil are fairly self-explanatory. Good characters are generally willing to make personal sacrifices for the benefit of others. Evil characters are usually willing to harm others for personal benefit. That is of course a simplification, but it gets across the main ideas.
Lawful means that the person is willing to follow laws and give up personal freedoms for the good of their society. Lawful characters tend to follow strict moral codes. Chaotic means that the person is unwilling to give up personal freedoms for the good of society. They do not want to follow laws or be restricted by social codes. Chaotic characters tend to act on a whim, with little regard for laws or codes of practice.
In reality, the system is seldom used by most DMs. Previous editions used it much more, for example in AD&D, there was a strict LG alignment restriction for paladins (which caused annoyed parties to call some characters 'lawful stupid'). In 5e, traps and the like can theoretically be triggered by certain alignments. I have used this, but I think I am in the minority here. For most people alignment is just used as a role-playing tool, and a way to gain inspiration.
In older editions, there used to be spells and other ways to identify alignment (such as detect alignment), but these no longer exist in 5th edition (the most recent version of the game). Alignment is still theoretically an objective quality, and something intrinsic in the universe. Most players, in my experience, tend to ignore it once they have chosen it.
A full history of the alignment system would be far beyond the scope of this format, but I shall try to provide a brief history of the term 'chaotic', by writing a summary of each edition's definition.
OD&D Basic Rules Book (red book) Chaos is the opposite of law. It is the belief that life is random, and that chance and luck rule the world. Laws are made to be broken, as long as the person can get away with it. The individual is more important than the group. Chaotic creatures often act on whims, and cannot be trusted. Chaotic behaviour is usually the same behaviour that can be called "evil" (!).
AD&D Player's Handbook No specific definition for 'chaos', but we can gleam the following from the list of alignments: Chaotic creatures view randomness and disorder as the way of the universe, and believe in personal freedom above law and order.
2e AD&D Player's Handbook The believers in chaos hold that there is no preordained order or careful balance of forces in the universe. Chaotics can be hard to govern as a group, since they place their own needs and desires above those of society.
3.5e Player's Handbook “Chaos” implies freedom, adaptability, and flexibility. On the downside, chaos can include recklessness, resentment toward legitimate authority, arbitrary actions, and irresponsibility. Those who promote chaotic behavior say that only unfettered personal freedom allows people to express themselves fully and lets society benefit from the potential that its individuals have within them.
4e Player's Handbook In 4e, the alignment system was drastically simplified to only include Lawful Good, Good, Unaligned, Evil, and Chaotic Evil. Chaotic essentially became a more random evil alignment, described as 'Entropy and destruction' rather than 'Tyranny and hatred' (for evil).
4e Essentials Retained the simplified alignment system. Chaotic evil is summarised as: "I don't care what I have to do to get what I want.' This was opposed to: 'It is my right to claim what other's possess.' (for evil).
5e Player's Handbook See descriptions above.
It is clear from this that, with the exception of 4e, the editions have pretty much agreed that chaos is the belief that life is governed by chance, and that we should disregard laws. In this sense, the meaning of this alignment has not changed much, but its usage has changed significantly.
As an addendum, I recommend these two articles by the Angry GM on alignment and its usage:
You might also want to read about the history of the alignment system.