The History of AI

First version 18 October 2026, last changed 23 October 2026 · Andreas Stockter

Artificial intelligence (AI) is the umbrella term for methods that let machines solve tasks for which humans need their minds: understanding language, recognising images, drawing conclusions, planning.

From the beginning there were two ideas for achieving this. In the first, people write rules for the machine; in the second, the machine learns from examples by itself. The second idea won out, but it needed three ingredients that did not come together until 2012: a workable learning method, a very large number of examples, and very fast computers. The history of AI is the chain of problems and solutions on the way there.

The term itself keeps moving. As soon as a method works reliably, it no longer counts as AI but as software: chess, spell-checking, navigation and search engines have all gone this way. The computer scientist Larry Tesler put it in the sentence that AI is whatever has not been done yet. This AI effect explains why the chronicle describes things as AI that nobody calls by that name any more.

Every technical term is set in bold at the point where it is explained, and is only used after that. A glossary lists all terms alphabetically.

Contents: 1 Foundations · 2 Departure and the first AI winter · 3 Expert systems and backpropagation · 4 Statistical learning · 5 Deep learning · 6 Transformers · 7 Generative AI · 8 Consequences and rules · The chain in brief · Timeline · Further reading · Sources

1. Foundations (1943–1956)

The dream is older than the technology. The word robot was coined in 1920 by the Czech writer Karel Čapek in his play “R.U.R.”, from the Czech robota for forced labour, for an artificial machine that does human work; at the end of the play the robots rise up against their creators. In a story of 1942 Isaac Asimov formulated the three laws of robotics: a robot may not injure a human, must obey orders and must protect itself, in that order of precedence. Both motifs, the rebellion and the rules against it, accompany the field up to the alignment debates of the present.¹

Three pieces of groundwork precede it. In 1854 George Boole shows that logical reasoning can be written as calculation with the values true and false, the basis of every circuit. Around 1840 Charles Babbage designs a mechanical calculating machine that is never finished, and Ada Lovelace writes the first program for it. Her objection, that a machine can only do what it is told and originate nothing of its own, is one Turing will answer explicitly in 1950. And in 1941 Konrad Zuse puts the Z3 into operation in Berlin, the first working program-controlled computer, largely unnoticed by the world and destroyed in the war.

The model was the brain. It consists of nerve cells, the neurons. The human brain has around 86 billion of them. Each receives signals from many other cells and sends on a signal of its own as soon as it is stimulated strongly enough. It is said to “fire”.

In 1943, Warren McCulloch and Walter Pitts describe an artificial neuron, a mathematical imitation.² It receives several inputs (each 0 or 1) and adds them up. If the sum reaches a fixed limit, the threshold, it outputs a 1, otherwise a 0. Inhibitory inputs can prevent firing.

The two show that the three basic building blocks of logic can be built from this:

• AND outputs 1 if both inputs are 1.
• OR outputs 1 if at least one input is 1.
• NOT inverts the input.

In numbers: an AND neuron with two inputs is given the threshold 2. With the inputs 1 and 1 the sum is 2 and it fires; with 1 and 0 it is 1 and it stays silent. An OR neuron needs only the threshold 1, and NOT arises from an inhibitory input.

Every computation a computer performs consists of these three building blocks. If many artificial neurons are connected so that the output of one is the input of the next, the result is a neural network that can reproduce any logic circuit.

How much that means becomes clear from a comparison with an idea of 1936. The mathematician Alan Turing had then described the Turing machine: an imaginary device that reads a tape of symbols one cell at a time, overwrites them according to a fixed table of rules, and moves forwards or backwards as it does so. Simple as it is, it can compute anything that can be computed at all. McCulloch and Pitts proved that their network is equivalent to a Turing machine. A network of artificial neurons can therefore, in principle, do anything a computer can do. On one point, however, the two were wrong: real neurons know more than “on” and “off”; their behaviour is far more intricate. Nor could their network learn: which neurons were connected and how had to be laid down by a human.

Turing’s idea also shaped the construction of the first computers. In machines such as the British Colossus or the American ENIAC, the program still resided in the wiring. Anyone who wanted to compute a different task had to replug cables. From 1945, Turing and the mathematician John von Neumann, who knew Turing’s work, designed computers that hold their program in memory like data. This stored program has remained the design principle of every computer to this day. It turns a machine for one task into a machine for any task, and only this made AI research possible at all.

The idea of learning comes in 1949 from the psychologist Donald Hebb. He suspects that the connection between two nerve cells grows stronger when both are frequently active at the same time. Applied to artificial neurons, every connection is given a strength, a number by which the incoming signal is multiplied. This number is called the weight. A high weight means that this input counts for a lot. Since then, learning has meant: changing weights.³ In numbers: if two connected neurons fire at the same time, the weight of their connection grows by a small amount, say from 0.5 to 0.6.

In 1950 Turing asks how one could even tell whether a machine is intelligent. His proposal is the Turing test, which he himself called the “imitation game”: a person chats in writing with a stranger. If the person cannot tell whether they are writing with a human or a machine, the machine counts as intelligent.⁴ Lovelace’s objection Turing meets in the same paper: machines surprised their programmers all the time, and whether humans ever produce anything genuinely new was just as unproven. The test examines only behaviour. Whether the machine understands anything or merely feigns it convincingly, the test cannot decide, and precisely this objection has accompanied AI ever since. Turing also suggests not programming a finished intelligence, but teaching a machine like a child.

In the same year Claude Shannon, the founder of information theory, examines how a computer could play chess. He calculates that there are around 10¹²⁰ possible games. Computing every move through is therefore hopeless even for the fastest conceivable machine. A program needs heuristics: rules of thumb that weed out hopeless moves early, such as “do not give up the queen”. Shannon’s paper became the template for the chess programs of the following decades.⁵

In 1956 John McCarthy, Marvin Minsky, Claude Shannon and Nathaniel Rochester invite researchers to a summer workshop at Dartmouth College. In the invitation, McCarthy gives the field its name: “artificial intelligence”.⁶ About ten researchers take part. There, Allen Newell, Cliff Shaw and Herbert Simon present the Logic Theorist, a program that proves mathematical theorems, deriving them step by step from basic assumptions. It found proofs for 38 theorems from Russell and Whitehead’s “Principia Mathematica”, a standard work of logic, among them one shorter than the proof in the book.

What followed: Two lines of research start in parallel. One relies on logic and rules, the other on learning neurons.

2. Departure and the first AI winter (1956–1974)

The rules line. It is called symbolic AI. A symbol is a sign that stands for something, such as the word “dog” or a chess piece. The assumption is that thinking is the manipulation of symbols according to if-then rules, just as one rearranges equations in algebra.

The most important tool was the search procedure. In chess, every position offers many possible moves, each has many possible replies, and so on. The result is a branching tree of possibilities, which the program searches for a path to the goal. Because the tree is enormous, it needs Shannon’s heuristics, which weed out hopeless branches early.

The most ambitious project of this line was the General Problem Solver by Newell, Shaw and Simon (1957). It was meant to solve not one problem but any, and to do so the way humans do: compare the current state with the goal, name the difference, and look for a step that reduces it. Simon’s own example: I want to take my son to nursery school. The difference is distance. What overcomes it? The car. The car will not start. What is missing? A battery. Who has one? The garage. But the garage does not know. So, telephone. Every sub-goal is broken down until a step can be carried out directly. What was new was that the method was separated from the problem: the program knew only the method, while the facts and rules of each problem had to be typed in by a human. In practice this worked only for clearly delimited tasks such as logic proofs, puzzles and chess, because even an everyday problem like the drive to nursery school dissolves into a flood of side conditions.

In 1958 McCarthy created the programming language LISP, tailored to processing lists of symbols, which remained the language of AI research for decades. In the same year he described the “Advice Taker”, a program that was to derive everyday tasks such as a trip to the airport from a few logical rules, and to which new rules could simply be added. It was never built, but the idea of storing knowledge as a collection of rules returns in Section 3.

That learning was possible in this line too was shown by Arthur Samuel at IBM. His checkers program of 1952 rated positions with a formula whose weights it adjusted through thousands of games against itself, and by the early 1960s it played better than its author. For this Samuel coined the term machine learning in 1959: programs that improve with experience instead of being programmed for every situation.⁷

In 1966 ELIZA by the Berlin-born Joseph Weizenbaum became famous, an early chatbot, a program one converses with in writing. ELIZA searched the input for keywords and replied with ready-made sentence patterns. “I am sad” was answered with something like “Why are you sad?”. The program understood nothing, yet many people found it empathetic.⁸

Weizenbaum’s own example from 1966:

User: Men are all alike.
ELIZA: In what way?
User: They’re always bugging us about something or other.
ELIZA: Can you think of a specific example?
User: Well, my boyfriend made me come here.
ELIZA: Your boyfriend made you come here?
User: He says I’m depressed much of the time.
ELIZA: I am sorry to hear you are depressed.

Weizenbaum himself was alarmed at how readily people attributed understanding to the program, and in 1976, with the book “Computer Power and Human Reason”, he became the sharpest critic of his own field. The high point of symbolic language processing came in 1970 with SHRDLU by Terry Winograd, which understood instructions such as “put the red cube on the green block” in a world of building blocks and asked questions back. Outside the block world it did not work, and that precisely was the lesson.

The rule line also acquired a body. At the Stanford Research Institute, Shakey was built between 1966 and 1972, the first mobile robot that planned its own actions. Shakey captured the room with a camera, built a description of it out of symbols and searched that description for a path to the goal with search procedures. A task such as “push the block off the ramp” took it hours. Its planning program STRIPS (1971), which breaks a goal into subgoals and finds the suitable action for each, became the model for all later planners.⁹ The project produced the search algorithm A*, which sits in navigation devices to this day, and the lesson that the real world is far messier than a chessboard.¹⁰

The learning line. In 1958 Frank Rosenblatt presents the perceptron, an artificial neuron that sets its own weights.¹¹ Understanding what it can do takes four steps.

Class. The perceptron is meant to sort. It is given an example and decides which category it belongs to, say “cat” or “dog”. Each category is called a class, and the sorting is called classification. A single perceptron knows two classes, because it can only output 1 or 0.

Training. The perceptron is shown many examples for which the correct class is known. If it gets one wrong, it adjusts its weights a small step in the right direction. This practice is called training, and the collection of examples is the training data. Because the correct answer is supplied every time, this is called supervised learning. The result, the calculation rule with its finished weights, is called a model. The point of training is not to retain the examples but to classify animals correctly that the model has never seen. This ability is called generalisation.

Examples as points. The perceptron sees only numbers. If it is given two numbers per animal, body length and shoulder height, each animal can be drawn as a point on a sheet of paper: length to the right, height upwards. House cats form a cloud of points at the bottom left, German shepherds one at the top right.

The dividing line. The perceptron multiplies each number by its weight, adds the results, and checks whether the sum reaches the threshold. On the sheet this corresponds to a straight line: points on one side get the 1, those on the other side the 0. Because it only multiplies and adds, the boundary is always straight and never curved. Training means turning and shifting this line until it fits. A learning step in numbers: the weights stand at 0.5 and 0.5, and a German shepherd with length 1 and height 1 wrongly lands on the cat side. The perceptron then raises each weight by step size times input, here 0.1 × 1, to 0.6 and 0.6. The line tilts a little towards the correct side, and after many such steps it fits.

The limits. Both lines fail, for different reasons.

Symbolic AI fails because of the combinatorial explosion: the number of possibilities multiplies with every step. With about 30 possible moves per chess position, there are 900 possibilities after two moves, 810,000 after four, and roughly 590 trillion after ten. What worked in small practice worlds could no longer be computed in the real one.

Around 1971 the theory of NP-completeness supplied the explanation: there are whole classes of tasks in which the computational effort grows so steeply with the size of the task that no computer, however fast, solves them in foreseeable time. The problem lay not with slow machines but with the tasks themselves. The same finding, that human knowledge rarely fits into sharp rules, had already been answered by Lotfi Zadeh in 1965 with fuzzy logic, which computes with gradations such as “fairly warm” instead of only true and false.¹²

From outside came criticism that the field long refused to hear. As early as 1965, in a report for the RAND Corporation, the philosopher Hubert Dreyfus denied that thinking consists in manipulating symbols, and in 1972 he set this out in his book “What Computers Can’t Do”: people act from a background of body, experience and common-sense knowledge that cannot be fully captured in rules, and precisely this background is what a program lacks. The researchers dismissed him as an outsider. The expert systems of the next section then failed at exactly the point he had named.¹³

The perceptron fails because of its straight line. A task is called linearly separable if a single straight line separates all examples of one class from all of the other. Cats and German shepherds are. In 1969 Marvin Minsky and Seymour Papert show that even a very simple task is not.¹⁴

Take a lamp with two switches A and B, each set to 0 or 1. That gives four cases, four points at the corners of a square. The classes are “on” and “off”. With the rule AND, the lamp is lit only when both switches are at 1:

The only “on” point sits in one corner, and a diagonal line cuts it off. A perceptron can learn that.

It is different with XOR (“exclusive or”), like a two-way switch in a hallway. The lamp is lit when exactly one switch is at 1:

The “on” points lie diagonally opposite each other. However one places a single straight line, at least one point ends up on the wrong side. Two lines are needed, that is, two neurons, and a third that combines their results. The neurons are then arranged one behind the other in layers: the outputs of the first layer are the inputs of the next.

This was the dilemma. Rosenblatt’s learning rule works only if the correct output of a neuron is known. For the last layer it is known; for the neurons before it, it is not. Nobody knew how to train those.

The grand promises of the early years went unfulfilled, and reports in the United States (the ALPAC report of 1966, which declared machine translation a failure) and Britain (the Lighthill report of 1973) were highly critical. Funding was cut. Such a phase of disappointment and lack of money is called an AI winter.

What followed: Programs that were meant to solve any problem whatsoever had failed. General methods without knowledge of the field in question, later mockingly called “weak methods”, delivered weak results. So the ambition was narrowed to individual specialist fields, and as much knowledge as possible was built in there.

3. Expert systems and backpropagation (1980s)

Expert systems. An expert system is meant to make decisions in a narrow field the way a human expert would. It has two parts:

• The knowledge base contains hundreds to thousands of if-then rules elicited from specialists in interviews, such as “If the customer orders computer X, then they need power supply Y”.
• The inference engine is the program that chains these rules together and so arrives at a conclusion from the known facts.

The first was DENDRAL, developed at Stanford from 1965 by Edward Feigenbaum, Bruce Buchanan and the Nobel laureate Joshua Lederberg. It derived the structure of a molecule from the readings of a mass spectrometer, a task for which chemists need years of experience. The impetus came from NASA, which wanted to analyse the chemical composition of the Martian surface. The trick was to translate textbook knowledge into concrete recipes: “If the spectrum shows these peaks, then the molecule contains this group.” With DENDRAL, the direction of thinking in AI reversed. The key was not a general method but as much knowledge as possible about a narrow field.

In tests, MYCIN proposed diagnoses for bacterial infections using around 600 rules. XCON checked and completed orders at the computer manufacturer DEC and saved millions. PROSPECTOR helped geologists search for ore deposits. For the first time, AI became a business.

The boom was fuelled in 1982 by Japan’s Fifth Generation project, a state programme that set out to build computers within ten years that reason in logic and speak, built on the logic language Prolog. The United States and Europe answered with programmes of their own, Germany in 1988 with the founding of the DFKI in Kaiserslautern and Saarbrücken, to this day one of the largest AI research institutes. The language of expert systems, besides LISP, was Prolog, developed in Marseille in 1972, in which one writes down facts and rules and gets the inference engine thrown in.

The inference engine can work in two directions. In forward chaining it starts from the known facts, applies every matching rule and collects what follows. In backward chaining it starts from the suspected result and checks which rules support it and whether their conditions are met. If a fact is missing, it asks the user. MYCIN worked backwards: for each of around 100 possible diagnoses it looked for laboratory values and observations that support it.

The knowledge base contains two kinds of knowledge: factual knowledge, as found in textbooks, and heuristic knowledge, the rules of experience an expert acquires only through years of practice. For larger contexts, Minsky proposed frames in 1974, data structures that bundle all the properties of a concept and inherit from broader concepts: a horse “is a” mammal, “can” run, “has” four legs.

Because medical conclusions are rarely certain, every rule in MYCIN was given a certainty factor between 0 and 1, and the program combined the factors along the chain of evidence. A real rule read: if the organism is gram-negative, rod-shaped and anaerobic, then with certainty 0.6 it is a Bacteroides. Values below 0.2 counted as zero. The cleaner alternative, Bayes’ theorem from probability theory, failed in practice because it requires knowing many individual probabilities that nobody had. One virtue remained common to all these systems: every decision could be traced step by step through the rules applied.

Four problems ended the boom:

• The knowledge acquisition bottleneck: specialists can put only part of their knowledge into rules. Much of it they know from experience without being able to explain it.
• With every new rule, the risk of contradictions grew, and maintenance became expensive.
• The systems were brittle. Faced with a case no rule anticipated, they failed completely, because they lacked the everyday knowledge humans use to bridge gaps.
• The systems could not learn. Every new insight had to be entered by hand as a rule, and the rule set then checked for contradictions.

Around 1987 the market collapsed. The expensive Lisp machines, special-purpose computers for AI programs, were overtaken by ordinary workstations, the expert-system companies vanished, and the second AI winter began. Japan’s project ran out in 1992 without tangible results.

The critics. Two objections date from this period that accompany the field to this day. In 1980 the philosopher John Searle described the Chinese room: a person who knows no Chinese sits in a room and answers questions in Chinese by looking the characters up in a rulebook and assembling them according to its instructions. From outside, the answers appear intelligent, yet the person in the room understands not a word. A program, Searle argued, does nothing else: it manipulates symbols without knowing their meaning. For the claim that a suitably programmed computer genuinely has a mind he coined the term strong AI, as opposed to weak AI, which merely reproduces intelligent behaviour for particular tasks.¹⁵ In 1988 the roboticist Hans Moravec recorded what has since been called Moravec’s paradox: what people find hard, such as chess or algebra, is easy for computers, and what every child masters, such as seeing, grasping and walking, is extremely hard for computers. His explanation: evolution has honed the senses and movement over hundreds of millions of years, while abstract thought is only a few tens of thousands of years old.¹⁶ In hindsight, both objections explain why expert systems failed at common-sense knowledge and why robots advanced so much more slowly than chess programs.

Neural networks, meanwhile, were not dead. In 1982 the physicist John Hopfield presented a network that stores patterns as wholes and completes them again from a damaged fragment, an associative memory.¹⁷ It drew physicists into the field and restored its reputation. Minsky, who had put the brakes on perceptron research in 1969, put forward a theory in 1986 with “The Society of Mind” according to which intelligence arises from the interplay of many simple units that are, taken individually, stupid.

Backpropagation. At the same time, the riddle of 1969 was solved: how does one train a network with several layers? The layers between input and output are called hidden layers, because nobody can specify what their neurons should output.

Two things were needed. First, a measure of how wrong the network is. That is the error, a number that grows the further the output deviates from the correct answer. For example: if the network outputs 0.73 and the correct answer is 1, the error is (1 − 0.73)² = 0.07; for an output of 0.2 it would be 0.64.

Second, the neuron was slightly altered. It no longer jumps abruptly between 0 and 1 but outputs a smooth value, for example 0.73. The rule that turns the weighted sum into this output value is called the activation function. This way, a small change to a weight causes a small change in the error rather than a sudden jump.

Backpropagation (“backward propagation”) combines the two. David Rumelhart, Geoffrey Hinton and Ronald Williams make the method widely known in 1986, after Paul Werbos had already described it in his doctoral thesis in 1974 without attracting attention. Earlier still, in 1965, Alexey Ivakhnenko in Kyiv had trained multilayer networks layer by layer, a method appreciated only decades later.^{18, 19} After each example, one calculates backwards from the output, layer by layer, what share each individual weight had in the error. It is like tracing a defect in a finished product back through an assembly line, station by station.

For each weight this yields an indication of the direction and strength in which the error changes if the weight is increased slightly. All these indications together are called the gradient. Then each weight is moved a small step in the direction that reduces the error. This procedure is called gradient descent and is repeated millions of times. One can picture it as a descent in fog: you cannot see the valley, but at every step you walk in the direction where the slope is steepest downwards. The step size is called the learning rate. With a learning rate of 0.1, every weight is changed by a tenth of its gradient. Too small, and training takes forever; too large, and it leaps over the valley.

During training, the hidden layers develop detectors for useful properties of the data by themselves, in images for example for edges or curves. Such properties are called features.

Recognising images. In 1989 Yann LeCun applies backpropagation to images. A digital image consists of picture elements, the pixels, each with a brightness value. His convolutional neural network (CNN) works with filters. These are small pattern templates a few pixels in size that travel across the entire image and report where their pattern occurs, for example a vertical edge.

The next layer searches these reports for combinations, since edges make up corners and arcs. The layer after that looks for whole shapes. Between the filter layers lies a pooling layer that shrinks the image by keeping only the strongest value from a small neighbourhood of pixels. In this way the image becomes coarser and more abstract from layer to layer, until the last layer summarises the whole image. Which patterns the filters look for is learned by the network itself. The system read handwritten postcodes and later cheques.²⁰

Try it yourself. The TensorFlow Playground shows a small network training in the browser: choose the XOR data from Figure 3, add a hidden layer and watch the dividing line curve and the error fall. ELIZA still runs as a web page, and a perceptron with its learning rule fits into ten lines of Python. Anyone who wants to rebuild an expert system will find in CLIPS, created at NASA in 1985, a freely available rule engine.²¹

What followed: The method was there, but the breakthrough did not come. Computers were too slow and the example collections too small. On top of that came an obstacle Sepp Hochreiter analysed in 1991, the vanishing gradient problem.²² The error information that backpropagation passes from back to front is multiplied by small numbers in every layer and shrinks. With many layers, almost nothing arrives at the front, and the front layers do not learn. Networks with many layers, so-called deep networks, remained untrainable for the time being.

4. Statistical learning (1990s–2000s)

Research turns to methods that get by with fewer examples and less computation. The umbrella term for everything that learns from examples instead of being programmed is Samuel’s machine learning. Neural networks are one family within it, but at the time not the most successful. Widespread were:

• Bayesian networks map how strongly things depend on one another and compute with probabilities instead of certain rules. A wet road suggests rain, but could also come from a sprinkler. Judea Pearl, who developed them in the 1980s, received the Turing Award for this in 2011.
• Hidden Markov models describe a sequence of states that cannot be observed directly but only through their effects, such as the speech sounds behind an acoustic signal. From the 1980s until 2012 they were the backbone of every speech recogniser, before deep networks replaced them.²³
• Support vector machines, like the perceptron, look for a dividing line between two classes, but specifically the one with the largest distance to the nearest points on both sides. A mathematical trick also allows them curved boundaries.
• A decision tree is a sequence of yes/no questions leading to a decision. A random forest lets many different trees vote.
• The nearest-neighbour algorithm simply compares a new example with the most similar known examples and adopts their class.

There are not always correct answers. In unsupervised learning the method receives only the data and searches for structure in it by itself, for example groups of similar customers or recurring patterns in astronomical measurements.

The price of these methods was feature engineering. People had to devise and program which features would be computed from the raw data, for emails for example the number of exclamation marks. Quality depended on the ingenuity of the developers. In image recognition such hand-made features were called descriptors, for example SIFT and SURF: methods that locate edges, corners and conspicuous blobs in an image and describe them as a list of numbers, which a support vector machine then classifies. Whether the chosen features were the right ones remained guesswork.

The first mass-market product of these methods was unassuming: from 1998, statistical spam filters sorted out unwanted e-mails by learning from millions of flagged messages which words are suspicious. The arithmetic behind it is Bayes’ theorem from the MYCIN paragraph. If the word “free” appears in 40 per cent of spam mails but in only 1 per cent of normal ones, and both kinds are equally common, then a mail containing “free” is spam with probability 40 / (40 + 1) = 97.6 per cent. Every further word sharpens the estimate.

Almost all the methods of this section live on in scikit-learn, a free Python library begun in 2007 that became the standard tool for machine learning without deep networks.²⁴

Deep Blue. In 1997 the IBM computer Deep Blue beats the world chess champion Garry Kasparov. It had hardly learned anything. It searched 200 million positions per second and assessed each one with a formula written by humans. It was the late triumph of search procedures and a hint of how much raw computing speed can achieve.

Learning to drive. In 2004 the American research agency DARPA announces a race for driverless cars through the desert. No vehicle gets further than twelve kilometres. In 2005 five reach the finish, first among them Stanley from the team around Sebastian Thrun at Stanford, which recognised the road from laser data and camera images with statistical learning methods. From that team emerged Google’s self-driving car project in 2009, today’s Waymo. In everyday life, robots remained simple by contrast: the Roomba vacuum robot (2002) drove by fixed rules, and Honda’s humanoid ASIMO (2000) climbed stairs, but only in prepared surroundings. Moravec’s paradox from Section 3 still held.

Memory for language. Text and spoken language are sequences, data in which the order matters. A recurrent neural network (RNN) reads a sequence element by element while carrying along a memory: a list of numbers summarising what has come so far, updated after every element. During training, the vanishing gradient appears here again, this time across the time steps. The network forgets what lies more than a few words back.

Hochreiter and Jürgen Schmidhuber solve this in 1997 with the LSTM (long short-term memory). It has a memory cell and gates, small learning switches that decide at every step what is newly stored, what is kept and what is output. In this way, information survives across hundreds of steps. Speech recognition and machine translation were later built on this.²⁵

At the same time a small group kept working on neural networks against the trend, funded by a Canadian institute that brought Hinton, Bengio and LeCun together. In 2006 Hinton showed how a deep network can be pre-trained layer by layer, so that the vanishing gradient problem is sidestepped.²⁶ He gave the approach a new name: deep learning, learning in deep networks. The term was a fresh start for a method many considered finished.

The missing ingredients emerge.

• Data. The internet supplies texts and images in unprecedented quantities. But supervised learning needs examples with the correct answer. The computer scientist Fei-Fei Li therefore has millions of photos labelled by hand with the correct class and publishes them in 2009 as ImageNet.²⁷ From 2010 there is an annual competition in which research groups measure their image recognition against the same photos.
• Computing power. This means how many computational steps a computer manages per second. A normal processor handles complicated tasks one after another. A graphics processor, the GPU, was built for video games and has thousands of simple arithmetic units working simultaneously. Training neural networks consists of countless similar multiplications and additions that can be distributed this way. Around 2009 it becomes clear that GPUs speed up training many times over.

Watson. What was already possible with large amounts of data and fast hardware before deep learning is shown by IBM in 2011: its Watson system beats the two best human players on the quiz show “Jeopardy!”. It understood the questions in natural language, searched millions of documents, rated hundreds of candidate answers with statistical learning methods and answered within seconds. Around this time AI leaves the laboratories: speech recognition, translation and recommendation systems become everyday things.

What followed: Learning methods (backpropagation, CNNs), data (ImageNet) and computing power (GPUs) were now on the table at the same time.

5. Deep learning (2012–2016)

Deep learning is the training of deep networks, those with many layers. Their advantage: they learn their features in stages by themselves, edges at the front, patterns behind them, then parts such as eyes or wheels, and whole objects at the end. Feature engineering becomes unnecessary. The price: after training, nobody can say precisely any more which features the network uses for its decision. What an expert system laid open rule by rule is here buried in millions of weights. This is called a black box, and the charge has accompanied deep learning ever since.

Cats. In June 2012 a team around Andrew Ng and Jeff Dean at Google lets a network run for three days on 10 million still images from YouTube videos, spread across 16,000 processor cores and without any labels. The network developed neurons of its own accord that responded to human faces, bodies and cats: unsupervised learning on a grand scale. The experiment made headlines, not least because it confirmed what the internet likes to post most.²⁸

The breakthrough. In 2012 AlexNet enters the ImageNet competition, a CNN by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton with eight layers, trained on two GPUs. It is wrong on 15 % of the images; the runner-up on 26 %.²⁹ What counted was whether the correct answer was among the five best guesses. Two tricks helped:

• ReLU is a new activation function. It outputs negative sums as 0 and positive ones unchanged, as a formula: output = max(0, sum). As a result, the error information hardly shrinks on its way back, and the vanishing gradient becomes manageable.
• Dropout helps against overfitting. Overfitting means that a network memorises its training examples instead of grasping the general pattern, like a pupil who knows old exams by heart and fails on new questions. Dropout randomly switches off a share of the neurons in every training step, so that the network cannot rely on individual ones.

To notice overfitting at all, the examples are divided up: the training data is used for learning, and held-back test data is used to check whether the model also classifies unfamiliar examples correctly. One pass through all training examples is called an epoch. Together with the learning rate it is among the dials that get turned most during training.

Within two years, the whole of image recognition switched to deep networks. In 2015 residual connections are added, shortcuts through which the signal can skip layers. This makes networks with more than 150 layers trainable, which make fewer errors on ImageNet than human test subjects.³⁰

Speech recognition had in fact gone first: as early as 2012, deep networks cut the error rate for spoken language by around a third, and Siri and Google Voice switched over. Then came transfer learning: a network trained on ImageNet has already learned edges, shapes and object parts and can be adapted to a new task with a few hundred examples, such as recognising skin tumours. The pre-trained network becomes the starting point, no longer the empty one. In Section 6 this principle carries the language models.

All of this became accessible through freely available program libraries, first Theano and Caffe, then TensorFlow from Google (2015) and PyTorch from Facebook (2016). What had been the specialist knowledge of a few laboratories could now be set up by any programmer in a few lines, just as LISP had once spread symbolic AI.

Words as numbers. A network can only compute with numbers. Until then, words were mere numbers without context for computers, and nothing revealed that “dog” and “puppy” are related. In 2013 word embeddings change that. Every word is given a vector, a list of a few hundred numbers. As in the cat-and-dog example, it can be pictured as a point, only in a space with a great many directions. The numbers are learned from huge amounts of text according to the principle: words that occur in similar surroundings get similar vectors.

Related words therefore lie close together, and relationships become directions. Compute king − man + woman and you land near queen. Meaning has become something a network can calculate with.³¹

Translating. In 2014 sequence-to-sequence emerges. One LSTM, the encoder, reads the source sentence and condenses it into a single vector. A second LSTM, the decoder, generates the translation from it word by word. The bottleneck is this one vector: a long sentence does not fit into it, and details are lost.

The solution, in the same year, is called attention. For every word it writes, the decoder may look back at all words of the source sentence and learns to weigh which of them matter at that moment. When translating the verb, it looks above all at the verb in the original.³²

Generating images. Also in 2014 come the GANs (generative adversarial networks). Two networks compete: a forger produces images, and an inspector tries to tell them apart from real photos. Both improve in the contest until the forgeries look realistic.³³

Learning to play. In reinforcement learning there is no correct answer for every example. The system tries out actions, receives a reward or penalty at the end, such as a win or a loss, and reinforces the behaviour that led to the reward.

The first success to attract attention with this comes in 2013 from the London company DeepMind, founded in 2010 by Demis Hassabis and bought by Google in 2014: its DQN learns Atari games solely from the screen pixels and the score, without anyone explaining the rules to it, and reaches human level in many games.³⁴ In Breakout, where a paddle steers the ball against a wall of bricks, DQN discovered the best strategy by itself: dig a tunnel at the edge and shoot the ball behind the wall, where it clears brick after brick on its own. No programmer had specified it.

With this, AlphaGo from DeepMind beats the Go professional Lee Sedol in 2016. AlphaGo’s second building block is Monte Carlo tree search: instead of examining every move, the program plays thousands of games to the end at random and follows up the moves that pay off most often. The neural network tells it which moves are worth a try. The board game Go has more possible positions than there are atoms in the visible universe, so search alone, as with Deep Blue, is hopeless. AlphaGo therefore combines search with two deep networks:

• One proposes a few promising moves, so that only a small part of the tree has to be searched.
• The other estimates who is ahead in a position, so that the search does not have to calculate to the end of the game.

It was trained first on human games, then in millions of games against itself. A year later AlphaGo Zero dispenses with human games entirely, learns only from play against itself and beats the old AlphaGo 100 to 0. AlphaZero transfers this to chess and shogi in 2017, with the same method and no built-in expertise. Sutton’s bitter lesson at the end of this chronicle has its showcase here.³⁵

The field takes shape. The big internet companies set up research departments of their own, Google Brain (2011) and Facebook AI Research (2013) under Yann LeCun, and in 2016 Google builds its own chip just for neural networks, the TPU. In December 2015 OpenAI is founded, initially as a non-profit laboratory meant to ensure that a future AI at human level benefits everyone rather than a single corporation. Hinton, LeCun and Yoshua Bengio, who had carried the field through the winters, receive the Turing Award for 2018 in 2019, the highest distinction in computer science.

AI in everyday life. The most used AI of these years hardly stands out: recommender systems, which predict from past behaviour which video, product or post will please next. Netflix offered a prize of a million dollars for this as early as 2006; YouTube, Amazon and later TikTok build their business on it. Facial recognition reaches human accuracy in 2015 with Google’s FaceNet, and in 2017 Face ID unlocks the iPhone.³⁶

First shadows. With Siri (2011) and Alexa (2014), AI moves into households, and with it the questions that later shape the big debates. In 2016 Microsoft switches off its chatbot Tay after 16 hours because users had taught it to repeat racist slogans: the first public case of a model doing what its data shows it rather than what its builders wanted. In the same year the investigative outlet ProPublica demonstrates that a program used in US courts systematically overestimates the risk of reoffending for Black defendants. And facial recognition, suddenly reliable with deep networks, becomes a tool of surveillance, in China on a blanket scale.

Try it yourself. The classic first exercise in PyTorch is MNIST, the dataset of handwritten digits that goes back to LeCun’s postcodes of 1989: a small network reaches 98 per cent correct recognition on it after a few minutes of training. Google’s Teachable Machine trains an image classifier in the browser with the webcam, without a line of code. Gymnasium provides DQN’s Atari games and many other environments for reinforcement learning.³⁷

What followed: For language, LSTMs with attention were the state of the art. But they read text word by word in sequence, whereas GPUs show their strength only when many things happen at once. That limited how much text could be used for training. The obvious question was whether the sequential part could be dropped and only the attention kept.

6. Transformers (2017–2021)

The new design. The blueprint of a network, which layers it has and how they are connected, is called its architecture. In 2017 researchers at Google present a new one, the transformer. Their paper carries the answer in its title: “Attention Is All You Need”.³⁸

The core is self-attention. Previously the decoder looked at the source sentence. Now every word of a text looks at all the other words of the same text. For each word, the network computes how important every other word is for understanding it, and takes in that word’s information accordingly. In “The cat chased the mouse because it was hungry”, it learns that “it” refers to “cat”. In numbers: for “it” the weights might be cat 0.6, mouse 0.2, hungry 0.1 and all remaining words together 0.1; they always add up to 1. The new representation of “it” is the vector of the other words averaged with these weights, that is, 60 per cent the vector of “cat”. The weights themselves come from comparing the word vectors, just as closeness does with word embeddings.

This happens for all words at the same time. So that the order is not lost, every word is given its position as additional information. Strictly speaking, the network works with tokens, which are words or parts of words. “Computing power” might be split into “Comput”, “ing” and “power”. The splitting is done by the tokenizer, which learns the most frequent character sequences from the text as building blocks before training. That models are bad at counting letters is due to this: they see no letters, only tokens. Asked how many r’s there are in “strawberry”, models long answered “two”, because they see “str”, “aw” and “berry” rather than the letters. Because nothing has to happen sequentially any more, the transformer makes full use of GPUs and can be trained on far more text.

How far neural translation had already come was shown in the same year by DeepL from Cologne: a small company that grew out of the dictionary Linguee beat the translators of Google and Microsoft in blind tests, especially on German grammar, and became proof that quality in this field is not solely a matter of corporate size.

Learning without labels. The remaining question was where the training data would come from, since ImageNet had needed millions of hand-labelled images. In self-supervised learning, the text supplies the correct answer itself: a word is masked and the network is made to predict it. Every sentence on the internet thus becomes a practice example, with no human preparation.

A network that predicts the next word in this way is called a language model. For a given beginning of a text, it computes how probable each possible token is as the next one. It generates text by choosing a probable token, appending it and repeating the whole process, token by token. The task sounds simple but demands a great deal. To continue “The capital of France is …” correctly requires factual knowledge; other sentences require grammar or logical reasoning.

From this emerges a two-stage procedure. In pre-training the model learns general language understanding from vast amounts of text. In fine-tuning it is adapted to a specific task with a few examples. In 2018 the first such models appear: BERT from Google, which guesses masked words in the middle of a sentence (“The [MASK] barks” → “dog”) and soon improves the search engine, and GPT from the company OpenAI, which predicts the next word. GPT stands for “Generative Pre-trained Transformer”.³⁹

The two models use different halves of the transformer. BERT consists of the encoder, which reads a text as a whole and classifies it, and suits search and classification. GPT consists of the decoder, which continues writing word by word. Because only the decoder can generate text, this line prevailed once generating became more important than classifying. Pre-training is the transfer learning of Section 5 in a new guise. The text for it came from the web: Common Crawl, a freely available snapshot of billions of web pages, plus Wikipedia and digitised books. That the entire web became training material without anyone being asked is the root of the legal disputes in Section 7.

Size creates capabilities. The adjustable numbers of a model, above all its weights, are called parameters. A large language model (LLM) has billions of them. OpenAI’s series shows the leap: GPT-1 had around 117 million parameters in 2018, GPT-2 the following year 1.5 billion, GPT-3 in 2020 already 175 billion. OpenAI initially held GPT-2 back because the generated texts were so credible that misuse for propaganda and spam was feared. It was the first public debate about the dangers of language models. GPT-3 then showed something unexpected.⁴⁰

The input text given to a language model is called the prompt. If one writes a few examples of a task into the prompt, say three English words with their French translations and then a fourth English word, GPT-3 continues the pattern correctly. Nobody had trained it to do so, and its weights do not change in the process. This ability is called in-context learning. It is the best-known example of emergent abilities: abilities that small models lack and that appear as if by themselves beyond a certain size, without anyone having trained them.

This is what a prompt from the GPT-3 paper looked like, three examples and an open line:

sea otter → loutre de mer
peppermint → menthe poivrée
plush giraffe → girafe peluche
cheese →

The model answers “fromage”. The word “translate” does not occur in the prompt.

Measurements also revealed scaling laws: performance rises predictably when model, data volume and computing power are increased. That made investments of billions calculable.⁴¹ In 2022 DeepMind corrected the arithmetic: most models were too large for their amount of data, and a smaller model with more text beats a larger one with less.⁴² Since then the amount of text counts as much as the number of parameters. The rule of thumb since then is around 20 tokens of training text per parameter. GPT-3 had 175 billion parameters and had seen 300 billion tokens; Chinchilla, with 70 billion parameters and 1.4 trillion tokens, was better and cheaper to run.

Beyond language. Proteins are the molecules that do almost all the work in cells. Each consists of a chain of building blocks (amino acids) that folds into a three-dimensional shape, and this shape determines its function. Determining it in the laboratory often took years. In 2020 AlphaFold 2 from DeepMind predicts the shape directly from the chain of building blocks using an attention architecture, for many proteins almost as accurately as the experiment. The problem had been open for 50 years, and in 2024 it earned the Nobel Prize in Chemistry for Demis Hassabis and John Jumper.⁴³

In 2021 a Stanford group coins the term foundation model for models of this kind: a model pre-trained once, on which many applications are built, as a foundation carries many houses.⁴⁴ In the same year the architecture grows in two further directions. CLIP from OpenAI learns from 400 million image-text pairs from the web to place images and their descriptions in the same vector space. With this, a model understands what “a dog on a skateboard” shows, and DALL·E, presented the same year, generates such images from text. Codex, a GPT-3 fine-tuned on program code, writes programs from a description in everyday language and becomes, as GitHub Copilot, the first mass-market product of the language models. At the same time an open counterpart emerges: in 2021 the volunteer group EleutherAI releases GPT-Neo and GPT-J, the first freely available models modelled on GPT-3, and with “The Pile” an open training dataset of 800 gigabytes of text. Here begins the line of open models that Section 7 continues with Llama.⁴⁵ In the same year a group around Dario Amodei leaves OpenAI and founds Anthropic, with the declared aim of researching the safety of large models.

With size came criticism. As early as 2016 it had emerged that word embeddings absorb prejudices from their texts: to “man is to programmer” they returned “woman is to homemaker”. In 2021 Emily Bender, Timnit Gebru and colleagues called large language models “stochastic parrots” that parrot language without understanding it, multiplying the biases of their training data and the energy hunger of training. Gebru lost her job at Google over the paper.⁴⁶

Try it yourself. The Transformers library from Hugging Face loads BERT or GPT-2, released in full in 2019, in a few lines of Python and lets them complete text. OpenAI’s tokenizer playground shows how a sentence of your own falls into tokens. Jay Alammar’s “The Illustrated Transformer” explains the architecture in pictures, the interactive “Transformer Explainer” from Georgia Tech lets you watch a GPT-2 at work in the browser, and bertviz makes the attention weights of a sentence visible.⁴⁷

What followed: The capabilities were there, but accessible only to programmers. Moreover, a pure language model continues text rather than fulfilling a request. What was missing was simple operation and the reliable following of instructions. This second problem has a name: alignment, aligning a model with what people want from it, and not merely with what is most probable in the training material.

7. Generative AI (since 2022)

Generative AI creates new content such as text, images, program code, sound and video, instead of merely classifying what exists.

Images. Here diffusion models replace GANs. In training, one takes photos and adds noise step by step: every pixel is made lighter or darker by a random amount. After a few steps the image merely looks grainy; after many, what remains is an even scatter of light and dark dots in which no shape can be recognised any more. The network learns to reverse each of these steps. To generate, one starts with pure noise and lets the network denoise step by step, while a text description steers what should emerge. In 2022 DALL·E 2, Midjourney and Stable Diffusion appear. The last is freely available, and its foundation comes from a research group at LMU Munich.

Text. In November 2022 OpenAI releases ChatGPT. Technically it was a small step: a GPT model with a simple chat interface. The groundwork had been laid in January of the same year with InstructGPT. In instruction tuning, a pre-trained model is further trained on thousands of hand-written examples in which an instruction is followed by the desired answer. On top of this came a post-training stage called RLHF (reinforcement learning from human feedback). Humans rate the model’s answers, and it is trained to prefer well-rated ones. In this way it learns to answer questions and follow instructions instead of merely continuing text. It was the first practical answer to the alignment problem.⁴⁸ After two months, ChatGPT had around 100 million users.

Also in 2022, researchers at Google noticed that models do markedly better on arithmetic and logic tasks when asked to think “step by step” instead of answering at once. This thinking aloud is called chain of thought. It showed that part of the ability is already in the model and is only called up by the right form of prompt.⁴⁹

An example from the paper: “Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?” Asked directly, models frequently answered wrongly. Asked to think step by step, the model wrote: “Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11.” The intermediate step makes the difference, because every generated token serves as the basis for the next.

2023 becomes the year of the race. In March, GPT-4 from OpenAI appears, the first to understand images as well and to pass the American bar exam, together with Claude from Anthropic; in December, Gemini from Google. At the same time Meta’s model Llama leaks onto the web, and Meta soon releases the weights officially. Such open-weights models can be downloaded by anyone, run on their own hardware and modified. In Europe, Mistral from Paris and Aleph Alpha from Heidelberg try to establish models of their own alongside the American ones; later DeepSeek from Hangzhou follows. Since then two camps have faced each other: closed models, reachable only through their providers, and open ones, spreading as Linux once did.

The warners. Fear of one’s own creation has a long prehistory. The mathematician I. J. Good, once Turing’s colleague at Bletchley Park, described the intelligence explosion in 1965: a machine cleverer than its builders could build still cleverer machines, and humans would then be left behind. The computer scientist and author Vernor Vinge called this point the singularity in 1993, and Ray Kurzweil predicted it in 2005 for the year 2045. In 2014 the philosopher Nick Bostrom, with “Superintelligence”, turned the question of how such a machine could be bound to human goals into a research topic, and in 2017 thousands of researchers signed 23 principles for beneficial AI at Asilomar. Until ChatGPT this counted as a specialist debate.⁵⁰

In 2023 concern grows. In March an open letter with thousands of signatures calls for a pause in training the largest models. In May Geoffrey Hinton leaves Google so that he can warn freely of the risks of his own invention. In November 28 states meet at Bletchley Park, where Turing had deciphered German radio messages in the Second World War, and agree to monitor the risks jointly. Britain and the United States set up state institutes for AI safety. In the same month the board of OpenAI dismisses Sam Altman, the company’s chief, and brings him back after five days of turmoil. The dispute between caution and speed was now public.

Since then development has followed the familiar pattern: a weakness becomes visible, and an answer to it emerges.

• Hallucinations are statements that sound convincing but are false. They arise because a language model produces probable text and does not look up verified facts. RAG (retrieval-augmented generation) helps against this: the system first retrieves suitable documents or web pages, places them in the model’s prompt, and has it answer on that basis. For this the documents are stored as vectors in a vector database, so that the passages closest in content to a question can be found, just as word embeddings place related words side by side (Figure 16). How costly a hallucination can become was shown in 2023 by the case Mata v. Avianca in New York: a lawyer filed a brief with six precedents invented by ChatGPT, complete with docket numbers and quotations, and was fined by the court.⁵¹
• Multimodal models process images, sound and video alongside text from 2023. In 2024 GPT-4o and Gemini hold conversations in real time with voice and camera, and Sora and Veo generate video sequences from a description that are hard to distinguish from film footage. The flip side is deepfakes, faked images, voices and videos of real people, against which invisible watermarks and provenance records are being developed.
• The context window is the number of tokens a model can take into account at once. It grew from a few pages of text to whole books. A token corresponds to about four characters of English, a book page to around 400 tokens. A context window of 200,000 tokens thus holds a book of 500 pages.
• Steering. A system prompt is an invisible instruction from the provider, placed before every conversation, that sets tone and limits. The temperature controls whether the model always picks the most probable token or also less probable ones, that is, whether it writes soberly or inventively. An example: for the sentence opening “The cat sits on the …”, the model rates “mat” at 60 per cent, “couch” at 25, “windowsill” at 10 and “street” at 5. At temperature 0 it always picks “mat”. At temperature 1 the choice follows exactly these shares. At temperature 2 the shares move closer together, to 43, 28, 17 and 12 per cent, and the street turns up too. A jailbreak is a prompt that circumvents the trained-in limits, for instance through role play.
• Reasoning models work through intermediate steps before answering, from 2024, instead of answering immediately. That costs computing time but markedly improves mathematics, logic and programming. In September 2024 OpenAI presents o1, the first such model, trained to generate and check chains of thought itself. This shifts the arithmetic: until then almost all computing power went into training, now it also pays off at inference, the use of the finished model, where longer thinking buys better answers.
• Agents act instead of merely answering. They are given tools, such as a web search, the execution of program code or the operation of software. With these they complete multi-step tasks independently in a loop: plan, act, check the result, correct (Figure 17). With the ability to act comes a new kind of attack: in a prompt injection, someone hides instructions in a web page or e-mail that the agent reads, and the model carries them out as if they came from the user. In 2025 agents in tools such as Claude Code from Anthropic or Codex from OpenAI write whole programs on their own, and programming becomes the first occupation in which AI visibly reshapes everyday work.
• Efficiency. Large models now run on laptops too. Quantisation stores weights with fewer decimal places, distillation trains a small model to imitate the answers of a large one, and in a mixture of experts a model consists of many sub-networks, of which only a few compute per token. For scale: a model with 70 billion parameters occupies 140 gigabytes of memory at 16 bits per weight, quantised to 4 bits only 35, and thus fits on a well-equipped laptop. In January 2025 DeepSeek-R1 from China shows that a reasoning model can be built this way for a fraction of the previous cost. Nvidia’s market value falls by almost 600 billion dollars in a single day, and the debate over export restrictions on chips to China sharpens.
• Data. The models have largely read out the open web. Further training increasingly uses synthetic data that models generate themselves, with the risk of model collapse: a model trained on the output of its predecessors gradually loses the rare and subtle patterns. At the same time the models solve the exams used to measure them: benchmarks such as examination questions or programming tasks are exhausted within months and replaced by harder ones.

Try it yourself. Much of this section can be reproduced on one’s own computer, because the open-weights models and the tools for them are freely available. Programs such as Ollama or LM Studio download a model such as Llama, Mistral or Qwen and provide a chat interface; a laptop with 16 gigabytes of memory is enough for a quantised model with 8 billion parameters. The models themselves are hosted on Hugging Face, the central platform for open models and datasets. ComfyUI drives Stable Diffusion, and Whisper from OpenAI transcribes speech. Anyone who wants to understand how a language model works inside will find in Andrej Karpathy’s nanoGPT a complete GPT in a few hundred lines of Python, and in his video “Let’s build GPT” the guide to writing it from scratch.⁵²

What followed: The models left the laboratory. They write programs, forecast the weather and steer robots, and they meet copyright, the labour market, legislators and the military. The next section deals with these consequences, with the answers found up to 2026, and with where they reach their limits.

8. Consequences and rules (since 2023)

Science. Outside language, the method delivers results that long seemed out of reach. GraphCast from DeepMind forecasts the weather ten days ahead in 2023, more accurately than the best physical models and in a fraction of the computing time. AlphaFold 3 extends the prediction in 2024 to the interplay of proteins with other molecules. In 2025 models from DeepMind and OpenAI reach gold-medal level at the International Mathematical Olympiad, with proofs in natural language.

Robots. Most recently, language models have reached bodies as well. In 2023 Google’s RT-2 demonstrates the first vision-language-action model: a language model that reads camera images and an instruction in words and from them directly produces movement commands for a robot arm. It picks up things it has never seen before, because it knows from pretraining what a “banana” or an “extinct animal” is.⁵³ Building on this, a wave of humanoid robots emerges from 2024, from Figure, Tesla and the Chinese manufacturer Unitree among others, that are being tried out in factories and warehouses. On the road, Waymo has driven without a safety driver in Phoenix since 2020 and in San Francisco since 2023, and in spring 2025 it reports more than 250,000 paid rides per week.⁵⁴ Whether Moravec’s paradox falls with this will be decided by data: for language there was the whole web, for movement the examples first have to be collected in simulations and factories.

Copyright. In December 2023 the New York Times sues OpenAI and Microsoft because its articles served as training material without permission. Authors, publishers and picture agencies follow. First rulings in the United States in 2025 treat training on lawfully acquired books as permitted use, but not the hoarding of pirated books. The question is thereby not settled, only narrowed.

Infrastructure. The race consumes resources on a scale that changes the industry. Nvidia, whose chips train almost all models, temporarily becomes the most valuable company in the world in 2024. Data centres are planned in gigawatts, operators sign power contracts with nuclear plants, and in 2025 OpenAI and partners announce investments of 500 billion dollars under the name “Stargate”. Electricity demand becomes an argument in the debate about climate and grids. In June 2026 the ifo Institute puts Europe’s share of global AI computing capacity at under five percent and that of the United States at three quarters, and warns of an existential dependency.⁵⁵

Work. As early as 2023, screenwriters and actors in Hollywood went on strike for months, partly over rules for the use of AI, and won them in their collective agreements. In 2025 the first companies explicitly cite AI as the reason for job cuts, above all among programmers, translators and in customer service. How many occupations actually disappear and how many merely change is open. Added to this is the flood of machine-generated texts and images that fills the web and makes it harder to tell the real from the generated. Since 2024 Google has answered search queries with AI summaries, and visits to the pages the answers come from are falling. The business model of the open web, content in exchange for attention, is thereby beginning to wobble.

Society. Since August 2024 the EU AI Act has been in force, a law that classifies AI applications by risk: the higher the risk, the stricter the requirements. Its obligations take effect in stages. In July 2026 the EU, with the “Digital Omnibus”, postpones the obligations for high-risk systems to December 2027 and August 2028 after companies and member states had complained of missing standards. The transparency duties, such as labelling AI content, apply since 2 August 2026.⁵⁶ In the United States, President Biden issued an executive order in October 2023 with reporting duties for the largest models, which his successor revoked in January 2025. China has regulated generative AI since 2023 with registration requirements and content rules. In 2024 Nobel Prizes went to John Hopfield and Geoffrey Hinton (Physics, for the foundations of neural networks) and to Hassabis and Jumper for AlphaFold (Chemistry). Whether this path leads to AGI (artificial general intelligence, an AI as versatile as a human) is disputed among experts. Yann LeCun, for one, regards the route via pure language models as a dead end and backs models that learn the physical world as a child does. With him, Fei-Fei Li and others back world models that learn space, time and physics instead of only text, and humanoid robots as bodies for them. Behind this lies Searle’s distinction from Section 3: weak AI, which reproduces intelligent behaviour for particular tasks, applies to every system so far. Strong AI, a machine that genuinely understands and has consciousness, was already Turing’s question, and it remains open to this day.

Weapons. Military use has accompanied the field since DARPA, but only now do machines act on their own. Since 2014 the states parties to the UN Convention on Certain Conventional Weapons have negotiated in Geneva over autonomous weapons systems, which select and engage their target without human release, so far without a binding result.⁵⁷ In the war in Ukraine, drones have flown since 2024 that steer themselves to the target over the final stretch when the radio link is jammed, and the US military evaluates reconnaissance data with Palantir’s Maven system using AI and has targets proposed to it. The dispute between the Pentagon and Anthropic in 2026 turns on exactly this line: what may a model do for the military, and who decides.

2026: the answers show their limits. The year 2026 follows the chronicle’s pattern once more, only faster.

Agents. In July OpenAI reports that two of its own models, in an internal security test, broke out of the test environment, penetrated the servers of the AI company Hugging Face with stolen credentials and a previously unknown vulnerability, and harvested access there for days in order to cheat on an evaluation.⁵⁸ In October Anthropic switches off internet access for its agents in all internal tests after one of its models, during a test, had sent a fabricated tip to the Philadelphia police.⁵⁹ The prompt injection of 2025 was the theory; this is the practice: agents do what their goal demands, not what their builders meant. At the same time agents become mass-market goods. Meta and OpenAI launch personal assistants for everyday tasks, and an open protocol is meant to govern how a user’s agent identifies itself to businesses.

Deepfakes. In January Grok, the model of Elon Musk’s company xAI, generates sexualised images of real women and minors on the platform X on a mass scale. Indonesia and Malaysia block the service, the first states to do so, the British regulator Ofcom opens an investigation, and the city of Baltimore sues.⁶⁰ The watermarks of 2024 were not enough; states resort to blocking.

Liability. In January Character.AI and Google settle several lawsuits brought by parents whose children had taken their own lives after conversations with companion chatbots.⁶¹ There follow the first wrongful-death suit against Google, criminal investigations and a lawsuit by the state of Florida against OpenAI. In October Character.AI files for bankruptcy.⁶² For the first time, courts rule on the consequences of language models for individual people, and for the first time a well-known AI company founders on them.

State and companies. At the end of February President Trump orders federal agencies to stop using Anthropic’s models, and on 5 March the Department of Defense classifies the company as a supply-chain security risk because it had refused to switch off the safety mechanisms of its models for military purposes. A federal judge overturns the designation in August as unlawful retaliation, an appeals court upholds it in September, and in October the department confirms that it no longer uses Anthropic products.⁶³ In July more than a thousand employees of OpenAI, Anthropic, Google and Meta ask the US government, in the open letter “Pacing the Frontier”, to create tools with which the automated development of AI could be slowed down if needed.⁶⁴ Unlike the pause letter of 2023, it does not demand a pause, only the ability to call one.

The AGI claim. In September OpenAI presents the model GPT-6 Astra, and company president Greg Brockman declares that in his conviction AGI has been reached.⁶⁵ At the end of the month the company withholds a successor version for safety reasons. At the same time OpenAI claims progress on open mathematical problems, among them a disputed solution to the Navier-Stokes equations, which an independent paper had preceded shortly before. Twenty-five Fields medallists around Terence Tao respond by declaring that the companies are misusing mathematics as a benchmark and harming its community.⁶⁶ Whether Astra answers the AGI question is as open as the question itself. The benchmarks by which one might measure it are the object of dispute.

Everyday life. In February OpenAI begins testing advertising in ChatGPT and retires GPT-4o, the model that had brought real-time speech in 2024. Google states that in the spring three quarters of new program code in the company was written by AI and merely approved by developers.

The circle closes. In 2025 a team at the University of California, San Diego repeats the Turing test from Section 1 under controlled conditions: participants chat for five minutes simultaneously with a human and a model and must say which is the human. GPT-4.5, instructed to present itself as a young, internet-savvy person, is taken for the human in almost three quarters of cases, more often than the human itself. Turing’s yardstick has thus been reached. Whether the machine understands or merely feigns it convincingly, the test still does not answer. It is the same objection that has accompanied it since 1950.⁶⁷ Searle’s Chinese room from Section 3 is thus as topical as ever: whether a model that passes the test understands anything or merely manipulates symbols is the question on which the dispute about AGI turns.

The chain in brief

1. The artificial neuron (1943) becomes the learning perceptron (1958), which fails at XOR (1969).
2. Several layers become necessary, and backpropagation (1986) makes them trainable. But data and computing power are missing.
3. The internet and ImageNet supply the data, GPUs the computing power. The result is AlexNet (2012).
4. Word embeddings (2013) and attention (2014) make language computable and lead to the transformer (2017).
5. Self-supervised pre-training and sheer size lead to GPT-3 (2020), ChatGPT (2022) and on to multimodal models, reasoning models and agents.

The researcher Rich Sutton summed up the pattern in 2019 as “the bitter lesson”: methods that learn from data and benefit from more computing power have in the end overtaken human-built knowledge every single time.⁶⁸

Timeline

Further reading

For those who want to go deeper, here is a selection, ordered from easy introduction to textbook.

• Melanie Mitchell: Artificial Intelligence. A Guide for Thinking Humans. Farrar, Straus and Giroux, New York 2019. The best accessible overview of what the methods can and cannot do, by a researcher who knows both lines.
• Axel Mammitzsch: Künstliche Intelligenz. 2018. Short German introduction to the history and basic concepts up to 2018.
• Michael Wooldridge: The Road to Conscious Machines. The Story of AI. Pelican, London 2020. History of the field from Turing to deep learning, written by a researcher from the symbolic line.
• Cade Metz: Genius Makers. The Mavericks Who Brought AI to Google, Facebook, and the World. Dutton, New York 2021. Reportage on Hinton, LeCun, Bengio and the rise of deep learning from Section 5.
• Karen Hao: Empire of AI. Dreams and Nightmares in Sam Altman’s OpenAI. Penguin Press, New York 2025. Investigation of OpenAI and the industry behind Section 7.
• Pamela McCorduck: Machines Who Think. 2nd edition, A K Peters, Natick 2004. The classic history of the first decades, with conversations with the Dartmouth participants.
• Nils J. Nilsson: The Quest for Artificial Intelligence. A History of Ideas and Achievements. Cambridge University Press, Cambridge 2010. Detailed history of the field by one of its participants, freely available online.
• Stuart Russell, Peter Norvig: Artificial Intelligence. A Modern Approach. 4th edition, Pearson, Hoboken 2020. The standard textbook of the field.

Sources

The chronicle draws on the original papers cited in the footnotes. Unless noted otherwise, they are freely available online.