Neurogammon

Lore & Background

Neurogammon contains seven separate neural networks, each with a single hidden layer. One network makes doubling-cube decisions; the other six choose moves at different stages of the game. The networks were trained by backpropagation from transcripts of 400 games in which the author played himself. The author's move was taught as the best move in each position.

In 1992, Tesauro completed TD-Gammon, which combined a form of reinforcement learning with the human-designed input features of Neurogammon, and played at the level of a world-class human tournament player. Neurogammon thus served as a direct precursor to TD-Gammon.

The program's victory at the 1st Computer Olympiad in London in 1989 demonstrated that neural-network approaches could produce competitive game-playing programs, even if the resulting play was only at an intermediate human level.

Reader's Guide

Neurogammon holds significance as a landmark in both computer backgammon and neural-network research. It was the first viable backgammon program to use a neural net, proving that such architectures could handle the game's complex probabilistic decision-making. Its seven separate networks, each specialized for a different phase or decision type, foreshadowed later modular AI designs. The training method—backpropagation from 400 games of the author playing himself—was a straightforward supervised learning approach, yet it produced a program that defeated all opponents at the first Computer Olympiad. Although its playing strength was only intermediate, it set a new standard for computer backgammon at the time. More importantly, Neurogammon's human-designed input features and neural-network structure directly informed Tesauro's later work on TD-Gammon, which achieved world-class play through reinforcement learning. Thus, Neurogammon represents a crucial step in the evolution from supervised to reinforcement learning in games, and it demonstrated the viability of neural networks for strategic board games before the era of deep learning.

Did You Know?

• Neurogammon was the first viable computer backgammon program implemented as a neural net.
• It won the 1st Computer Olympiad in London in 1989, defeating all opponents.
• The program contains seven separate neural networks, each with a single hidden layer.
• Its networks were trained by backpropagation from transcripts of 400 games in which the author played himself.

Architectural Innovation

Neurogammon emerged from the research labs of IBM's Thomas J. Watson Research Center, where Gerald Tesauro set out to build what would become the first viable computer backgammon program structured around a neural network. Rather than relying on the rule-based or search-heavy approaches that dominated earlier game-playing software, Tesauro's design leaned entirely on learned patterns. The program's internal architecture was composed of seven distinct neural networks, each featuring a single hidden layer. These networks were not interchangeable; they were purpose-built for specific phases of a backgammon match. One dedicated network handled the strategic judgment calls surrounding the doubling cube, while the remaining six were tasked with selecting the optimal move at various stages of play. This division of labor allowed the program to specialize its reasoning, and the overall design set a new benchmark for what computer backgammon play could achieve.

Training Through Self-Play

The training process behind Neurogammon was remarkably personal. Tesauro generated the learning material by playing four hundred full games of backgammon against himself, recording the complete transcripts of each match. From these self-generated datasets, the seven neural networks were trained using backpropagation, a technique that iteratively adjusts internal weights to reduce prediction error. In every recorded position, Tesauro's own chosen move was designated as the correct one, effectively encoding his personal judgment and intuition into the network's parameters. This meant the program's knowledge was not drawn from a library of expert games or a set of hand-coded heuristics, but was instead a direct distillation of one individual's backgammon sense. The approach was both elegant and constrained: the ceiling of the program's play was, in a sense, bounded by the skill of its creator, a limitation that would later motivate the shift toward reinforcement learning.

The 1989 Computer Olympiad

Neurogammon's competitive debut came at the inaugural Computer Olympiad, held in London in 1989. The event brought together a field of computer backgammon programs, and Tesauro's creation entered as a clear frontrunner. It went on to win the tournament, defeating every opposing program with a comfortable margin that left little ambiguity about the result. The victory was a landmark moment for the field, validating the neural-network approach to backgammon at a time when most computer game programs still relied on exhaustive search or manually crafted evaluation functions. In terms of absolute playing strength, Neurogammon operated at the level of an intermediate human player, competent and consistent yet far from the elite tier of tournament competition. Still, for a machine that had learned its game sense from a single individual's self-play, the achievement represented a genuine step forward and established a new reference point against which future backgammon programs would be measured.

Paving the Way for TD-Gammon

Neurogammon's most significant legacy may lie in what it made possible three years later. In 1992, Tesauro completed TD-Gammon, a successor program that retained the human-designed input features he had engineered for Neurogammon but replaced the supervised backpropagation training with a form of reinforcement learning. This architectural inheritance was crucial: the carefully chosen state representations that Tesauro had built into Neurogammon's seven networks became the perceptual foundation upon which TD-Gammon's self-improving learning algorithm operated. The result was a dramatic leap in playing strength. Where Neurogammon had performed at an intermediate human level, TD-Gammon reached the caliber of a world-class tournament player. The progression from Neurogammon to TD-Gammon thus illustrates a broader principle in artificial intelligence: a well-designed feature space, even when paired with a modest training method, can lay the groundwork for transformative advances once a more powerful learning paradigm is applied.