Donald Michie · 1961 Mechanical Reinforcement Learner

MENACE — Matchbox Educable Noughts & Crosses Engine

Before digital reinforcement learning, Donald Michie built a tic-tac-toe computer from 304 matchboxes and colored glass beads. Each box holds beads for every legal move; winning adds beads, losing removes them.

01 / Untrained Baseline Current

At Game 0, every square in a matchbox starts with equal beads (4 beads on Turn 1). MENACE plays purely at random. Play a game as O and beat it—watch its chosen boxes lose a bead (−1).

02 / Train 20 Games Next

Click Train 20 Games. MENACE spars against a tactical partner that punishes blunders: winning trails gain +3 beads, draws gain +1, and losses lose −1.

03 / Test & Rewind Later

Play another game against the trained engine and compare the bead counts inside its matchboxes. Use the Timeline below to rewind, replay, or branch from any past move.

Timeline LIVE

Playing Board

MENACE = X · You = O
How to read the machine:
Boxes
Rotated/mirrored boards share 1 canonical matchbox so MENACE learns 8× faster.
Beads
Numbers in empty squares show bead counts. Higher count = higher probability of being drawn.
+3/+1/−1
After a game, every box opened in that game's trail has its chosen bead reinforced.

Current Game Matchboxes Opened (Trail)

Boxes opened by MENACE on Turns 1, 3, 5, 7
Bead drawn this game Number = Beads in square (hover for %) Σ = Total beads in box +3 / +1 or −1 = Game-end reinforcement

All Matchboxes in Memory

Canonical 3×3 board states sorted by turn ply