AI Origins · Interactive Lab 12

The 20Q Mind Reader

One secret thought. Twenty questions.

Think of an animal, historical figure, object, place, or fictional character. Answer on the handheld, then watch the inspector narrow its search. Use Not sure to skip a clue.

103 built-in candidates · Adaptive questions · Learns in this browser

20Q · BAYESIAN GUESSING ORB
Q: 01/20 H: 6.78 BITS
PROBING CLUES

Think of any animal, scientist, tool, or wonder. Press any button to start!

Shannon Split: +1.00 bit expected Information Gain

UNCERTAINTY LOWER = LESS UNCERTAINTY

Your answers are clues. The machine updates its beliefs.

Shannon Telemetry Bits & Entropy
6.78
Entropy (H)
103
Live Candidates
0.00
Bits Removed
0.98
Info Gain / Q
Claude Shannon's Optimal Split Top Information Gain (IG)

The AI evaluates every remaining question and chooses the one that maximizes Information Gain (IG = Hbefore − Hafter). The best yes/no question splits the remaining candidates close to half and half, which removes up to one bit.

Bayesian Belief Distribution Top Candidates

Candidate probabilities updated continuously after every clue using Bayes' Theorem.

Decision Tree Path Past Clues

No questions asked yet. Think of an animal, thinker, or invention and press a button!

The Mathematical Secret Behind the Game

The 20-Question Mind Reader: Claude Shannon, Information Theory & AI

How 20 simple yes/no questions can pinpoint 1 in a million possibilities — and how modern AI uses entropy to make optimal decisions.

8 possibilitiesBefore the first clue
4 possibilitiesOne balanced question
2 possibilitiesTwo balanced questions
1 answerThree bits of information

1. The 1988 Handheld Miracle: Robin Burgener's 20Q

In 1988, inventor Robin Burgener wrote a small neural-network program that played the traditional game of Twenty Questions. It went online in the 1990s, where players’ answers trained it, and in the early 2000s it was manufactured as the iconic spherical handheld toy 20Q by Radica Games. Millions of players around the world were shocked: they could think of something completely obscure — like a platypus, a telescope, or a stapler — and within 20 questions, the palm-sized plastic orb would reveal their exact thought on its retro liquid-crystal display.

Later, web algorithms like Akinator expanded this concept to millions of fictional and real characters. But neither 20Q nor Akinator possesses supernatural intuition. Instead, they exploit one of the most profound mathematical breakthroughs of the 20th century: Claude Shannon’s Mathematical Theory of Communication, published at Bell Laboratories in 1948.

2. Claude Shannon and the Birth of the “Bit” (1948)

Before 1948, the word information was vague. People spoke of messages having meaning, wisdom, or noise, but there was no mathematical scale to measure how much information a message actually contained.

Claude Shannon realized that information is fundamentally about surprise and uncertainty reduction:

  • If someone tells you, “The Sun will rise in the East tomorrow morning,” they have told you something with near 100% probability. You learned virtually nothing new: zero surprise = zero information.
  • If someone flips a fair coin under a cup and tells you, “It landed on Heads,” you were completely undecided between two equally likely outcomes (50/50). Learning the result gives you exactly one unit of information.

Shannon gave this fundamental unit a name suggested by his Bell Labs colleague, the statistician John Tukey: the binary digit, or bit. One bit is the amount of information required to choose between two equally likely alternatives.

Portrait photograph of Claude Shannon, the founder of information theory
Claude Shannon: Father of Information Theory (1916–2001) Claude Shannon worked at Bell Laboratories, where his 1948 paper A Mathematical Theory of Communication showed that any message can be measured in bits, and set the limit on how much a noisy channel can carry. Unknown photographer · Tekniska museet (Sweden), item 43069 · CC BY 2.0 · cropped, via Wikimedia Commons

Exponential Scaling

3. Why 20 Questions? The Power of 220

Most people are amazed that 20 questions is enough to guess virtually anything. But the math of binary branching explains why:

220 = 2 × 2 × 2 × … × 2 = 1,048,576

Every time a question splits the remaining possibilities in half, the search space shrinks exponentially:

Distinguishable Items (2N): 1,048,576
Over a million distinct items!

With just 10 questions, an ideal split separates 210 = 1,024 possibilities. With 20 questions, you can theoretically pinpoint one specific object out of over one million possibilities!

The Formula

4. Shannon Entropy: Measuring Uncertainty in Bits

How do we mathematically measure the uncertainty of a system when candidates aren't all equally likely?

Shannon defined the Information Entropy H(X) of a probability distribution P = {p1, p2, …, pn} as:

H(X) = − ∑ pi log2(pi)

pi (Probability)

The current belief or likelihood that item i is the one the user has in mind. The sum of all pi always equals 1.0.

log2(pi) (Bits)

Logarithm base 2 converts probabilities into bits. Because pi ≤ 1, log2(pi) is negative, so the leading minus sign makes entropy positive.

H(X) (Total Entropy)

The weighted average number of bits of surprise remaining. When H = 0, certainty is 100% — we know the answer!

Interactive Experiment: The Shannon Entropy Curve for a Binary Decision

Drag the probability slider p below to see how uncertainty changes. Notice that entropy reaches its maximum value of exactly 1.000 bit when p = 0.50 (a pure 50/50 question)!

Remaining Uncertainty: 1.000 bits
Formula: H = −p log2(p) − (1−p) log2(1−p)
Why the peak at 50/50? If an answer is guaranteed to be Yes (p = 1.0) or No (p = 0.0), asking the question gives you zero new information. The question that teaches the AI the most is always the one that divides the remaining candidates in half!

The AI Algorithm

5. How Decision Trees & Machine Learning Pick the Perfect Split

How the 20Q engine and machine learning algorithms decide what to ask next.

This game compares a curated set of candidates, including familiar fictional characters. It is independent of the Akinator service. Inspector percentages describe relative belief among known candidates, so a high percentage does not prove that your item is in the knowledge base. Taught items, their answer history, and new distinguishing questions are stored in this browser when local storage is available.

When a machine learning algorithm builds a decision tree (such as ID3 or C4.5; CART and Random Forests usually score splits with a close cousin, Gini impurity), it doesn't choose questions at random. It calculates the Information Gain (IG) for every candidate question:

IG(Q) = Hcurrent − [ P(Yes) · H(X | Yes) + P(No) · H(X | No) ]

The question with the highest IG is the one chosen. In our 20Q engine:

  1. 1Evaluate Candidates: For all informative questions not yet asked, calculate what the entropy would be if the user said “Yes” versus “No”.
  2. 2Maximize Gain: Pick the question that brings the remaining entropy closest to zero the fastest.
  3. 3Bayesian Update: When the user responds, multiply each candidate's current probability by how likely that answer would be for it, then re-normalize so the probabilities add up to 1.

A Binary Decision Tree Splitting 23 = 8 Outcomes with 3 Bits

Q1 YES NO Q2 Q3 Elephant Eagle Telescope Pyramids

Connection to Modern AI

6. How Large Language Models (ChatGPT, Gemini, Claude) Use Information Theory

Information theory is also the mathematics used to train modern AI models.

Cross-Entropy Loss (L)

When training an LLM, the loss function is Cross-Entropy. It measures how many unnecessary bits of surprise the model suffered when trying to guess the next word of human text. Lower cross-entropy means the model predicted the text better.

Perplexity (2H)

AI research papers often report this as Perplexity, defined as 2H. A perplexity of 16 means the language model is as confused as someone making a blind choice between 16 equally likely words.

Temperature & Entropy

When you set an AI's temperature to 0.0, it always picks the single most probable next token, so there is no randomness left in the choice. Higher temperature flattens the probabilities (more entropy), making responses more varied and less predictable.

Classroom Challenge Cards

Test the boundaries of Information Theory with these hands-on lab missions.

Lab Missions

  • Mission 1The 10-Question Speedrun: Think of a widely known mammal (e.g. Lion or Elephant). Can the AI guess it in under 10 questions? Observe how many bits of entropy were removed per question.
  • Mission 2The Obscure Test: Think of an unusual science invention or astronomical concept (e.g. Microscope, International Space Station, or Black Hole). Watch which question acts as the decisive filter.
  • Mission 3Noise & Fault Tolerance: Answer one question with “Probably Not” even when it is true. Observe how Bayesian inference recovers from human error without getting stuck!

Information Theory Investigations

  • Mission 4The Entropy Tracker: Record the entropy H after Question 1, Question 5, and Question 10 in your lab notebook. Graph the curve of decay.
  • Mission 5Beat the Oracle: Think of an entity that is not in the database. When the Oracle gives up, press Teach my item and use the form to add it and its distinguishing question.
  • Mission 6The Perfect 50/50 Question: If you had to divide your entire classroom in half with a single question, what question would you ask to maximize Shannon Information Gain?

Reflection + Teacher Notes

Prompts for class discussion, STEAM notebooks, and lesson worksheets.

Reflection Prompts

  • Prompt 1Why is a question with a 50/50 answer more valuable to an AI than a question with a 99/1 answer?
  • Prompt 2What does it mean for an AI to “learn” from feedback when a player teaches it a new item?
  • Prompt 3How does Claude Shannon's concept of a “bit” connect phone communication, computer memory, and twenty questions?

Teacher Moves

  • VocabularyBit, Entropy (H), Information Gain (IG), Bayesian Inference, Prior Distribution, Decision Tree, Cross-Entropy.
  • Classroom GameHave two students play Twenty Questions aloud. After each question, ask the class: “Did that cut the possibilities in half, or was it a wasted question?”
  • WorksheetPrint this lesson for student notebooks using the Print button at the top of the page.