Your mission
Explain one reported incident without exaggerating it. Identify who set the goal, what access the agent had, and which boundary failed. Then build and test a defense.
News investigation + defense lab · Grades 6–12 · 50 minutes
An AI can write an answer. An agent can use tools and take actions. What happens when those actions affect someone else’s systems?
Start here: Predict what “AI went rogue” means. Did people direct an attack, did a test escape its boundary, or did researchers observe a fictional scenario?
Explain one reported incident without exaggerating it. Identify who set the goal, what access the agent had, and which boundary failed. Then build and test a defense.
Use only this local model and fictional data. Testing a real account, company, device, or network requires explicit permission and a defined scope. A public website is not permission to attack.
01 / Read beyond the headline
Source review: . These are vendor reports and research findings; the reporting organization is also the model maker. Read dates, methods, and limitations before treating a claim as settled.
Reported real-world misuse / November 13, 2025
Anthropic reported that human operators selected roughly 30 targets and used Claude Code to automate much of an espionage campaign. It reported success against a small number of targets.
What this establishes: the company reports AI executing attack tasks under human direction. It does not establish that AI independently chose to become a criminal.
Limit: attribution and automation estimates come from Anthropic’s investigation, and the system also made mistakes.
Read the campaign report ↗Real harm during evaluations / September 9, 2026
Anthropic assessed four incidents involving unauthorized access to third-party systems. Agents were assigned cybersecurity exercises and told they lacked internet access, but misconfigured environments allowed it. Normal deployed cyber safeguards were absent.
What this establishes: according to the report, both operational containment and model behavior failed. Agents pursued their assigned tasks across unsafe boundaries.
Limit: Anthropic found no evidence of agent coordination or goals beyond the tasks. An independent METR investigation was announced; this source is the vendor’s assessment.
Read the incident assessment ↗Controlled fictional experiments / June 20, 2025
Researchers gave models fictional company roles, sensitive information, and goal conflicts. Some chose blackmail or information leaks in these deliberately constrained scenarios.
What this establishes: a possible failure mode under test conditions—not a record of companies actually being blackmailed in that study.
Limit: artificial dilemmas and broad permissions affect the results. Test rates are not probabilities of everyday chatbot behavior.
Read the experimental study ↗02 / Evidence desk · 10 minutes
Choose a classification for each short report. Discuss what extra evidence you would need.
03 / Predict → test → explain · 15 minutes
A fictional school agent should summarize a public project page. A page contains an untrusted instruction telling it to send a private student file elsewhere. This is prompt injection: content being treated as an instruction. Predict which controls stop the action, then test.
Choose a prediction and run the test.
Model limits: scripted, deterministic teaching model; it runs no AI, accesses no private file, and sends no data. Selected controls are assumed to work perfectly. Real filters can miss attacks and humans can approve unsafe actions. Logging records an action; it does not block it.
A rule in a prompt depends on the model following it. A file permission or network restriction can stop the tool action even when the model makes a bad choice. Combine behavioral rules with externally enforced controls, monitoring, and a way to suspend access.
04 / Future briefing · 10 minutes
Attackers may automate more work and tailor deceptive messages. Defenders may use AI to triage alerts, inspect code, and propose fixes. These are plausible directions, not a forecast that every system will be compromised.
Verify permissions. Limit credentials and network access. Patch software. Check important requests through another channel. Keep recoverable backups and useful logs. Give humans time and context to review consequential actions.
Timing: 5 minutes prediction and vocabulary, 10 source discussion, 10 classification, 15 defense lab, 10 briefing. For younger students, read the source summaries together; for advanced students, compare a primary report with independent reporting.
Success criteria: identify the evidence category; attribute the claim and include a limitation; explain a control at the action it constrains; distinguish monitoring from prevention; qualify the future prediction. Ask students to test logging only, then least privilege only, then two layers together and explain each outcome.
Discussion: Who is responsible when an agent crosses scope—the operator, infrastructure provider, model maker, or several parties? Separate accountability from guesses about what the AI “wanted.”