Automation Complacency is reduced vigilance with automated systems. It sits in the Incentives dimension (INC) of the Human Behavior Taxonomy™ as element HBT-INC-0038, within the Trust family. The core principle: reduced vigilance with automated systems. In incentive terms, it matters because it changes the payoff people perceive before they choose — which means it can be designed for, or exploited.
Scientific Definition
Reduced vigilance with automated systems.
Plain-English Definition
Reduced vigilance with automated systems.
Feynman Explanation
Human-in-the-loop is fine until the human isn't.
Core Principle
Reduced vigilance with automated systems.
Mechanisms
Pending editorial review.
Pending editorial review.
Pending editorial review.
Pending editorial review.
Pending editorial review.
Pending editorial review.
Pending editorial review.
Inputs (Triggers)
Pending editorial review.
Outputs (Behaviors)
Pending editorial review.
Behavioral Signature
Human-in-the-loop is fine until the human isn't.
Examples
- Aviation, manufacturing, and now AI agents.
- Predictable human factor in oversight systems.
Pending editorial review.
Original analysis from The Incentives Lab — how this element behaves inside real payoff structures.
Why this element matters to incentive design
When this element shows up in a diagnostic, the instinct is to train people out of it. Training rarely moves it. The mechanism underneath it operates in the Incentives dimension — what makes behavior more or less likely?. You can recognize it in the field by its signature: human-in-the-loop is fine until the human isn't. Every element in the Incentives dimension changes the perceived payoff of an action before the action happens, which is exactly where incentive design has leverage.
How it gets exploited
Left undesigned, predictable human factor in oversight systems. It is amplified whenever predictable human factor in oversight systems. Inside organizations that shows up as predictable human factor in oversight systems. The pattern is the same one Goodhart's Law describes: the measurable proxy attracts the effort, and the purpose behind it quietly loses funding.
How the Lab designs around it
The redesign move is to designed friction. Periodic manual exercises. Active monitoring. Treat it as infrastructure. Once you can see it in your own system, most of the argument about culture resolves itself.
Famous Experiments
Pending editorial review.
Design Principles
- Designed friction. Periodic manual exercises. Active monitoring.
Measurement Approaches
Pending editorial review.
Evidence
Pending editorial review.
Pending editorial review.
The Perverse Incentive Lens™
How this behavior is exploited — and how to redesign around it.
- Designed friction. Periodic manual exercises. Active monitoring.
Pending editorial review.
Pending editorial review.
Interactive Mini Network
Click any neighbor to re-center the graph and follow the threads of connection.
Knowledge Graph Neighbors
Auto-linked to the rest of the Human Behavior Taxonomy by family, domain, dimension, and shared keywords.
Treating AI as more humanlike than it is.
Quantifying uncertainty in model outputs.
Confident outputs that are factually wrong.
Matching trust in a system to its actual reliability.
Augment when judgment matters. Automate when scale matters.
Written rules about how AI may be used internally.
Innovators → early adopters → majority → laggards, AI-specific.
AI system that takes actions to achieve goals, often across tools.
When the agent acts, who's responsible?
Augmentation strategy vs. substitution strategy.
Logging of AI inputs, outputs, and decisions.
Inventory of models, data, tools, and dependencies in an AI system.
Where Automation Complacency is cited in the corpus
Essays, field guides, and diagnostics from The Incentives Lab that apply this element.
- Field guideIncentives: definition, types, examples
The parent field guide for this element.
- ReferenceThe laws of incentives
Goodhart, Campbell, and the Cobra Effect.
- EssayAI Agents Inherit Your Incentives
How this element propagates into automated systems.
- CourseIncentives 101
The free ten-part primer on reading a payoff structure.
- ReferenceThe Periodic Table of Human Behavior
The full 1,267-element map this page belongs to.
Questions about Automation Complacency
- What is Automation Complacency?
- Automation Complacency is reduced vigilance with automated systems. It sits in the Incentives dimension (INC) of the Human Behavior Taxonomy™ as element HBT-INC-0038, within the Trust family. The core principle: reduced vigilance with automated systems. In incentive terms, it matters because it changes the payoff people perceive before they choose — which means it can be designed for, or exploited.
- What is an example of Automation Complacency?
- Predictable human factor in oversight systems. The Incentives Lab catalogs everyday, organizational, and historical instances of this element on its Human Behavior Taxonomy™ page (HBT-INC-0038).
- How is Automation Complacency exploited?
- Predictable human factor in oversight systems.
- How do you design around Automation Complacency?
- Designed friction. Periodic manual exercises. Active monitoring.
- Which behavioral dimension does Automation Complacency belong to?
- Automation Complacency is classified in the Incentives dimension (INC) of the Human Behavior Taxonomy™, family "Trust", class "AI-Behavioral Coupling". Its permanent identifier is HBT-INC-0038 and its evidence grade is C.