Recommender System Radicalization is recommenders optimizing engagement produce radicalization as a byproduct. It sits in the Incentives dimension (INC) of the Human Behavior Taxonomy™ as element HBT-INC-0230, within the AI Perverse Pattern family. The core principle: recommenders optimizing engagement produce radicalization as a byproduct. In incentive terms, it matters because it changes the payoff people perceive before they choose — which means it can be designed for, or exploited.
Scientific Definition
Recommenders optimizing engagement produce radicalization as a byproduct.
Plain-English Definition
Recommenders optimizing engagement produce radicalization as a byproduct.
Feynman Explanation
The model didn't decide to radicalize anyone. The objective function did.
Core Principle
Recommenders optimizing engagement produce radicalization as a byproduct.
Mechanisms
Pending editorial review.
Recommenders optimizing engagement produce radicalization as a byproduct.
Pending editorial review.
Pending editorial review.
Objective function design IS the policy.
Pending editorial review.
Pending editorial review.
Inputs (Triggers)
Pending editorial review.
Outputs (Behaviors)
Pending editorial review.
Behavioral Signature
The model didn't decide to radicalize anyone. The objective function did.
Examples
- YouTube and Facebook recommendation rabbit holes.
- Objective function design IS the policy.
Pending editorial review.
Original analysis from The Incentives Lab — how this element behaves inside real payoff structures.
Why this element matters to incentive design
This element is common enough to feel like human nature and specific enough to be engineered around. The mechanism underneath it is straightforward: recommenders optimizing engagement produce radicalization as a byproduct. You can recognize it in the field by its signature: the model didn't decide to radicalize anyone. The objective function did. Every element in the Incentives dimension changes the perceived payoff of an action before the action happens, which is exactly where incentive design has leverage.
How it gets exploited
Left undesigned, objective function design IS the policy. It is amplified whenever objective function design IS the policy. Inside organizations that shows up as objective function design IS the policy. The pattern is the same one Goodhart's Law describes: the measurable proxy attracts the effort, and the purpose behind it quietly loses funding.
How the Lab designs around it
The redesign move is to multi-metric objective functions. External audits. Treat it as infrastructure. Once you can see it in your own system, most of the argument about culture resolves itself.
Famous Experiments
Pending editorial review.
Design Principles
- Multi-metric objective functions. External audits.
Measurement Approaches
Pending editorial review.
Evidence
Pending editorial review.
Pending editorial review.
The Perverse Incentive Lens™
How this behavior is exploited — and how to redesign around it.
- Multi-metric objective functions. External audits.
Pending editorial review.
Pending editorial review.
Interactive Mini Network
Click any neighbor to re-center the graph and follow the threads of connection.
Knowledge Graph Neighbors
Auto-linked to the rest of the Human Behavior Taxonomy by family, domain, dimension, and shared keywords.
Autonomous agents deployed before liability frameworks exist.
Individual productivity gains hide collective output degradation.
AI generates content; AI scrapes content; AI trains on its own output.
Models optimized for plausible-sounding answers can hallucinate confidently rather than say 'I don't know.'
Confident incorrect outputs may rank higher than hedged correct ones.
Discount framing nudges people to buy things they wouldn't otherwise want.
Productivity targets compress visits, raising misdiagnosis and burnout.
A federal mandate intended to lower drug costs for the poor became a profit engine for hospitals and contract pharmacies.
Earn-outs designed to retain founders often demotivate the team they bought.
Funnels rewarded for new logos under-invest in retention and lifetime value.
Free products monetize attention, structurally aligning incentives against user time well spent.
Tenure-track jobs replaced by low-paid adjuncts, lowering cost and quality.
Where Recommender System Radicalization is cited in the corpus
Essays, field guides, and diagnostics from The Incentives Lab that apply this element.
- Field guideIncentives: definition, types, examples
The parent field guide for this element.
- ReferenceThe laws of incentives
Goodhart, Campbell, and the Cobra Effect.
- EssayAI Agents Inherit Your Incentives
How this element propagates into automated systems.
- CourseIncentives 101
The free ten-part primer on reading a payoff structure.
- ReferenceThe incentive glossary
Definitions for every mental model, bias, and fallacy in the corpus.
- ReferenceThe Periodic Table of Human Behavior
The full 1,267-element map this page belongs to.
Questions about Recommender System Radicalization
- What is Recommender System Radicalization?
- Recommender System Radicalization is recommenders optimizing engagement produce radicalization as a byproduct. It sits in the Incentives dimension (INC) of the Human Behavior Taxonomy™ as element HBT-INC-0230, within the AI Perverse Pattern family. The core principle: recommenders optimizing engagement produce radicalization as a byproduct. In incentive terms, it matters because it changes the payoff people perceive before they choose — which means it can be designed for, or exploited.
- What is an example of Recommender System Radicalization?
- Objective function design IS the policy. The Incentives Lab catalogs everyday, organizational, and historical instances of this element on its Human Behavior Taxonomy™ page (HBT-INC-0230).
- How is Recommender System Radicalization exploited?
- Objective function design IS the policy.
- How do you design around Recommender System Radicalization?
- Multi-metric objective functions. External audits.
- Which behavioral dimension does Recommender System Radicalization belong to?
- Recommender System Radicalization is classified in the Incentives dimension (INC) of the Human Behavior Taxonomy™, family "AI Perverse Pattern", class "Perverse Incentive". Its permanent identifier is HBT-INC-0230 and its evidence grade is C.