The Hanson-Yudkowsky AI-Foom Debate
In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on Overcoming Bias, mainly between Robin Hanson and Eliezer Yudkowsky. It focused on the likelihood of hard AI takeoff ("FOOM"), the need for a theory of Friendliness, and the future of AGI, whole brain emulation, and recursive self-improvement in general. A stand-up debate on the same themes took place at Jane Street Capital in June 2011, and Hanson made a further post in 2014.
The best resource for most readers new to the debate will be MIRI's 2013 The Hanson-Yudkowsky AI-Foom Debate eBook, available in PDF, EPUB, and MOBI formats. This includes
- the original series of blog posts,
- a transcript of the 2011 debate
- Kaj Sotala's summary and analysis of the debate, and
- a 2013 technical report on AI takeoff dynamics (“intelligence explosion microeconomics”) written by Yudkowsky.
Prologue
- Fund UberTool? by Robin Hanson
- Engelbart as UberTool? by Robin Hanson
- Friendly Teams by Robin Hanson
- Friendliness Factors by Robin Hanson
- The Weak Inside View by Eliezer Yudkowsky
- Setting The Stage by Robin Hanson
- The First World Takeover by Eliezer Yudkowsky
- Abstraction, Not Analogy by Robin Hanson
- Whence Your Abstractions? by Eliezer Yudkowsky
Main sequence
- AI Go Foom by Robin Hanson
- Optimization and the Singularity by Eliezer Yudkowsky
- Eliezer's Meta-Level Determinism by Robin Hanson
- Observing Optimization by Eliezer Yudkowsky
- Life's Story Continues by Eliezer Yudkowsky
- Emulations Go Foom by Robin Hanson
- Brain Emulation and Hard Takeoff by Carl Shulman
- Billion Dollar Bots by James Miller
- Surprised by Brains by Eliezer Yudkowsky
- “Evicting” brain emulations by Carl Shulman
- Cascades, Cycles, Insight... by Eliezer Yudkowsky
- When Life Is Cheap, Death Is Cheap by Robin Hanson
- ...Recursion, Magic by Eliezer Yudkowsky
- Abstract/Distant Future Bias by Robin Hanson
- Engelbart: Insufficiently Recursive by Eliezer Yudkowsky
- Total Nano Domination by Eliezer Yudkowsky
- Dreams of Autarky by Robin Hanson
- Total Tech Wars by Robin Hanson
- Singletons Rule OK by Eliezer Yudkowsky
- Stuck In Throat by Robin Hanson
- Disappointment in the Future by Eliezer Yudkowsky
- I Heart CYC by Robin Hanson
- Is the City-ularity Near? by Robin Hanson
- Recursive Self-Improvement by Eliezer Yudkowsky
- Whither Manufacturing? by Robin Hanson
- Hard Takeoff by Eliezer Yudkowsky
- Test Near, Apply Far by Robin Hanson
- Permitted Possibilities, & Locality by Eliezer Yudkowsky
- Underconstrained Abstractions by Eliezer Yudkowsky
- Beware Hockey Stick Plans by Robin Hanson
- Evolved Desires by Robin Hanson
- Sustained Strong Recursion by Eliezer Yudkowsky
- Friendly Projects vs. Products by Robin Hanson
- Is That Your True Rejection? by Eliezer Yudkowsky
- Shared AI Wins by Robin Hanson
- Artificial Mysterious Intelligence by Eliezer Yudkowsky
- Wrapping Up by Robin Hanson
- True Sources of Disagreement by Eliezer Yudkowsky
- The Bad Guy Bias by Robin Hanson
- Disjunctions, Antipredictions, Etc. by Eliezer Yudkowsky
- Are AIs Homo Economicus? by Robin Hanson
- Two Visions Of Heritage by Robin Hanson
- The Mechanics of Disagreement by Eliezer Yudkowsky
Conclusion
- What Core Argument? by Robin Hanson
- What I Think, If Not Why by Eliezer Yudkowsky
- Not Taking Over the World by Eliezer Yudkowsky
Postscript
- We Agree: Get Froze by Robin Hanson
- You Only Live Twice by Eliezer Yudkowsky
- Debating Yudkowsky by Robin Hanson
- Foom Debate, Again by Robin Hanson
Jane Street debate
A stand-up debate between Hanson and Yudkowsky on the same themes took place on Wednesday June 30 2011 at Jane Street Capital.
- Video with better quality audio 1.57G
- Poor quality WMA audio 97M
- Poor quality MP3 audio 97M
- Full transcript
- Partial transcript by Chris Hallquist
Summary and analysis
In Spring 2012 Luke Muehlhauser asked Kaj Sotala to write a summary and analysis of the debate; he made a draft of that analysis available in November 2012. The final version of the summary is included in the ebook.
Post-book material
Not included in MIRI's 2013 book on the debate:
- Robin Hanson's 2014 blog post on the subject in response to the publication of "Superintelligence", I Still Don’t Get Foom
- Katja Grace's 2014 hosted discussion on Less Wrong, Superintelligence 6: Intelligence explosion kinetics
- Eliezer's 2017 post, AlphaGo Zero and the Foom Debate and Robin Hanson's reply, What Evidence Is AlphaGo Zero Re AGI Complexity?