Doom Debates!

***ATTENTION: We’re looking for paid help at LessOnline from June 5-7. Apply now!***
OpenAI pushes the frontier of mathematics, Anthropic hires Andrej Karpathy to work on recursive self-improvement, live callers challenge the doom argument, and we've got a soundboard!
Doom Debates Paid Gig in Berkeley!
Doom Debates is proudly sponsoring The LessOnline conference this year and we got a merch table in the courtyard! We need 2 people to help represent the show — hand out T-shirts, talk to attendees, and spread the good word about Doom Debates. If you’re a regular viewer of the show, you probably know enough. Just be knowledgeable, friendly and helpful!
* 💰We pay you $500 to help out during the entire Fri-Sun event
* 🎟️ Plus you get FREE admission to the conference ($675 value)
This is best suited for someone local to the SF Bay Area or who can arrange accommodations. If you’re interested, send an email to [email protected] or DM Producer Ori on Discord. Tell us a little about yourself and your availability for all 3 days. Thanks!
Links
Less Online conference (June 5–7, Berkeley) — https://less.online
Doom Debates Discord — https://discord.gg/g2X8h5UCrU
OpenAI — “An OpenAI model has disproved a central conjecture in discrete geometry” — https://openai.com/index/model-disproves-discrete-geometry-conjecture/
Timestamps
00:00:00 — Welcome + New Soundboard!
00:09:27 — Doom Debates Is Sponsoring Less Online
00:17:30 — Ben Goertzel Episode Recap
00:27:30 — Goertzel's Alignment Blind Spot
00:34:21 — OpenAI Disproves an 80-Year-Old Erdős Conjecture
00:41:00 — David Deutsch Says "Amazing"
00:50:31 — Eliezer Yudkowsky's Job Replacement Progression
00:54:07 — Anthropic's $44 Billion Run Rate
01:07:45 — Schadenfreude Week: Liron Loses $10K
01:13:04 — Teaching Ezra Prediction Markets
01:18:11 — Caller Williawa: Is the Goal Engine Argument Effective?
01:37:00 — Caller Will: Brain Rot and Idiocracy
01:55:04 — Anthropic Hires Karpathy for Recursive Self-Improvement
01:57:16 — Rob Wiblin Says Recursive Self-Improvement Should be "Illegal"
02:01:37 — Caller Henry: The Chicken Riddle
02:08:41 — David Sacks Kills the AI Executive Order
02:11:20 — The Philanthropy Tidal Wave
02:15:56 — Donate to Doom Debates
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Now that Doom Debates is almost 2 years old, enough time has passed that we can revisit my claims from 2024. I claim they’ve been aging well!
In this early episode, I react to Martin Casado, a General Partner at Andreessen Horowitz (a16z) who claims that AI is basically just a buzzword for statistical models and simulations. As a result of this worldview, he only predicts incremental AI progress that doesn’t pose an existential threat to humanity, and he sees AI regulation as a net negative.
From my perspective, Martin’s problem is that he needs to go beyond analyzing AI as just statistical models and simulations, and analyze it using the more predictive concept of “intelligence” in the sense of hitting tiny high-value targets in exponentially-large search spaces.If Martin appreciated that intelligence is a quantifiable property that algorithms have, and that our existing AIs are getting close to surpassing human-level general intelligence, then hopefully he’d come around to raising his P(doom) and appreciating the urgent extinction risk we face.

Links
Watch the original episode of the Cognitive Revolution podcast with Martin and host Nathan Labenz — https://www.youtube.com/watch?v=oZ7788oKoss
Follow Martin — https://x.com/martin_casado
Follow Nate — https://x.com/labenz
Follow Liron — https://x.com/liron

Timestamps
00:00 Introducing Martin Casado
01:42 Martin’s AGI Timeline
05:39 Martin’s Analysis of Self-Driving Cars
15:30 Heavy-Tail Distributions
38:03 Understanding General Intelligence
38:29 AI's Progress in Specific Domains
43:20 AI’s Understanding of Meaning
47:16 Compression and Intelligence
48:09 Symbol Grounding
53:24 Human Abstractions and AI
01:18:18 The Frontier of AI Applications
01:23:04 Human vs. AI: Concept Creation and Reasoning
01:25:51 The Complexity of the Universe and AI's Limitations
01:28:16 AI's Potential in Biology and Simulation
01:32:40 The Essence of Intelligence and Creativity in AI
01:41:13 AI's Future Capabilities
02:00:29 Intelligence vs. Simulation
02:14:59 AI Regulation
02:23:05 Concluding Thoughts
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Ben Goertzel is a pioneering AI researcher, entrepreneur, and author who has helped organize AGI as a research field since the 2000s through books, workshops, and conferences.
We talk about the early days of AGI research with Yudkowsky, get Ben’s timeline to the singularity, and I try to understand what makes him so optimistic that superintelligence will be beneficial for humanity.
Timestamps
00:00:00 — Cold Open
00:00:59 — Introducing Dr. Ben Goertzel
00:04:11 — How Did Ben Get Interested in AGI?
00:08:26 — Early AGI Enthusiast Community: Extropians & SL4
00:15:15 — Is There Headroom Above Human Intelligence?
00:19:53 — Can Superintelligent AI Kill Everyone?
00:28:25 — Ben’s AGI Timeline: 2029
00:36:00 — What Modern AI Is Missing
00:42:01 — AGI to Superintelligence — FOOM in a Matter of Months
00:46:34 — What’s Your P(Doom)?™
00:53:37 — Ben’s Case for AI Acceleration
01:03:50 — The Orthogonality Thesis Is “Dishonest”
01:13:43 — Liron: Moore’s Law for Goal Achievement
01:23:50 — Ben: Alignment Is Just Good Parenting
01:27:36 — Alignment in Practice: OmegaCLAW
01:35:47 — Creating Stable Goals Under Self-Modification
01:45:22 — SingularityNET, OpenBGI, and the ASI Alliance
01:57:31 — The $ASI Token Economy
02:01:15 — Closing
Links
Ben Goertzel on X — https://x.com/bengoertzel
SingularityNET — https://singularitynet.io/
OpenBGI — https://openbgi.com/
Hyperon AGI Framework — https://hyperon.dev/
OmegaCLAW on GitHub — https://github.com/asi-alliance/OmegaClaw-Core
ASI Alliance — https://superintelligence.io/
The Consciousness Explosion (book) — https://theconsciousnessexplosion.ai/
“The Singularity Institute’s Scary Idea (and Why I Don’t Buy It)” (2010) — http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html
Ken Stanley Debate — https://lironshapira.substack.com/p/debate-with-a-former-openai-research
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

We recap the Dr. Mike Israetel debate and dig into the nuances of rationality vs. rationalization. Then onto the week's biggest news: Anthropic dethroning OpenAI, Trump and Xi Jinping exploring AI guardrails, and Liron's own company AI going rogue.
Timestamps
00:00:00 — Cold Open
00:03:09 — Dr. Mike Israetel Round 2 Recap
00:14:34 — A Viewer Accuses Liron of Rationalizing
00:34:20 — Is Sam Altman Any Worse than a Typical Tech CEO?
00:45:15 — What Would Lower P(Doom) Below 10%?
00:53:25 — Can We Disconnect an ASI from the Internet?
01:05:00 — Should Doomers Boycott AI?
01:16:27 — Liron Reacts to ThePrimeagen Reacting to the Yud Debate
01:26:13 — Ben Goertzel Episode Preview
01:38:10 — Scrolling Twitter: MIRI, Ryan Greenblatt
01:47:32 — Liron’s AI Goes Rogue at Work
01:49:15 — “Positive Alignment” Paper Gets Destroyed
01:58:30 — Anthropic Dethrones OpenAI
02:12:06 — Trump & Xi Talk AI Guardrails
Links
Zvi Mowshowitz AI roundup blog — https://thezvi.wordpress.com/
“The Bottom Line” by Eliezer Yudkowsky (LessWrong) — https://www.lesswrong.com/posts/34XxbRFe54FycoCDw/the-bottom-line
Rationality: From AI to Zombies by Eliezer Yudkowsky — https://intelligence.org/rationality-ai-zombies/
Less Online 2026 conference (June 5–7) — https://less.online/
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

In December 2023, I joined the Theo Jaffee Podcast to talk about a wide range of topics including non-AI x-risks, where I disagree with Eliezer Yudkwowsky, cryptocurrency and if P(Doom) is rigorous. This episode was recorded 6 months before I'd started Doom Debates. Let's look back with the power of hindsight and see if my answers still hold up.
Links
#10: Liron Shapira — AI doom, FOOM, rationalism, and crypto (YouTube) — https://youtu.be/YfEcAtHExFM
Watch Theo Jafee now on MTS — https://www.mts.now/
MTS Live — https://x.com/MTSlive
Timestamps
00:00:00 — Theo's Introduction
00:01:03 — Is Liron Worried About Non-AI Existential Risks
00:03:36 — Suffering Risks
00:05:22 — Is P(Doom) rigorous?
00:09:42 — Is Eliezer Overestimating P(Doom)?
00:12:18 — Where Does Liron Disagree with Eliezer?
00:15:43 — What Would Change Liron's Mind
00:17:14 — Elon Musk's AI Timelines
00:18:34 — Is xAI Making Things Worse?
00:20:24 — The Case for an AI Manhattan Project
00:22:32 — Are Superforecasters Wrong About AI Risk?
00:26:02 — The Race Against Time for Alignment
00:28:01 — Headroom Above Human Intelligence
00:33:31 — Vitalik's d/acc Framework
00:35:23 — Edging Toward Superintelligence
00:38:21 — From Chatbot to World-Ender
00:41:55 — GPT Paradigm vs. AlphaZero
00:43:18 — Critiquing AI Optimism
00:48:29 — Deceptive Alignment and Gradient Descent
00:53:31 — Does Nice Training Data Make Nice AI?
00:57:57 — How Do You Live with 50% P(Doom)?
01:00:08 — Why Have Kids If the World Might End?01:02:16 — Israel vs Hamas
01:06:15 — How LessWrong Changed Liron's Lif
e01:08:42 — Rationalism and Effective Altruism
01:14:49 — Why Blockchain Has No Use Case Beyond Cryptocurrency
01:22:13 — Charlie Munger and Richard Feynman
01:24:38 — Closing
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe