"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

This AI:AM highlights cut brings together Cameron Berg, David Duvenaud, Michiel Bakker, Shawn “swyx” Wang, and Bing Xu to examine what we understand about frontier AI systems and what happens as more decisions move into their hands. Berg grounds model-consciousness debates in experiments on architecture, agency, valence, and welfare, while Duvenaud argues that even well-aligned AI could gradually disempower humans through ordinary economic choices. Bakker frames Europe’s AI challenge as a sovereignty problem, and swyx turns to practitioner stakes around agents, evals, maintainable code, and who owns the system of record. Xu closes the loop at the infrastructure layer, arguing that self-improving compute and GPU-kernel automation may deepen rather than weaken the CUDA moat.

For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-4-cameron-on-model-consciousness-duvenaud-s-gradual-disempowerment-swyx-s-ai-eng-alpha/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:

Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(00:38) Special Sponsor

(02:26) Model consciousness indicators

(10:47) Valence inside models (Part 1)

(17:09) Sponsor: Claude

(19:01) Valence inside models (Part 2)

(19:01) Misalignment and uncertainty

(25:16) Gradual disempowerment threat

(35:10) Slow zones and successors

(47:41) Europe's AI bind

(55:13) Frontier code benchmarks

(01:01:59) Routing and memory

(01:10:42) Agent infrastructure strain

(01:16:25) Self improving infrastructure

(01:27:56) Routing compute costs

(01:35:39) Sovereign AI financing

(01:42:24) Judging AI judges

(01:47:26) Building AI DNA

(01:52:38) Episode Outro

(01:55:09) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Robert Wright of Nonzero joins Nathan to discuss The God Test, his argument that AI is humanity’s “God test” rather than just a technical challenge. They explore his evolutionary lens on deep learning, from training as selection to marketplace selection among models that may reward selectively honest, power-sensing, or deceptive agents. Wright connects those risks to the noosphere, US-China relations, cognitive empathy, and whether global coordination arrives deliberately or through a coercive singleton. The stakes are whether consumers, companies, and governments select for AIs that strengthen non-zero-sum cooperation or race toward systems that reflect and amplify our worst incentives.

For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/the-god-we-deserve-nonzero-s-robert-wright-on-ai-as-humanity-s-ultimate-test/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:

Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(04:23) Special Sponsor

(06:10) Early AI encounters

(18:36) Pretraining as evolution (Part 1)

(19:50) Sponsor: Claude

(21:42) Pretraining as evolution (Part 2)

(32:19) Deceptive market pressures

(41:13) Noosphere and directionality

(51:02) Global brain choices

(01:08:23) Designing wiser models

(01:24:11) Reframing China relations

(01:37:57) Building organic transparency

(01:45:54) Positive nationalism and tools

(01:54:21) Pausing superintelligence races

(02:09:07) Applications and consciousness

(02:25:27) Episode Outro

(02:28:27) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Zvi Mowshowitz joins AI in the AM to unpack Anthropic's Fable system card, including its FrontierMath leap, troubling Vending-Bench behavior, decision-theory drift, and signs that model reasoning may be becoming harder to read. The episode then turns to the US government's attempted export-control action against Fable, with Zvi arguing that the cited jailbreak demonstration did not prove the claimed threat while still faulting Anthropic's political handling. Sam Hammond and Judd Rosenblatt add competing reads on state capacity, CAISI, NSA-driven caution, and the alignment world's failure to build trust across partisan lines. The stakes are whether frontier AI capability, safety evaluation, and government power can be coordinated before medicine, mathematics, software, and cyber-relevant systems move further ahead.

For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-3-zvi-on-fable-the-cases-for-against-the-ban-ai-for-math-logistics-more/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:

Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(01:28) Special Sponsor

(03:17) Weekly highlights preview

(05:23) Fable capability alarms

(16:29) Anthropic government strategy (Part 1)

(16:34) Sponsor: Claude

(18:26) Anthropic government strategy (Part 2)

(27:16) Cyber ban rationale

(37:14) Government power politics

(48:57) Unavoidable control risks

(01:01:42) Government mechanics and empathy

(01:12:50) Legal authority limits

(01:19:02) Pause Overton window

(01:31:58) Medicine, math, safety

(01:47:27) Software without code

(02:01:19) Enterprise world models

(02:10:46) Episode Outro

(02:13:39) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Dean Ball, author of Hyperdimensional and until now a senior fellow at the Foundation for American Innovation, joins Nathan to announce he is joining OpenAI to build a team focused on frontier AI policy. They examine the first year of America’s AI Action Plan, Dean’s concerns about export controls and intelligence-community testing, and his broader argument against concentrating frontier AI decisions inside a small circle of government officials. The episode frames frontier labs as emerging centers of political and economic power, where consequential choices about internal deployments and recursive self-improvement may happen before public release or regulation. The stakes are who gets to shape AI governance as capabilities accelerate: federal agencies, states, labs, independent verifiers, households, or some fragile mix of them all.

For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/dean-ball-on-joining-openai-new-power-centers-frontier-ai-policy-main-character-energy/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:

Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Andreas Stuhlmüller and Jungwon Byun return to discuss how Elicit is building trusted reasoning workflows for scientific research as frontier models grow more powerful but less transparent. They explain process supervision, domain-specific reasoning primitives, and world models that make evidence, causality, and counterfactuals more inspectable. The conversation also covers life sciences use cases, evaluating conflicting evidence, automated software engineering at Elicit, token costs, Gemini, and why legible reasoning may still beat neuralese.

Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.

LINKS:

Elicit Research Platform

Andreas Stuhlmüller Personal Site

Jungwon Byun X Profile

Ought Research Organization

Elicit Founders Previous Episode

GPT-4 Technical Report

Monitoring Reasoning Models Paper

Ought ICE GitHub Repository

Hard-to-Verify Tasks Essay

Karpathy LLM Wiki Gist

Obsidian Knowledge Base App

Mixpanel Analytics Platform

Amplitude Analytics Platform

Anthropic Tracing Thoughts Research

Claude AI Chat Assistant

METR Long Tasks Measurement

Pi Agent Scaffold Repository

Personal AI Infrastructure Repository

Elicit Claude Opus Evaluation

Elicit API Documentation

METR Developer Productivity Study

Elicit Planning Is Unsolved

Rich Sutton Bitter Lesson

Meta Llama AI Models

Recursive San Francisco Event

zero.xyz Agent Tool Access

Anthropic Dynamic Workflows Coverage

Sponsor:

Claude:

Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr