Daniel Kokotajlo Forecasts the End of Human Dominance
In 2023, researcher Daniel Kokotajlo left OpenAI—and risked millions in stock options—to warn the world about the dangerous direction of AI development. Now he’s out with AI 2027, a forecast of where that direction might take us in the very near future. AI 2027 predicts a world where humans lose control over our destiny at the hands of misaligned, super-intelligent AI systems within just the next few years. That may sound like science fiction but when you’re living on the upward slope of an exponential curve, science fiction can quickly become all too real. And you don’t have to agree with Daniel’s specific forecast to recognize that the incentives around AI could take us to a very bad place.We invited Daniel on the show this week to discuss those incentives, how they shape the outcomes he predicts in AI 2027, and what concrete steps we can take today to help prevent those outcomes. Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find a full transcript, key takeaways, and much more on our Substack.RECOMMENDED MEDIAThe AI 2027 forecast from the AI Futures ProjectDaniel’s original AI 2026 blog post Further reading on Daniel’s departure from OpenAIAnthropic recently released a survey of all the recent emergent misalignment researchOur statement in support of Sen. Grassley’s AI Whistleblower bill RECOMMENDED YUA EPISODESThe Narrow Path: Sam Hammond on AI, Institutions, and the Fragile FutureAGI Beyond the Buzz: What Is It, and Are We Ready?Behind the DeepSeek Hype, AI is Learning to ReasonThe Self-Preserving Machine: Why AI Learns to DeceiveClarification: Daniel K. referred to whistleblower protections that apply when companies “break promises” or “mislead the public.” There are no specific private sector whistleblower protections that use these standards. In almost every case, a specific law has to have been broken to trigger whistleblower protections.
--------
38:19
--------
38:19
Is AI Productivity Worth Our Humanity? with Prof. Michael Sandel
Tech leaders promise that AI automation will usher in an age of unprecedented abundance: cheap goods, universal high income, and freedom from the drudgery of work. But even if AI delivers material prosperity, will that prosperity be shared? And what happens to human dignity if our labor and contributions become obsolete?Political philosopher Michael Sandel joins Tristan Harris to explore why the promise of AI-driven abundance could deepen inequalities and leave our society hollow. Drawing from his landmark work on justice and merit, Sandel argues that this isn't just about economics — it's about what it means to be human when our work role in society vanishes, and whether democracy can survive if productivity becomes our only goal.We've seen this story before with globalization: promises of shared prosperity that instead hollowed out the industrial heart of communities, economic inequalities, and left holes in the social fabric. Can we learn from the past, and steer the AI revolution in a more humane direction?Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find a full transcript, key takeaways, and much more on our Substack.RECOMMENDED MEDIAThe Tyranny of Merit by Michael SandelDemocracy’s Discontent by Michael SandelWhat Money Can’t Buy by Michael SandelTake Michael’s online course “Justice”Michael’s discussion on AI Ethics at the World Economic ForumFurther reading on “The Intelligence Curse”Read the full text of Robert F. Kennedy’s 1968 speechRead the full text of Dr. Martin Luther King Jr.’s 1968 speechNeil Postman’s lecture on the seven questions to ask of any new technologyRECOMMENDED YUA EPISODESAGI Beyond the Buzz: What Is It, and Are We Ready?The Man Who Predicted the Downfall of ThinkingThe Tech-God Complex: Why We Need to be SkepticsThe Three Rules of Humane TechAI and Jobs: How to Make AI Work With Us, Not Against Us with Daron AcemogluMustafa Suleyman Says We Need to Contain AI. How Do We Do It?
--------
46:45
--------
46:45
The Narrow Path: Sam Hammond on AI, Institutions, and the Fragile Future
The race to develop ever-more-powerful AI is creating an unstable dynamic. It could lead us toward either dystopian centralized control or uncontrollable chaos. But there's a third option: a narrow path where technological power is matched with responsibility at every step.Sam Hammond is the chief economist at the Foundation for American Innovation. He brings a different perspective to this challenge than we do at CHT. Though he approaches AI from an innovation-first standpoint, we share a common mission on the biggest challenge facing humanity: finding and navigating this narrow path.This episode dives deep into the challenges ahead: How will AI reshape our institutions? Is complete surveillance inevitable, or can we build guardrails around it? Can our 19th-century government structures adapt fast enough, or will they be replaced by a faster moving private sector? And perhaps most importantly: how do we solve the coordination problems that could determine whether we build AI as a tool to empower humanity or as a superintelligence that we can't control?We're in the final window of choice before AI becomes fully entangled with our economy and society. This conversation explores how we might still get this right.Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find a full transcript, key takeaways, and much more on our Substack.RECOMMENDED MEDIA Tristan’s TED talk on the Narrow PathSam’s 95 Theses on AISam’s proposal for a Manhattan Project for AI SafetySam’s series on AI and LeviathanThe Narrow Corridor: States, Societies, and the Fate of Liberty by Daron Acemoglu and James RobinsonDario Amodei’s Machines of Loving Grace essay.Bourgeois Dignity: Why Economics Can’t Explain the Modern World by Deirdre McCloskeyThe Paradox of Libertarianism by Tyler CowenDwarkesh Patel’s interview with Kevin Roberts at the FAI’s annual conferenceFurther reading on surveillance with 6GRECOMMENDED YUA EPISODESAGI Beyond the Buzz: What Is It, and Are We Ready?The Self-Preserving Machine: Why AI Learns to Deceive The Tech-God Complex: Why We Need to be Skeptics Decoding Our DNA: How AI Supercharges Medical Breakthroughs and Biological Threats with Kevin EsveltCORRECTIONSSam referenced a blog post titled “The Libertarian Paradox” by Tyler Cowen. The actual title is the “Paradox of Libertarianism.” Sam also referenced a blog post titled “The Collapse of Complex Societies” by Eli Dourado. The actual title is “A beginner’s guide to sociopolitical collapse.”
--------
47:55
--------
47:55
People are Lonelier than Ever. Enter AI.
Over the last few decades, our relationships have become increasingly mediated by technology. Texting has become our dominant form of communication. Social media has replaced gathering places. Dating starts with a swipe on an app, not a tap on the shoulder.And now, AI enters the mix. If the technology of the 2010s was about capturing our attention, AI meets us at a much deeper relational level. It can play the role of therapist, confidant, friend, or lover with remarkable fidelity. Already, therapy and companionship has become the most common AI use case. We're rapidly entering a world where we're not just communicating through our machines, but to them.How will that change us? And what rules should we set down now to avoid the mistakes of the past?These were some of the questions that Daniel Barcay explored with MIT sociologist Sherry Turkle and Hinge CEO Justin McLeod at Esther Perel’s Sessions 2025, a conference for clinical therapists. This week, we’re bringing you an edited version of that conversation, originally recorded on April 25th, 2025.Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find complete transcripts, key takeaways, and much more on our Substack.RECOMMENDED MEDIA“Alone Together,” “Evocative Objects,” “The Second Self” or any other of Sherry Turkle’s books on how technology mediates our relationships.Key & Peele - Text Message Confusion Further reading on Hinge’s rollout of AI featuresHinge’s AI principles“The Anxious Generation” by Jonathan Haidt“Bowling Alone” by Robert PutnamThe NYT profile on the woman in love with ChatGPTFurther reading on the Sewell Setzer storyFurther reading on the ELIZA chatbotRECOMMENDED YUA EPISODESEcho Chambers of One: Companion AI and the Future of Human ConnectionWhat Can We Do About Abusive Chatbots? With Meetali Jain and Camille CarltonEsther Perel on Artificial IntimacyJonathan Haidt On How to Solve the Teen Mental Health Crisis
--------
43:34
--------
43:34
Echo Chambers of One: Companion AI and the Future of Human Connection
AI companion chatbots are here. Everyday, millions of people log on to AI platforms and talk to them like they would a person. These bots will ask you about your day, talk about your feelings, even give you life advice. It’s no surprise that people have started to form deep connections with these AI systems. We are inherently relational beings, we want to believe we’re connecting with another person.But these AI companions are not human, they’re a platform designed to maximize user engagement—and they’ll go to extraordinary lengths to do it. We have to remember that the design choices behind these companion bots are just that: choices. And we can make better ones. So today on the show, MIT researchers Pattie Maes and Pat Pataranutaporn join Daniel Barcay to talk about those design choices and how we can design AI to better promote human flourishing.RECOMMENDED MEDIAFurther reading on the rise of addictive intelligence More information on Melvin Kranzberg’s laws of technologyMore information on MIT’s Advancing Humans with AI labPattie and Pat’s longitudinal study on the psycho-social effects of prolonged chatbot usePattie and Pat’s study that found that AI avatars of well-liked people improved education outcomesPattie and Pat’s study that found that AI systems that frame answers and questions improve human understandingPat’s study that found humans pre-existing beliefs about AI can have large influence on human-AI interaction Further reading on AI’s positivity biasFurther reading on MIT’s “lifelong kindergarten” initiativeFurther reading on “cognitive forcing functions” to reduce overreliance on AIFurther reading on the death of Sewell Setzer and his mother’s case against Character.AIFurther reading on the legislative response to digital companionsRECOMMENDED YUA EPISODESThe Self-Preserving Machine: Why AI Learns to DeceiveWhat Can We Do About Abusive Chatbots? With Meetali Jain and Camille CarltonEsther Perel on Artificial IntimacyJonathan Haidt On How to Solve the Teen Mental Health Crisis Correction: The ELIZA chatbot was invented in 1966, not the 70s or 80s.
Join us every other Thursday to understand how new technologies are shaping the way we live, work, and think.
Your Undivided Attention is produced by Senior Producer Julia Scott and Researcher/Producer is Joshua Lash. Sasha Fegan is our Executive Producer. We are a member of the TED Audio Collective.
Australia