The Growth Podcast

Today’s Episode
Everyone’s demoing AI features. Few are shipping them to production reliably.
The gap? Evals.
Not the theoretical kind. The real-world kind that catches bugs before users do.
Hamel Husain and Shreya Shankar train people at OpenAI, Anthropic, Google, and Meta on how to build AI products that actually work. Their Maven course is the top-grossing course on the platform.
Today, they’re walking you through their complete eval process.
----
Brought to you by:
* The AI Evals Course for PMs & Engineers: You get $800 with this link
* Vanta: Automate compliance, Get $1,000 with my link
* Jira Product Discovery: Plan with purpose, ship with confidence
* Land PM job: 12-week experience to master getting a PM job
* Pendo: the #1 Software Experience Management Platform
----
If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, and Mobbin - for free, grab Aakash’s bundle.
Are you searching for a PM job? Join me + 29 others for an intensive 12-week experience to master getting a PM job. Only 23 seats left.
----
Key Takeaways:
1. AI evals are the #1 most important new skill for PMs in 2025 - Even Claude Code teams do evals upstream. For custom applications, systematic evaluation is non-negotiable. Dog fooding alone isn't enough at scale.
2. Error analysis is the secret weapon most teams skip - Looking at 100 traces teaches you more than any generic metric. Hamel: "If you try to use helpfulness scores, the LLM won't catch the real product issues."
3. Use observability tools but don't depend on them completely - Brain Trust, LangSmith, Arise all work. But Shreya and Hamel teach students to vibe code their own trace viewers. Sometimes CSV files are enough to start.
4. Never use agreement as your eval metric - It's a trap. A judge that always says "pass" can have 90% accuracy if failures are rare. Use TPR (true positive rate) and TNR (true negative rate) instead.
5. Open coding then axial coding reveals patterns - Write notes on 100 traces without root cause analysis. Then categorize into 5-6 actionable themes. Use LLMs to help but refine manually.
6. Product managers must do the error analysis themselves - Don't outsource to developers. Engineers lack domain context. Hamel: "It's almost a tragedy to separate the prompt from the product manager because it's English."
7. Real traces reveal what demos hide - Chat GPT said the assistant was correct but missed: wrong bathroom configuration, markdown in SMS, double-booked tours, ignored handoff requests.
8. Binary scores beat 1-5 scales for LLM judges - Easier to validate alignment. Business decisions are binary anyway. LLMs struggle with nuanced numerical scoring.
9. Code-based evals for formatting, LLM judges for subjective calls - Markdown in text messages? Write a simple assertion. Human handoff quality? Need an LLM judge with proper rubric.
10. Start with traces even before launch - Dog food your own app. Recruit friends as beta testers. Generate synthetic inputs only as last resort. Error analysis works best with real user behavior.-
---
Where To Find Them
* LinkedIn:
* Hamel: Hamel’s LinkedIn
* Shreya: Shreya’s LinkedIn
* AI Evals Course: World’s best AI Evals Course (You get $800 off with this link)
Related Content
Newsletters:
* AI Evals
* AI PM Observability
* AI Testing
* LLM Judge
* AI Prototype to Production
* AI Product Strategy
* How to Build AI Products
* Prompt Engineering
Podcasts:
* AI Evals: Everything You Need to Know to Start
* Everything you need to know about AI (for PMs and builders)
* Carl Vellotti on Claude Code
* Marily Nika on Google AI PM Tool Stack
* Pawel Huryn on n8n for PMs
PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!


This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe

Today's Episode
Carl Vellotti is back.
His first episode hit 30k views. My most popular episode in months.
Today, he’s giving you the advanced playbook for Claude Code.
Claude Code hit $1 billion ARR in December. Fastest product ever to reach that milestone. Under 6 months.
Carl has been building with Claude Code every single day. He’s made all the mistakes. He’s learned all the expert workflows and tips.
In today’s episode, he shares everything.
----
If you want access to my AI tool stack - Dovetail, Arize, Linear, Descript, Reforge Build, DeepSky, Relay.app, Magic Patterns, and Mobbin - for free, grab Aakash’s bundle.
Are you searching for a PM job? Join me + 29 others for an intensive 12-week experience to master getting a PM job.
----
Brought to you by:
Amplitude: The market-leader in product analytics
Pendo: the #1 Software Experience Management Platform
Jira Product Discovery: Move discovery and roadmapping out of spreadsheets
Miro: The innovation workspace
----
Key Takeaways
1. Claude Code hit $1B ARR in 6 months - fastest product ever. Anthropic optimized for power users doing deep work. Fewer users, insanely deep usage. Carl: "I spend all day in Claude Code. It gets to the point where you never really have to leave."
2. MCPs connect Claude to everything in 2 minutes - Model Context Protocol invented by Anthropic. One command to install Linear, Google Workspace, Slack. Essential stack: documents first, task management second, communication third, data sources fourth.
3. End-to-end PM workflow takes one morning instead of one week - Survey creation → analysis → PRD → presentation → 19 tickets. All through MCPs. Carl didn't use templates. Opus 4.5 handled everything.
4. Skills are pre-built templates but don't always auto-trigger yet - Solution: specify explicitly. "Create presentation using your presentation skill." Build Skills for anything you do more than twice. That's the compound effect.
5. Real Google Slides, not images - Carl's presentation skill created 19 fully editable slides from PRD. Takes a few minutes because Claude creates slides in parallel. "This is saving you hours. We used to spend entire afternoons turning PRDs into decks."
6. GitHub turns Claude into remote worker - Create issues from your phone. @mention Claude. It works while you're away. Carl's journal is a private GitHub repo. Voice transcriptions → issues → Claude updates markdown.
7. Workflows beat agents for production - Level 1 workflow: 5,000 tokens, 40 seconds. Level 3 agent: 90,000 tokens, 90 seconds. Carl's rule: "If something can be expressed as code, code it. Leave agents for cognitive work."
8. Production best practices separate hobbyists from pros - Set error workflows. Add retry logic (3 attempts, 1-second delays). Pin data during development. Use reasoning effort appropriately. Write detailed tool descriptions.
9. Real challenge is building reusable workflows - Learning basics takes a day. First workflow takes hours. But having 50 workflows built over months means everything is automated. Carl: "Because I have workflows for all these things already built, I can do it all in one shot."
10. Tools improve faster than you can learn them - Carl: "By the time you see this, there might be Claude 5." Your workflows from today will 10x when better models drop. PMs who started 6 months ago aren't just 6 months ahead, they're 100x ahead because systems compound with every improvement.
----
Where to Find Carl Vellotti
* Linkedin
* X (Twitter)
* Instagram
* Newsletter
Related Content
Newsletters:
* How to Use Claude Code Like a Pro
* Steal 6 of My Claude Skills
* Claude Skills Tutorial
* AI Agents: The Ultimate Guide for PMs
* Practical AI Agents
Podcasts:
* The Ultimate Guide to n8n for PMs, with Pawel Huryn
* This is what a Google AI PM’s Tool Stack Looks Like, with Marily Nika
* How to Use Claude Code Like a Pro (Carl’s First Episode)
----
PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!
If you want to advertise, email productgrowthppp at gmail.


This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe

Marily Nika and I filmed my 7th most popular episode a year ago.
She broke down everything on how to become an AIPM because she’s done it. She is an AIPM at Google with 11 years of experience.
Today, she’s back with a completely new episode. She’s dropping every AI tool she uses daily as an AIPM.
Not theory. Not hype. The actual 6 tools she uses multiple times every single day.
Today she’s dropping all the expert tips and tricks. She’s made the mistakes so you don’t have to.----Today’s guide covers:
* The AI PM Tool Stack
* Google AI Studio for Prototyping
* Opal for Mini Apps
* Notebook LM for Domain Expertise
* Perplexity for Reddit Research
* ChatGPT for Your Voice
* Fireflies and Getting Tools at Work
* The 18-Month AIPM Roadmap
* The AIPM Interview Red Flags
----Check out the conversation on Apple, Spotify and YouTube.
Brought to you by:
* Jira Product Discovery: Move discovery and roadmapping out of spreadsheets
* Vanta: Automate compliance across 35+ frameworks
* Maven: Improve your PM skills with awesome courses. Discount with my link
* Mobbin: Discover real-world design inspiration
* Product Faculty: Get $500 off the AI PM Certification with code AAKASH25----
Key Takeaways
1. Marily's stack is just 6 tools - Google AI Studio for prototyping, Opal for mini apps, Notebook LM for learning, Perplexity for user research, ChatGPT for PRDs, and Fireflies for meetings. Each maps to one PM workflow. No tool hopping.
2. Prototype before writing PRDs. Marily builds a working app in AI Studio first, then brings engineers in to debate the actual thing. Saves weeks of back-and-forth on documents. PRDs are now for complex cross-functional work only.
3. Notebook LM learned a 4-hour video in 15 minutes. Marily had an interview the next day. She uploaded the job description and a 4-hour investor relations video. It gave her 15 key points. She memorized them and crushed the interview.
4. Perplexity's Reddit filter is your secret weapon. Turn off web search, turn on discussions and opinions. Ask "would young professionals be interested in a fitness ring?" Get 20+ Reddit sources instantly. Know what users actually want.
5. Marily's PRD generator has 10,000+ users. She trained a ChatGPT custom GPT with her voice and old PRDs. It asks probing questions before generating. Gets PRDs done in days instead of weeks. She's not embarrassed to use AI at work.
6. Tool selection has 4 rules. Does it save 10x time (not 2x)? Does it work across contexts? Does it work with your company's limitations? Does it compound over time? If not all 4, delete it.
7. "Be like a crab" for AIPM roles. Move adjacent to your experience. Hearing aids → AirPods PM. ESPN journalism → Meta sports AI PM. Don't ignore your past experience. It's your competitive advantage.
8. Red flag in AIPM interviews - missing PM craft. Ex-ML scientists dive into algorithms without asking why, who, and how to measure success. AI changes HOW you fulfill use cases, not the use cases themselves.
9. Who replaces PMs? Not AI. PMs who use AI when you're not. Don't be embarrassed about using AI at work. It's not cheating. The gap is widening between PMs who use these tools and those who don't.
10. Every PM will become an AIPM by 2026. Show Marily any product and she'll find the AI use case. Even retail stores use cameras and AI to monitor which areas get traffic. The AIPM title will just become PM.
----
Where to Find Marily
* LinkedIn
* Twitter
* Website
* YouTube
* Newsletter
* Maven - Her course is part of my special curation with Maven
----
Related Content
Newsletters
* AI Foundations for PMs
* How to Become an AIPM
* AI Prototyping for PMs
* AI PRDs
* The Complete AI PM Course
* AI Product Strategy
* ChatGPT for PMs
Podcasts
* Pawel Huryn on n8n
* Alex Danilowicz on AI Prototyping
* Hamza Farooq on Landing AIPM Jobs
* Todd Olson on Upskilling to AIPM
* Rachel Wolan on AI Product Leadership----
If you want access to AI tools for free, get Aakash’s bundle and access to 9 AI products for an entire year.
Are you searching for a PM job? Join me + 29 others for an intensive 12-week experience to master getting a PM job.
----
PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!If you want to advertise, email productgrowthppp at gmail.


This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe

This is a free preview of a paid episode. To hear more, visit www.news.aakashg.com

Pawel Huryn is the guest behind my most popular episode ever (52K+ views).
Today, he’s back to give you a masterclass in one of the most exciting AI tools out there: n8n.
n8n is the most powerful workflow automation tool that combines two things: traditional workflow automation and building AI agents.
And Pawel has been knee-deep in n8n more than almost anyone else in the world.
He’s tried everything. He’s made all the mistakes. He’s learned all the expert workflows and tips and tricks.
In today’s episode, he walks through building real n8n workflows from scratch.
-----
Check out the conversation on Apple, Spotify and YouTube.
Brought to you by:
* Amplitude: The market-leader in product analytics
* Vanta: Automate compliance across 35+ frameworks
* Testkube: Leading test orchestration platform
* Kameleoon: AI experimentation platform
* Pendo: The #1 software experience management platform
-----
Key Takeaways
1. N8N combines traditional workflow automation AND AI agent building in one platform - making it more powerful than Zapier or Make for complex automation needs.
2. Real use cases span from simple business workflows to chatbots, automatic competitor monitoring, multi-agent research systems, and inbox workers that take actions based on emails. Sky is the limit.
3. Pavel's competitor monitoring workflow costs $1-2/week using the FREE version of N8N. Just needs Perplexity API ($1-2 for hundreds of calls) and OpenAI credits. Enterprise tools charge $500+/month.
4. Pin your data during development. N8N caches API responses so you don't burn credits while testing workflows. Click the pin icon and N8N uses cached data instead of making new API calls.
5. N8N automatically loops through items - no need to write for-loops or while-loops. When you connect a node with 6 items, N8N repeats the action 6 times automatically.
6. Compress context before sending to LLMs. Pavel cuts 70% of tokens by extracting only summary content and citation URLs from Perplexity results, ignoring irrelevant snippets and metadata.
7. Use ChatGPT to write N8N code snippets. Pavel never writes code blocks himself - just takes a screenshot of the data and asks GPT "how do I compress this information?"
8. Traditional workflows are more efficient (saves tokens, very reliable) for predictable tasks. AI agents are more flexible but use more tokens and can make mistakes. Use workflows when you know the steps.
9. Set GPT reasoning effort to "low" for simple tasks. When you just need formatting or summarization (not complex thinking), low reasoning effort saves tokens significantly.
10. Best practices: Set dedicated error probes to catch errors before they break workflows. Use max iterations to prevent infinite loops. Set retry on fail to 3x attempts. Pin data during development.
-----
Where to Find Pawel
* LinkedIn
* Newsletter
* YouTube
-----
Related Content
Newsletters:
* Ultimate Guide to AI Prototyping Tools
* AI Agents: The Ultimate Guide for PMs
* Practical AI Agents
Podcasts:
* AI Agents Demo, with CEO of Relay.app
* How to Build AI Agents (and Get Paid $750K+)
* 5 AI Agents Every PM Should Build, with CEO of Lindy
If you want access to a n8n competitor for free, get Aakash’s bundle and access to Relay.app.
-----
If you want to advertise, email productgrowthppp at gmail.

Today’s Episode
Laura Burkhauser started as an IC PM at Descript, the $550M AI video editing platform.
Three years later, she’s CEO.
She shipped AI features that worked.
In today’s episode, you’ll hear the exact features Laura built, the eval framework she used, and her complete IC → CEO path.
This might be the most actionable career episode you listen to all year.
----
Check out the conversation on Apple, Spotify and YouTube.
Brought to you by:
* Maven: Improve your PM skills with awesome courses
* Pendo: #1 Software Experience Management Platform
* Vanta: Automate compliance across 35+ frameworks like SOC 2 and ISO 27001
* NayaOne: Airgapped cloud-agnostic sandbox
* Kameleoon: Leading AI experimentation platform
----
Key Takeaways1. Map your user journey BEFORE picking AI features. DeScript identified pain points (retakes, eye contact, rambling), then asked "what just became possible with LLMs?" Build that intersection.2. Build prepackaged buttons, not blank chat boxes. Each DeScript AI tool is a carefully crafted prompt behind a single button that delivers reliable results every time.3. Use human evals on production data before shipping. Test on real customer data, ask "would I use this as a customer?" If yes, ship. If no, don't.4. The ultimate metric is export rate. If users apply your AI feature then remove it before exporting, it didn't meet their quality bar.5. Switch from buttons to chat when you hit 30+ parameters. When users wanted topic selection, speaker choice, and platform optimization, chat became better than buttons.6. Match your eval data to actual use case. DeScript failed with Studio Sound because they tested on terrible audio (vacuuming, jackhammers) when real users had laptop microphones. Different models handle different quality levels.7. Test agents with real customer language early. Don't use toy data or employee terminology. Mix sophistication levels—some advanced at video and AI, some complete beginners—to understand how real people prompt.8. Launch AI agents to new users first. Video editing is hard and many people quit. DeScript tested Underlord on activation and it won, so new users got it first before existing users.9. Choose breadth over depth for product-wide agents. DeScript chose breadth—Underlord works across all features because "we're not a point solution." Requires more context, tool coverage, and evals but serves the product vision.10. Earn founder trust by getting command, not by being strategic. Use the product extensively. Talk to customers constantly. When you speak, people think "Smart" and invite you to more rooms. Ship features before focusing on strategy.
----
Where to Find Laura Burkhauser
* LinkedIn
* Company
----
Related Content
Podcasts:
* AI Product Leadership Masterclass
* Conversation with the CEO and Founder of Bolt
* This $20M AI Founder Is Challenging Elon and Sam Altman | Roy Lee, Cluely
Newsletters:
* Ultimate Guide to AI Prototyping Tools
* AI Agents: The Ultimate Guide for PMs
* How to Land a $300K+ AI Product Manager Job
PS. Please subscribe on YouTube and follow on Apple & Spotify. It helps!


This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.news.aakashg.com/subscribe