Sweat the Tokens

A product management heuristic for the AI era: you must understand AI behavior at the token/transcript level with the same rigor that traditional PMs applied to pixel-level UI design. “You have to sweat the tokens as much as you sweat the pixels.”

Where it comes from

Dianne Penn describing how PMs at Anthropic investigate model quality: > “In the past, we might do a user interview and have the user walk you through their user flow, the pixels. Here, you have to sweat the tokens as much as you sweat the pixels.” (~42:52)

The insight

Traditional PM craft (user interviews, flow analysis) doesn't go away — it evolves. The new “pixels” are the tokens, the conversation transcripts, the failure trajectories. You can't manage what you don't deeply understand.

Where it is argued

2 notes

01AI Going VerticalReading full conversation transcripts to understand AI failure modes, not just surface complaints 02Systems Thinkers Over SpecialistsElizabeth Stone, CPTO of Netflix, argues that the AI era demands systems thinkers who abstract across domains over narrow specialists — while insisting that craft excellence remains scarce and essential. She explores how AI blurs functional roles without making them obsolete, why Netflix's “excellence as an operating system” (high talent density, autonomy, comfort with discomfort) maps directly onto how top AI labs operate, and how AI is reshaping entertainment from production through personalization.