5 posts with this tag
I built an eval framework to grade my AI podcast pipeline across 40 real episodes in 7 languages. The review step was making scripts worse (3.34 → 2.91). Citation URLs all looked fake (score: 1.28). Chinese was the weakest locale (2.88). Six fixes, one day, and a new model later — here are the before/after numbers and every change I made.
I spent days crafting a system prompt for my AI podcast hosts. They read every word and ignored almost all of it. The fix wasn't better words — it was understanding that LLMs process rules differently depending on where they live in your pipeline.
I almost built a clever, AI-specific offline mode for DIALØGUE's iOS app. Then I asked a more boring question — what does iOS already give me? — and the real work turned out to be resisting the urge to be clever. Here's the standard, unglamorous machinery behind "download an episode and listen on a plane," and why boring was the senior choice.
I shipped DIALØGUE's iOS app as a port of the web product, then rebuilt it natively — three tabs, lock-screen audio, a synced transcript, resilient offline, and Siri — because a web app shrunk to a phone is still a web app.
I pushed what should have been an upgrade to my podcast platform's voice system. Six days and several commits later, I deleted 2,724 lines of code and rolled back to what worked. Here is what happened and what it taught me about testing production AI changes.