I discovered something embarrassing last week.
My AI podcast pipeline has an elaborate system prompt — a "dialogue guide" — that I spent days crafting. It tells the hosts (two AI-generated football commentators) exactly how to talk to each other. Don't repeat the same opening. Vary the structure across segments. Use catchphrases sparingly. Never fabricate statistics.
The LLM read every word. Then ignored almost all of it.
The Vietnamese host said "Trời ơi" (Oh my god) 13 times in one episode. The Japanese host opened every single segment with "いやー、健一さん！" (Well, Kenichi-san!). The English host confidently cited xG 0.35 and 65% possession — numbers that exist nowhere in my database.
The prompt was politely acknowledged. And then disregarded.
This is the single most useful thing I've learned about prompt engineering: LLMs have two reading modes, and they treat them completely differently.
The Two Reading Modes Problem
Here's what my original dialogue guide looked like. I've translated the Vietnamese version to English since the structure is identical:
STYLE GUIDELINES:
HOST INTERACTION PATTERN: Emotional Host + Analytical Host.
STRICT RULES FOR EVERY SEGMENT:
0. NEVER REPEAT THE SAME OPENING. Vary which host initiates,
which match moment they react to first.
1. EMOTIONAL START (mandatory): Every segment begins with
the emotional host's pure reaction.
2. DATA-GROUNDING RULE (mandatory): Every claim must cite
specific numbers. Not "they dominated" but "61% possession,
3.17 xG to 0.65."
3. CATCHPHRASES:
Emotional host: "You see that?!" "Wait wait wait..."
Analytical host: "There's a deeper reason..." "Let me tell you."
Looks thorough, right? I genuinely thought I'd done the work.
But when I read the actual generated scripts, here's what happened:
| Rule | What the guide said | What the LLM did |
|---|---|---|
| Rule 0 (never repeat) | Vary the opening | Every segment opened with the emotional host's catchphrase |
| Rule 2 (data) | Cite specific numbers | Cited fabricated numbers with total authority |
| Rule 3 (catchphrases) | List of signature phrases | Used each phrase 5–13 times per episode |
The rules were there. The LLM read them. But they had the force of a suggestion, not a requirement.
This is when I realized: the LLM was processing my rules as "style context" — not as "enforceable constraints."
The Pattern: Context vs. Command
Think about how you read different types of text. If I hand you a restaurant menu, you read it as information to act on. If I hand you a Wikipedia article about the history of pasta, you read it as background knowledge.
LLMs do something similar with prompts. When rules appear in a section labeled "STYLE GUIDELINES" or "HOST INTERACTION PATTERN" or "RULES FOR SEGMENTS," the model treats them as ambient information — context that colors the response but doesn't constrain it.
When rules appear in a section labeled "CRITICAL IMPROVEMENTS REQUIRED" or "SPECIFIC PHRASE LIMITS," the model treats them as enforceable commands — things it must verify and correct.
The insight isn't about the words "critical" or "required." It's about where in the pipeline the rule lives.
The Fix: Make the Review Step Enforce What the Generation Step Ignores
My pipeline has two LLM calls:
- Generation step — writes the script segment by segment, using the dialogue guide as context
- Review step — reads the complete assembled script and improves it
The generation step was where all my rules lived. The review step had generic anti-repetition rules ("don't say 'That's fascinating' twice") that were entirely English-centric and knew nothing about my host catchphrases or structural requirements.
The fix was structural, not lexical. I moved enforcement to the review step, where the full script is visible at once.
Here's what changed:
Before: Rules in the generation prompt (ignored)
STYLE GUIDELINES:
0. NEVER REPEAT THE SAME OPENING...
1. EMOTIONAL START...
2. DATA-GROUNDING...
3. CATCHPHRASES: "You see that?!" ...
After: Rules extracted and injected into the review prompt (enforced)
CRITICAL IMPROVEMENTS REQUIRED ACROSS THE ENTIRE SCRIPT:
2a. ENFORCE STYLE GUIDELINES' STRUCTURAL RULES:
The STYLE GUIDELINES above contain specific requirements
about segment structure. These are REQUIREMENTS, not suggestions.
- If the guidelines say "X of Y segments must vary," CHANGE
segments that violate this.
- Check how EACH segment opens. If 4+ out of 5 open identically,
VARY at least 2.
3b. LOCALE-SPECIFIC PHRASE LIMITS (extracted from host profiles —
MAXIMUM 2 TIMES each across entire script):
- "Trời ơi" / "Trời má" — max 4x combined
- "Có một lý do sâu hơn" — max 2x
- "Mày thấy chưa?!" — max 2x
...
Two things happened here:
-
The structural rules got promoted from context to command — rephrased as "ENFORCE this" and "CHANGE segments that violate this" rather than "here's the preferred pattern"
-
The catchphrases got extracted from the host profiles and rendered as explicit limits, not as a list of suggestions. "Catchphrases: here are some" became "These phrases: MAXIMUM 2 TIMES each."
Both changes live in the review prompt, not the generation prompt. That's the key architectural decision. The generation step gets the style guide as context — it needs the personality flavor. The review step gets the rules as enforcement — it needs the compliance mandate.
The Extraction Detail (Why This Works Across 7 Languages)
I didn't want to maintain per-language phrase lists by hand. Every time I tweaked a host profile, I'd have to update the review prompt too. That's the kind of drift that kills a solo developer's product.
Instead, I wrote a small extraction function that parses the host profiles at review time. Each locale's profile has a catchphrase section with a standard label:
- Vietnamese:
Câu cửa miệng: - Japanese:
口癖: - Spanish:
Latiguillos: - French:
Phrases fétiches: - Korean:
입버릇: - Chinese:
口头禅:
The function finds that label, extracts any quoted strings that follow, and formats them as enforceable limits:
@staticmethod
def _extract_catchphrase_limits(host_profiles: str) -> str:
patterns = [
(r'Câu cửa miệng:?', r'"([^"]+)"'), # Vietnamese
(r'口癖:?', r'「([^」]+)」'), # Japanese
(r'Latiguillos:?', r'"([^"]+)"'), # Spanish
# ... etc for all 7 languages
]
limits = []
for marker_pattern, quote_pattern in patterns:
match = re.search(marker_pattern, host_profiles)
if match:
phrases = re.findall(quote_pattern, host_profiles[match.end():])
for phrase in phrases:
limits.append(f' - "{phrase}" — max 2x')
return "\n".join(limits)
Now when I edit a host profile and change their catchphrases, the review prompt automatically updates. No drift. No forgotten language.
What This Actually Fixed (With Numbers)
Here's what changed between the episode generated before these fixes and one generated after:
| Metric | Before | After |
|---|---|---|
| "Trời ơi" count (quota: max 4) | 13 | Not yet tested in vi, but in ja: all 4 catchphrases within quotas |
| "Có một lý do sâu hơn" (quota: max 2) | 5 | Within quota in equivalent ja test |
| Fabricated statistics (xG, possession %, formation width) | 4 invented numbers | Zero invented numbers |
| Segment openings with identical pattern | 6/6 identical | Still 5/5 in ja — see below |
| Vietnam/Japan connections preserved | 1/4 survived (review stripped 3) | 11/11 survived |
| Host voice cross-contamination | Both used each other's phrases | Clear separation |
The catchphrase quotas, anti-fabrication rules, and localization preservation all worked. The host profiles now dictate the hosts' personalities clearly enough that the LLM no longer confuses who says what.
What's Still Broken
One rule still fails: structural variety. The dialogue guide says "at least 2 of 5 segments must use different structural entries." After the fixes, the Japanese episode still opened every segment identically: "いやー、健一さん！" — five times in one episode.
This isn't a prompt enforcement problem. It's an architecture problem.
The generation step produces each segment independently. Segment 3 has no awareness that segments 1 and 2 opened with the emotional host's catchphrase. Each segment is generated in isolation with the same dialogue guide, so each arrives at the same "let's open with the emotional host" decision.
The review step should catch this — it sees the full script at once. And I've added explicit enforcement language telling it to vary the openings. But Gemini 3.5 Flash, the model running the review, apparently finds it harder to restructure segment openings than to trim catchphrases or soften fabricated claims. Opening variety requires regenerating parts of the content, not just tightening or removing. That's a heavier lift for the model.
I'm still working on this — probably a more capable review model or a pipeline restructure where segments share opening state. The broader point: prompt enforcement can't fix architecture problems. If your pipeline's structure makes a rule impossible to follow, no amount of "CRITICAL" or "MANDATORY" in your prompts will save you.
The Framework: Three Questions for Every Rule
After this experience, I now audit every system prompt I write against three questions:
1. Where does this rule live — generation or review? Rules about personality and tone belong in generation. Rules about compliance, consistency, and structure belong in review. If a rule crosses both (like catchphrases — they're personality but need limits), put it in both.
2. Is this rule phrased as context or command? "Here are the catchphrases" is context. "Maximum 2 times each" is command. Both are needed for the same rule — the generation step needs to know what the catchphrases ARE, and the review step needs to enforce how OFTEN they appear.
3. Can the pipeline actually follow this rule? Per-segment generation cannot enforce cross-segment variety. No prompt will fix this. You need to either move the rule to the review step (which sees the full script) or restructure the pipeline so segments share state.
This framework isn't specific to podcast generation. It applies to any multi-step LLM pipeline — content generation, code review, document summarization, anything where you have separate generation and quality-control passes.
I'm still iterating on this, but if you're building your first multi-step LLM pipeline: split it into at least two steps. The first step generates with personality and freedom — give it context, examples, tone guidance. The second step reviews with precision and enforcement — give it explicit limits, compliance checks, and the authority to change things.
The generation step should feel like briefing a creative collaborator. The review step should feel like handing a checklist to an editor.
And when your rules are still being ignored after you've done both, ask the architecture question first. Is it the prompt, or is it the pipeline?
Frequently Asked Questions
Does this apply to a single-prompt setup (no pipeline)?
Yes, but in a limited way. In a single prompt, you're asking the same model to be both creative and compliant simultaneously — it will always favor creativity over compliance when the two conflict. Even in a single prompt, separating your instructions into a "context" section and an explicit "constraints" section helps. But you'll hit a ceiling. Two separate calls with different instructions almost always outperform one.
Why didn't you just put the rules in the generation prompt more forcefully?
I tried. I added "MANDATORY," "CRITICAL," "NEVER," and all-caps everything. The generation step has too many competing demands — it's trying to be engaging, natural-sounding, varied, locally authentic, and compliant all at once. When the model has to choose between "sound like an excited Vietnamese football fan" and "only say 'Trời ơi' twice," the personality always wins. The review step has no such conflict — its only job is to enforce.
What model are you using for the review step?
Gemini 3.5 Flash (for cost — the review runs on every episode across 7 languages). It handles simple enforcement (trimming catchphrases, softening fabricated stats) well. It struggles with tasks that require content regeneration, like restructuring segment openings. A stronger model would help there, but I haven't yet justified the cost.
I build DIALOGUE, an AI podcast platform, solo in my evenings and weekends. The code for the catchphrase extraction and review prompt changes is open — you can read the full system prompt and enforcement logic in my podcast-engine repository. I write about what I learn along the way.
If you've run into a similar problem — rules that are read but not enforced — I'd be curious: what was the rule, and where did it live in your pipeline?
That's it from me for now.
Cheers, Chandler