Phase 4 of 6
Improve the campaign
Completion fell. What should the team change, and what is still unknown?
Campaign phases
Separate synthetic September cohorts show a decline but do not identify its cause; they are not results of launching this proposed plan. Compare each group after its full seven-day period to decide what to investigate.
The work · Dated signal, diagnosis and decision trail
What the team knew, and when
The comparison is retrospective. Compare the groups after both seven-day observation periods have ended.
Quiz completion fell from 45% to 28% between the two September groups after their seven-day observation periods.
Paid mobile · two September groups, each observed for seven days
- Week 1 ·
- 45%3,780 / 8,400
Week 1: 3,780 of 8,400 paid-mobile starts completed.
Seven-day observation window. Cohort is a retained-journey count. - Week 2 ·
- 28%2,352 / 8,400
Week 2: 2,352 of 8,400 paid-mobile starts completed.
Seven-day observation window. Both fictional cohorts were built using the same rules.
- Change
- -17 percentage points
- Fixed mix
- 35.33%descriptive, not causal
Paid-mobile groups cover September 1–14, 2026. Both seven-day observation periods ended before the comparison on September 21, 23:59:59 UTC; the figures were recorded September 22.
Fixed-mix check
Across 12 mobile channel × audience × creative strata, Week 2 is 35.33% using Week 1 weights, versus 28.0% observed. The gap shows composition contributes to the observed/fixed-mix difference; all 12 stratum rates also fell from Week 1. This is a descriptive decomposition, not a causal estimate.
Source: Checked campaign data; Week 1 weights for the 12 channel × audience × creative mobile strata.
- September 21, 23:59:59 UTC
- First comparison after Week 2’s seven-day observation period.Both seven-day observation periods had ended; the figures were recorded on September 22.
- Decision
- Investigate audience mix and the mobile journey before moving budget.Cause remains unresolved; no early alert is claimed.
From signal to bounded next review
Newly authored decision illustration using the existing synthetic cohort evidence; not a recorded AI response or human session.
- Checked signal
- After both seven-day windows, paid-mobile completion was 45% (3,780/8,400) for Sep 1–7 and 28% (2,352/8,400) for Sep 8–14: down 17 percentage points. First comparable at Sep 21, 23:59:59 UTC.
- Proposed recommendation
- Investigate audience mix and reproduce the mobile journey and event capture before moving budget. The counts show a change, not its cause.
Human disposition
Defer budget and creative changes. Accept investigation as the next step, not as a confirmed diagnosis. A human review disposition is illustrated here; none is recorded in the source pack.
Bounded next evidence
For the next review, reconcile audience mix, compare the cohorts’ page/build release history, and test one consented mobile journey against quiz_start and quiz_complete capture and deduplication. Reassess after those checks; no alert or targeting change is claimed.
Why this choice
The decline is observed; the cause is not. A change in audience mix and a defect in the mobile journey can both lower completion, and this dataset cannot separate them.
Next step
The diagnostic question, comparable cohorts and evidence requests go to reporting.
Inspect further: Evidence that would change the recommendation
The team should name this before looking at another chart.
- Version-assignment records showing a different quiz build in Week 2.
- Event capture that missed completions on one mobile browser.
- A creative or audience change large enough to explain the mix shift.
- A reproduced journey that fails before the completion event.
Full reference: Monitoring and decision log
Record what changed, where the figures came from and when both seven-day periods ended. The monitoring steps below are a proposal; they did not happen in this example.
- Proposed early monitor: launch day through hour 48, check delivery, consented event capture and duplicate journeys at the start and end of each shift. Owner: growth lead with analytics support. Stop and investigate if events fail reconciliation; this example does not claim an alert fired.
- September 1–7 and September 8–14: compare each paid-mobile cohort after its seven-day window. Week 2 is lower; audience mix and mobile journey remain competing explanations.
- The first full comparison was possible September 21, 23:59:59 UTC. Before changing budget, compare audience composition and the mobile journey, and confirm which page version visitors saw.