Skip to main content

Release impact, before vs. after

Releases either land or they don't. The fastest way to know is to compare the same signal in the window before and the window after the ship. This pattern works for any signal: rage clicks, dead clicks, errors, drop-off, or conversion rate.

Turn 1: Find what changed​

Fullstory MCP
Compare frustration signals on the checkout page in the 7 days after May 10 vs. the 7 days before. What got worse?
Fullstory MCP
AI Response

Checkout frustration: 7 days after May 10 vs. 7 days before

Got worse:

  • Rage clicks on shipping-step button: 1,206 → 1,617 (+34%)
  • Dead clicks on "Apply Promo": 412 → 519 (+26%)

Roughly flat:

  • 404 errors, JS exceptions, console errors

Got better:

  • Dead clicks on shipping address autocomplete: 247 → 198 (−20%)

Turn 2: Drill into the worst regression​

Fullstory MCP
Pull 3 sessions where users rage-clicked the shipping-step button after May 10. What's the common pattern?
Fullstory MCP
AI Response

3 sessions, common pattern:

  1. User fills in the shipping address
  2. Clicks "Continue to Payment"
  3. Nothing visibly happens for 2 to 4 seconds
  4. User clicks 3 to 5 more times in quick succession
  5. The page eventually advances to the payment step

The button does work — it has a noticeable delay with no loading state. This is a perceived-performance regression, not a functional break.

Turn 3: Size the cost​

Fullstory MCP
How many users hit that pattern per week, and what's the conversion rate of affected sessions vs. unaffected?
Fullstory MCP
AI Response

Weekly impact:

  • ~1,617 users per week hit the rage-click pattern
  • Conversion rate of affected sessions: 51.2%
  • Conversion rate of unaffected sessions: 69.4%
  • Delta: 18.2 percentage points

Estimated lost conversions: about 290 per week, assuming the gap is causal rather than correlative.

Turn 4 (optional): Set up the ongoing watch​

Fullstory MCP
Build me a metric tracking rage clicks on the shipping-step button as a daily trend. I want to monitor whether this stays elevated.
Fullstory MCP
AI Response

Built a draft metric: "Shipping-step button rage clicks (daily trend)". View and save it in Fullstory to make it permanent: [metric link]

What you walk out with​

A delta (+34%), a pattern (perceived-performance delay, not a break), a per-week user count (1,617), an estimated weekly conversion cost (~290), and a draft metric ready to save and watch going forward.

Behind the scenes​

The MCP routes this workflow through: discover_groups with compare_to_previous=true (which returns the same-window vs. prior-window comparison sorted by biggest regressions), get_sessions, get_session_events, build_metric for the cost-sizing step, and build_metric again for the ongoing watch.