This week: six reads with a common theme: what we measure, when we stop, and whether going faster is always better. And as always the latest open roles for experimenters: more than 1k jobs this time.
Brought to you by
Convert—A/B tests & personalization for growth teams
Experimentation Jobs—Find your next role
🔎 Interesting things you might have missed
The “art” part of setting metrics
By Randy Au — Picking metrics sounds technical but is mostly a messy co-ideation exercise. Teams default to vanity metrics; experienced practitioners push back by asking what decision the metric forces. The goal isn’t the perfect metric, but to pick a sensitive enough alarm bell that triggers when something needs investigating. READ
____
Experiment stopping rules
By Jakub Linowski — An interactive simulator comparing five stopping rules on cumulative gain, speed, exaggeration and error rates. No single winner, but naive peeking is more dangerous than most think, especially under low traffic with many losers in the mix. PLAY
____
Submit your case: Experimentation Heroes 2026
Three categories: Conversion, Experimentation Culture, and Beyond Web. New quick-entry process takes just 20 minutes. Shortlist deadline June 26. SUBMIT
____
Ryanair dark UX patterns
By Dan O’Sullivan — Nine steps stand between you and a free Ryanair check-in. This is a masterclass in friction as revenue trick. READ
____
When AI writes the code, who decides what ships?
By Johan Rydberg (Spotify) — AI has made building 8x faster. The bottleneck is now validation. Data across 22,000 developers shows bugs are up 54% and incidents up 58% as AI adoption rises. We need experimentation to validate what we ship. READ
____
Metrics maxxing
By Scott Galloway — Optimising your life is fine until the metric becomes the goal. Value capture replaces rich, meaningful living with dashboards. This piece is a bit of a pushback to the optimisation trend. READ
____
Last week’s favourite:
Good metrics maximize surprise
By Zach Flynn — If your metric is guaranteed to go in the direction you want, you’re running propaganda, not an experiment. Good experiment metrics can go either way, and let customer behavior decide the outcome. READ
🚀 Job opportunities
Find 1000+ open roles on ExperimentationJobs.com. This week’s featured roles:
Lead UX Consultant / Team Lead at Creative CX (London, United Kingdom)
Fractional Growth Lead – GTM Experiments at Reflow (USA & Canada, Remote)
CRO Manager – Personalisation & A/B-Testing at Teufel (Berlin, Germany)
Conversion Rate Optimization Specialist at Pluxee (Lisbon, Portugal)
Conversion Strategy & Offer Psychology Manager at Everdrop (Munich, Germany)
📅 Upcoming events
A running list of upcoming events. Subscribe here. (👋= join me, 🎁= discount)
🎁15 Jun: Advanced Topics in Practical A/B Testing with Ronny Kohavi (online,
$1,199$999 USD)16 Jun: Process Mining 101 (online)
17 Jun: TLC Panel: Accessibility, AI Discoverability, & SEO with Shirley van Haalem, Alisa Scharf, and Sani Manic (online)
23 Jun: Causal Summit (online)
🔥26 Jun: Deadline shortlist submission Experimentation Heroes
👋16 Jul: Experimentation Culture Awards (online)
9-10 Sep: Women in Experimentation Summit (online)
👋24 Sep: Experimentation Elite Awards (London, UK)
👋5 Nov: Experimentation Heroes (Amsterdam, Netherlands)
😃 Something that made me smile
When you’re ready, here’s how I can help
If this newsletter sparked something and you want to talk it through, a few ways I help:
Scale experimentation
Strategy, setup, metrics, and ways of working for teams that want more impact.Careers and hiring
Support with your next role, or help finding and hiring strong experimenters.Quick sparring
A fresh outside perspective on your ideas, roadmap, or experimentation setup.
Interested?
Just hit reply and tell me what you’re thinking about.
Have a great week and keep experimenting.
Thanks, Kevin