Experimental Mind #242
Your weekly overview of interesting reads, events and jobs for the experimental mind.
Hi folks! Another collection of things I’ve been learning this week. Enjoy.
➡️ Reasons to Run Experiments With Group Sequential Testing
Group sequential testing is a more efficient statistical model that allows your team to check in on test results at multiple predefined points during an experiment. It helps to make accurate evaluations earlier. Five reasons you may want this.
Link to article from Sitespect
Thanks to Convert, Sitespect, Zoho PageSense and Eppo for their support.
🔎 Interesting things you might have missed
The Measurement Trap
tl;dr Metrics are helpful but shouldn’t be the whole story. Great leaders look beyond numbers to include what can’t be easily measured, for a balanced, long-term approach to success. Link
____
Getting buy-in for experimentation
David Bland created these AI prompts to customise your pitch per role with points and counterpoints. Link
____
Using Microsoft Clarity in the EU? Make sure you have consent
tl;dr Make sure that Clarity is only activated on your website if the person visiting gives their consent for marketing activities. Clarity must not be categorised under analytics or statistics. Link
____
Jeff Bezos on one-way vs two-way door decisions
One-way door decisions are irreversible and demand careful consideration, while two-way door decisions are reversible, allowing for faster action and experimentation. Bezos explains that recognizing the difference can help organizations innovate more effectively. Most decisions are two-way door decisions and should be taken by people on the ground, close to the context.
____
Last week’s most clicked item:
A recap of the Experimentation Conference 2024 at Booking.com
A recap of this inhouse conference where 90 practitioners from 24 companies joined, and discussed common challenges, exchange learnings, and advance experimentation, on several topics. Great event. Link
🎞️ Talk of the week
Georgi Georgiev presented at TLC about observed power. He emphasizes that post-test estimates, like observed power or observed effects, should not be confused with pre-test targets, such as the chosen power level or minimum detectable effect. He warns against interpreting low observed power in non-significant outcomes as a reason to extend or rerun tests, as non-significant results often appear underpowered by default. Worth a watch.
🚀 Job opportunities
Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:
CRO Consultant at REO (USA or Canada)
Senior Product Analyst at Vista (Spain)
Digital Experience Experimentation Analyst at Norwegian Cruise Line (USA)
CRO specialist at Voordeeluitjes (Amstelveen, Netherlands)
Senior Strategy Consultant (Experimentation) at Creative CX (United Kingdom)
📅 Upcoming events
A running list of upcoming events.
7 Nov: DDMA Experimentation Heroes (Amsterdam, NL)
7 Nov: Best In Test ceremony (virtual)
12 Nov: Dag van het gedrag (The Hague, NL)
13 Nov: Experimentation London Meetup (London, UK)
14 Nov: A/B Test Talks (New York, USA)
14 Nov: EXL Meetup (London, UK)
🆕20 Nov: Experimentation Meetup (Berlin, Germany)
22-24 Nov: Conversion Hotel (Texel, Netherlands)
Meet me there. Final batch of tickets now available.22-25 Nov: Society for Judgement and Decision Making (New York, USA)
28 Nov: Asia Pacific Experimentation Summit (virtual)
2 Dec: Accelerating Innovation with A/B Testing (virtual)
$500 discount for Experimental Mind readers3 Dec: CRAP Talks BTN #1 - CRAPmas Spectacular! (Brighton, UK
10 Dec: Experimentation Elite (London, United Kingdom)
19-20 Dec: Conference on Field Experiments in Strategy (Fontainebleau, France)
Check out more experimentation conferences & events in 2024
💬 Quote to think about
“Most decisions are two-way doors...you pick a door, you walk out, you spend a little time there; it turns out to be the wrong decision, you can come back in and pick another door. Some decisions are so consequential and so important and so hard to reverse that they really are one-way door decisions...those decisions have to be made very deliberately, very carefully." — Jeff Bezos
😃 Something that made me smile
📣 How was this edition?
Let me know:
If you have made it to here and you are enjoying this newsletter, I would love to read a short testimonial from you.
Read what others are saying and add your own.
Have a great week — and keep experimenting.
Thanks, Kevin