Experimental Mind #224

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Jul 01, 2024

Hi folks!

In today’s edition: meta-experiments, checkout optimization, SRM issues and an alternative to A/B testing.

Have a good week.

➡️ 14 AB test tools and their characteristics
Zoho Pagesense created an overview of different A/B testing tools in the market. For each tool their is a short description and their most important characteristics.
Link to article from Zoho PageSense

Thanks to Convert, Sitespect and Zoho PageSense for their support.

🔎 Interesting things you might have missed

Meta-experiments
Melanie Muller explains how Booking.com runs experiments on experiments to causally link intervention to improvements.
Link to article

____

Checkout Optimization
The Baymard Institue shows five specific ways to minimize the number of visible form fields. It is more important to reduce the things what user have to do at form steps, than to lower the number of steps.
Link to article

____

SRM issues in the Upworthy Archive
The Upworthy Archive is an overview of 32k experiments. New analysis shows that 22% of A/B tests had sample ratio mismatch (SRM) issues. They are everywhere: make sure you check for SRM.
Link to article

____

Causal Effects when A/B Tests are Infeasible through Adopter Analysis
Avanti Chande, Principal Data Scientist at Walmart, shows an alternative way when conducting an A/B test is not feasible due to a lack of control group, resource limitations, or strategic business decisions.
Link to article

🚀 Job opportunities

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

Growth Product Manager at SE Ranking (Europe)
Conversion Rate Optimisation (CRO) Manager at Malaberg (London, United Kingdom)
Experimentation Manager at Surfshark (Vilnius, Lithuania)
Sr. Manager, Software Engineering – MLOps & Experimentation at Realtor (Austin, USA)
Ecommerce Manager – Conversion Rate Optimization (CRO) at Life360 (USA)

Experimentation Jobs has been shortlisted for an Experimentation Elite Award + nominated for the Experimentation Culture Award.🎉

📅 Upcoming events

A running list of upcoming events.

5 Jul: Nudgestock (virtual)

Check out more experimentation conferences & events in 2024

💬 Quote to think about

We have a more experimental culture than a fancy restaurant—we encourage Duos to try things and take big swings. Those inevitably will come with failures. But if we can learn from those failures for our next big swing, then we’re more likely to succeed.
— Luis von Ahn, CEO Duolingo (source)