Experimental Mind #220

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Jun 03, 2024

Hi folks!

Did you know that the t-test originated in the Guinness brewery? You can read it by following the links in this newsletter. Including other reads you might have missed, a career interview with Stefanie Grimmling and something that made me smile.

Let’s go.

➡️ Best A/B testing tools (2024): Alternatives to Google Optimize
List of popular A/B testing tools with common features highlights.
Link to the article from Zoho PageSense

Thanks to Convert, Sitespect, Exatom and Zoho PageSense for their support.

🔎 Interesting things you might have missed

The Cult of Stat Sig
Eppo’s Giorgio Martini explains how the cult around statsig was born, how it can harm decision-making and how we can excape from it.
Link to article

____

How the Guinness Brewery Invented the Most Important Statistical Method
”It was the birthplace of the t-test, one of the most important statistical techniques in all of science. When scientists declare their findings “statistically significant,” they very often use a t-test to make that determination. How does this work, and why did it originate in beer brewing, of all places?”
Link to article

____

Relevant and non-relevant users in A/B-testing
Lars Hirsch states that it is common to make errors around identifing the relevant set of participants in the experiment: users who were eligible to be impacted by the experiment.
Link to article

____

A/B Testing Gets an Upgrade for the Digital Age
Researchers from Stanford Graduate School of Business are exploring new ways of doing experimentation.
Link to article

____

[Paper] False Positives in A/B Tests
Ronny Kohavi and Nanyu Chen have their paper accepted to KDD 2024. You can still give feedback to this final draft version until June 12.
Link to paper

____

New course: Accelerating Innovation with A/B Testing
BTW: Ronny Kohavi starts a new course on June 24.
Enroll (and get 500 dollar off)

____

Experimenation Events Budget Guide
Kelly Wortham from TLC put this resource together to help you understand how much bduget you would need to attend popular events in our industry.
Link to overview

____

Split gets acuired by Harness
Split is joining Harness, an end-to-end platform for software delivery.
Link to article

💬 My Experimentation Career Journey

Today’s interview is with Stefanie Grimmling. Stefanie is a member of the Management Team & Senior Managing Consultant at konversionsKRAFT. And she will be presenting at the growth marketing SUMMIT in Frankfurt am Main, later this month.

It’s important to be clear about what you believe in and how you think about trying new things. You should have a strong set of ideas about how to focus on users and how to experiment well, in addition to knowing how to do the experiments themselves. — Stefanie Grimmling

Read the full interview with Stefanie

🚀 Job opportunities

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

CRO Specialist at Sofatutor (Germany/Global)
Engineering Manager – Experimentation at LaunchDarkly (USA)
Experimentation Analyst at The Financial Times (London, United Kingdom)
CRO Data Analyst at Hostinger (Lithuania)
CRO Lead at CommerceCore (Vilnius, Lithuania)

Experimentation Jobs has been shortlisted for an Experimentation Elite Award + nominated for the Experimentation Culture Award.🎉

📅 Upcoming events

A running list of upcoming events.

4 Jun: CRO Belgium (Ghent, Belgium)
4-7 Jun: Marketing Analytics Summit (Phoenix, USA)
6 Jun: Women in Experimentation Summit (virtual)
6 Jun: Digital Marketing Live (Amsterdam, Netherlands)
6-7 Jun: NeuroPsychoEconomics (Milan, Italy)
8 Jun: MeasureCamp (Copenhagen, Denmark)
🎁11-12 Jun: Experimentation Elite (Birmingham, United Kingdom)
Use code ’EXPJOBS10‘ to get 10% off
12 Jun: A/B Test Talks with Statsig (Berlin, Germany)
15 Jun: MeasureCamp (Paris, France)
🎁19 Jun: Growth Marketing SUMMIT (Frankfurt am Main)
🎁 24 Jun: Accelerating Innovation with A/B Testing with Ronny Kohavi (virtual, ~~$1,999~~ $1,499 USD)

Full list of experimentation conferences & events in 2024

👋 On June 19th I’ll be attending growth marketing SUMMIT, Europe's premier real life conference for digital growth, experimentation and optimization in Frankfurt, Germany. Use code ’experimental-mind‘ to get 10% off and let’s meet there.