Experimental Mind #200
Your overview of interesting reads, events and jobs for the experimental mind: #200
This is the 200th newsletter I have published and it took me 9 years. I’m planning to write down some things I have learned from writing, publishing and growing this little newsletter. Let me know what you would like to know about it.
In this edition:
Interesting things you might have missed
Melanie Kyrklund’s: My Experimentation Career Journey
5 featured job opportunities
Upcoming events in the next two months
A quote to think about
And to wrap-up … something to make you smile
Enjoy.
➡️ How to Choose an A/B Testing Tool in 2024 by Convert
The A/B testing tool market is bursting at the seams in 2024, with over 50 options to choose from. What are the factors you should consider when making your decision?
🔎 Interesting things you might have missed
Matt Gershoff wrote Privacy Engineering in A/B Testing and explains how Conductrics is designed with privacy by design principles in mind. They only collect and store the minimum amount of data necessary to run the test. Conductrics does this by de-identifying and aggregating data as it is collected. Interesting approach. READ
Optimizely published an ebbok with lessons learned from running 127,000 experiments. Very nice overview and insights. In response Ronny Kohavi wrote a review with the good and the
badthings to improve.Matteo Courthoud wrote Understanding Group Sequential Testing and explains how to run valid experiments, with peeking and early stopping.
Eppo added Certified Metrics. Which ensures that “core metric definitions are in sync across experimentation in Eppo, BI, and other data platforms, all managed via GitHub.”
Ronny Kohavi ran a poll where he asked about the best reasons NOT to A/B test. Winner: not enough users.
Datapoint: “At Spotify, we've been exercising our experimentation muscle for more than 10 years. We now have over 300 teams running tens of thousands of experiments annually.”
💬 Melanie Kyrklund: My Experimentation Career Journey
Today’s interview is with Melanie Kyrklund, Global Head of Experimentation at Specsavers. Melanie describes herself as a veteran experimenter who is still learning every day, even after 15 years in this field. She's half British, half Swedish Finn, was raised in Italy (Rome) and has been calling Amsterdam home for a long time.
I feel the best learning comes from living through the challenges and thought processes that give rise to solutions yourself. — Melanie Kyrklund
Read the full interview with Melanie
Or check out the other recent interview with Tibor Sisarica, Carlos Trujillo and Shiva Manjunath.
🚀 Job opportunities
Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week’s featured roles:
Senior Product Analyst at Vista (Spain) 👉 become my colleague
Digital Experience Analyst at Journey Further (Multiple locations, United Kingdom)
Senior Data Engineer(Experimentation Platform) at GetYourGuide (Berlin, Germany)
Experimentation Strategy Lead -Insights & Experimentation Enablement at Liberty Mutual (Boston, USA)
CRO Website Senior Analyst at Pearson (Germany)
Are you hiring? Submit here
Also, make sure to follow Experimentation Jobs on LinkedIn.
📅 Upcoming events
A running list of upcoming events.
18 Jan: Can You Guess The Test? (virtual)
20 Jan: MeasureCamp (Malmo, Sweden)
29 Jan – 2 Feb: SUPERWEEK (Hungary)
5 Feb: Accelerating Innovation with A/B Testing (virtual)
7 Feb: Convert Meetup (Bristol, United Kingdom)
10 Feb: MeasureCamp (Karachi, Pakistan)
20-22 Feb: Marketing Festival (Brno, Czechia)
27-29 Feb: Quantum Leap (New Orleans, USA & virtual)
Full list of experimentation conferences & events in 2024
👋 On June 19th I’ll be attending growth marketing SUMMIT, Europe's premier real life conference for digital growth, experimentation and optimization in Frankfurt, Germany. Use code ’experimental-mind‘ to get 10% off and let’s meet there.
💬 Quote to think about
“Experimentation can’t be solved bottom up. If the leadership of an organization doesn’t care about scientific measurement, no one will.” — Chad Sanderson (source)
😃 Something to make you smile
👍 Thanks for reading
If you are enjoying the Experimental Mind newsletter please share this email with a friend or colleague. Looking for inspiration how to best do that? Take a look at what others are saying. Have a great week — and keep experimenting.