Experimental Mind: Matteo Courthoud, Lightweight Experiments, Outcome Roadmaps...
Your overview of interesting reads, events and jobs for the experimental mind: #195
Another overview of interesting reads, events and jobs in the experimentation space. Plus a great interview with Matteo Courthoud from Zalando.
➡️ From Spaghetti Testing to Strategic Experimentation by Convert
Iterative testing is the process of continuously testing and optimizing based on learnings from previous experiments. It goes beyond win/loss predictions to uncover insights through experimentation. Read the full article
💬 Matteo Courthoud: My Experimentation Career Journey
Today’s career interview is with Matteo Courthoud. Matteo Courthoud is an Applied Scientist at Zalando, where he specializes in causal inference and experimentation methodology. He recently obtained his PhD in Economics from the University of Zurich and is passionate about translating academic research into industry impact.
AI will definitely impact experimentation, but, if anything, it will only increase the need for people who can study causal questions.
Read the full interview with Matteo
🔎 Interesting things you might have missed
How to design lightweight experiments to test your hypothesis
Nice little video by Jeff Gothelf where he walks you through a hypothetical example that shows the thought process from business problem to solution hypothesis to lightweight experiments. He explains how to think through the biggest risks to the hypothesis, what you need to learn about them and how you might go about learning those things in 3 time horizons. WATCHPlanning with outcome roadmaps
“Classic roadmaps come with a big promise: they allow you to plan resources, track progress, and coordinate launches. While they don’t really deliver, the desire to have these things isn’t likely to go away.” Itamar Gilad propose an outcome based roadmap. READ
A quick business guide to randomized experiments
Robson Tigre wrote a post on designing randomized experiments in a business setting, tailored for a non-technical audience: “as a business professional, understanding experimental design is crucial”. READ
Raising the bar by lowering the bound
Post by Nils Skotara from the Booking.com Data Science team wrote this post stressting that using point estimates from underpowered experiments to measure impact can provide overly optimistic results; instead, the lower bound of the confidence interval better accounts for uncertainty. READ
7 key questions about how to participate appropriately in complex systems
An older post that was doing the rounds again. When designing experiments it is really important to think about broader impact. This post has a list of questions “that we can ask ourselves to avoid the most common mistakes in dealing with complex systemic issues”. READ
🚀 Job opportunities
Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week’s featured roles:
Data Analist/Scientist at Online Dialogue (Utrecht, Netherlands)
CRO Account Manager at Croud (Shrewsbury, United Kingdom)
Data engineer (AI systems) at Textkernel (Amsterdam, Netherlands)
Conversion Rate Optimization (CRO) Specialist at Pilothouse (Victoria, Canada or Remote)
Sr. Director of Product Management – Online Direct Sales Experience at Global Payments (Oklahoma City, USA)
Are you hiring? Submit here
📅 Upcoming events
I will be attending Conversion Hotel next month. Will I see you there?
17-20 Nov: Society for Judgement and Decision Making (virtual)
17-19 Nov: Conversion Hotel (Texel, Netherlands) I’m here, come say 👋!
🆕29 Nov: Experimentation Meetup (Barcelona, Spain)
🎁4 Dec: Accelerating Innovation with A/B Testing with Ronny Kohavi (virtual,
$1,999$1,499 USD)7 Dec: Experimentation Elite (London)
See all experimentation related conferences & events 2023.
👍 Thanks for reading
If you are enjoying the Experimental Mind newsletter please share this email with a friend or colleague. Looking for inspiration how to best do that? Take a look at what others are saying. Have a great week — and keep experimenting.