Experimental Mind #252

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Jan 27, 2025

Hi folks! In today’s edition a good read on probablity, a fun game on dark patterns, a podcast with Jorden Lentze, fresh job opportunities and some discounts for events.

➡️ Stats 101 — A Visual Guide
Do you feel confident navigating the complexities of p-values and standard deviations to make confident choices from your test results? This visual guide will equip you with the foundational knowledge you need to interpret results more confidently.
Link to article from Convert

Thanks to Convert, Sitespect and Eppo for their support.

❔Question of the week

January is almost over. Can you believe it? Only ~330 days untill Christmas.

🔎 Interesting things you might have missed

‘Stop A/B testing’
Avinash Kaushik is someone I admire a lot. He influenced much of my thinking about analytics. Recently Avinash published a post where he states that we can better stop A/B testing. That was a shocker to me. Of course, I agree that we should not do bad A/B testing, but stopping it all together is blaming the method, not the real thing that is broken. Ronny Kohavi wrote a detailed response, including a link to Avinash’s original piece. Link

____

Dark Patterns Detective
Fun game where you can uncover the hidden design tricks that manipulate user decisions online. Link

____

Probability Probably Doesn’t Exist
Life is filled with uncertainty, which we often express with words or numbers, though both can be misleading. Probability, while widely used in science and predictions, is not an objective truth but a construct shaped by judgments and assumptions.
Link

____

Does it matter that online experiments interact?
Experiments on mature websites often run simultaneously, raising concerns about potential interactions where one experiment's effects depend on another's variants. However, most interaction effects are small and rarely impact decision-making, offering reassurance even in complex experimental setups. Link

____

Causal analysis overview
While experimentation is a powerful tool for answering causal questions, there are other methodologies that can also help derive causal conclusions from observational data. This is an introductory overview of causal analysis describing three methodologies used to generate causal insights to power data-driven decision making. Link

____

[Paper] Experimentation in Industrial Software Ecosystems: an Interview Study
A study that explores the challenges of conducting experiments in industrial software ecosystems through interviews with 25 industry professionals across 17 ecosystems. It identifies key impediments like data integration issues, regulatory constraints, and organizational culture, offering actionable insights to enhance experimentation effectiveness. Link

____

Last week’s most clicked item:

New experimentation and causal inference tech blog
Alejandro Alvarez Pérez, lead Data Scientist at Adevinta, launched his experimentation and causal inference tech blog. The first post is about mastering the Poisson distribution: Intuition and Foundations. Link

🎧 Podcast of the week

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

🚀 Job opportunities

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

Senior Technical Consultant (Experimentation) at Creative CX (United Kingdom)
Senior Conversion Rate Optimisation (Experimentation) Consultant at Conversion.com (Farringdon, United Kingdom)
Conversion Rate Optimization (CRO) Assistant at WKND Digital (Sacramento, USA)
CRO Manager (Mobile Apps) at Leadtech (Barcelona, Spain)
Experimentation Commercial Lead at Sky (London, United Kingdom)

📅 Upcoming events

A running list of upcoming events.

27 Jan-7 Feb: The Good Experimental Design Workshop (virtual)
27-31 Jan: SUPERWEEK (Hungary)
30 Jan: EXL Meetup (Sao Paulo, Brazil)
30 Jan: EXL Meetup (Rio, Brazil)
🎁26-28 Feb: Experimentation Island (USA) Get $250 discount on your ticket!

🎁Also, don't miss GMS 2025 on June 18, 2025, in Frankfurt! As Europe's #1 growth conference, it's the perfect platform to learn, network, and grow. Use my promo code KEVIN100 to get €100 off your ticket and be part of this event! For more details, visit the official website: www.growthmindedsuperheroes.com (still early bird tickets)