Experimental Mind #235

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Sep 16, 2024

Hi folks!

Happy to welcome Eppo as a new sponsor of this newsletter.

In today’s edition: four interesting things you might have missed, a podcast episode with Lukas Vermeer, five fresh jobs, two new events and something that made me smile. And the question of the week is about spelling.

Thanks to Convert, Sitespect, Zoho PageSense and Eppo for their support.

❔Question of the week

I asked something similar on LinkedIn: how do you spell this word? Curious to see how many people are on team slash, dash or nothing at all?

Read the options carefully…

🔎 Interesting things you might have missed

Peeking not considered harmful
tl;dr Peeking at experiment results before they're finished isn't bad if you do it right. In fact, it can help end experiments sooner when results are clear, saving time. The key is adjusting how you interpret the data to avoid mistakes. You shouldn’t peek too often, but checking a few times during an experiment is optimal for quicker results without sacrificing accuracy.
Link to article

____

Estimating long-run heterogeneous treatment effects using surrogate indices
This post, the fourth in a series on Instacart's Economics Team, discusses the use of surrogate indices to estimate long-run heterogeneous treatment effects of Instacart’s membership incentive portfolio, in collaboration with the Growth Data Science team.
Link to article

____

Last batch of Conversion Hotel tickets
The final 15% of #CH2024 tickets are now available for registration now. This is a great conference, so hope to meet you there.
Get your tickets

____

Book announcement: Tiny Experiments
Anne-Laure Le Cunff, neuroscientist and founder of Ness Labs, is writing a book called Tiny Experiments. It is a guide for the experimental mind. You can already preorder it (like I did). Or first watch the short video by Anne-Laure.

____

Last week’s most clicked item:

The frequentist vs Bayesian split in online experimentation
Georgi Georgiev reflects the frequentist vs bayesian split in online experimentation and puts in context of recent developments in the experimentation space.
Link to article

🎧 Podcast of the week

Lukas Vermeer shares how he got into experimentation and why leadership buy-in + integration with the way of working is essential for adoption of experimentation.

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

🚀 Job opportunities

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

Senior Strategy Consultant (Experimentation) at Creative CX (UK)
Principal Conversion Rate Optimisation (CRO) Consultant at Browser to Buyer (Europe)
CRO & Digital Analytics Specialist at NTT Data (Rome, Italy)
Senior Marketing Analyst, Web and Experimentation at Klaviyo (Denver. USA)
Optimisation Manager (CRO) at iDHL ( United Kingdom)

📅 Upcoming events

A running list of upcoming events.

every Friday: CRO Talks (virtual)
18-19 Sep: Nonprofit Innovation & Optimization (Indianapolis, USA)
21 Sep: MeasureCamp (London, UK)
23 Sep: Accelerating Innovation with A/B Testing (virtual)
$500 discount for Experimental Mind readers
24-26 Sep: ConvEx (virtual)
🆕25 Sep: Significance Summit (San Francisco, USA)
9 Oct: Advanced Topics in Practical A/B Testing (Austin, USA)
10-11 Oct: Experimentation Live + Unite (Austin, USA)
17 Oct: Conversion Jam (Stockholm, Sweden)
18 Oct: Monetate Elevate (London, UK)
18-19 Oct: CODE@MIT (Boston, USA)
7 Nov: DDMA Experimentation Heroes (Amsterdam, NL)
🆕22-24 Nov: Conversion Hotel (Texel, Netherlands)
Meet me there. Final batch of tickets now available.

Check out more experimentation conferences & events in 2024

💬 Quote to think about

“Customers don’t know what they feel, don’t say what they know, and don’t do what they say. Market research is three steps removed from real behaviour.”
— David Ogilvy