Experimental Mind #262

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Kevin Anderson

Apr 14, 2025

Hi folks!

Back with some interesting reads, jobs, a podcast and something that made me smile.

Thanks to Convert and Sitespect for their continued support.

🔎 Interesting things you might have missed

How to build a scaled experimentation engine
A post from Aakash Gupta with insights from OpenAI, Notion, Atlassian, and other leading companies on how to build trustworthy experimentation systems that scale impact and drive consistent product wins. Happy I could contribute to this. Link

____

Booking.com celebrates 20 years of their Experimentation Tool
The video mentions 150k tested ideas and 54k product changes, which adds up to an average ‘win-rate’ of 36%. Link

____

Experimentation maturity program reports 2025
Speero releases their first report on experimentation strategy & culture from 150+ experimentation teams: “Only 12% of survey participants rated their experimentation strategy and culture as transformative”. Link

____

How Airbnb measures listing lifetime value
Airbnb employs a machine learning framework to estimate the lifetime value (LTV) of listings, focusing on three key metrics: baseline LTV (total expected bookings over a year), incremental LTV (bookings that wouldn't occur without the listing), and marketing-induced incremental LTV (additional bookings resulting from Airbnb's marketing efforts). This approach helps in identifying valuable listings, optimizing host support, and enhancing marketplace strategies by accounting for factors like demand-supply dynamics and external disruptions such as the pandemic. Link

____

Effect of basic income on willingness to work
A German study found that providing individuals with €1,200 monthly for three years improved their well-being without reducing their work hours. “This study confirms growing evidence that we and others have long presented: there is no evidence that Basic Income reduces economic and labour market activity”. Link

____

The hidden cost of successful experiments
A Kellogg School study argues that while successful experiments can drive short-term gains, they often introduce hidden complexity that can limit future innovation. This accumulating "complexity debt" can slow down experimentation and may eventually necessitate a complete system overhaul to restore agility. Link / paper

____

Last week’s most clicked item:

Experimentation Program ROI calculator
Juan Cruz Giusto developed a handy experimentation program ROI calculator, in Google Spreadsheets. Link

🎧 Podcast of the week

Lukas Vermeer on Shiva’s From A to B podcast on experimentation, leadership, trust and taco’s.

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

🚀 Job opportunities, supported by Eppo

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

Senior Data Engineer, Data Quality & Experimentation at Strava (San Francisco, USA)
Experimentation Lead at Spreetail (Remote)
Senior Product Manager (Experimentation Platform) at Surfshark (Warsaw, Poland)
CRO Specialist at Veneta (Zwolle, Netherlands)
Data Scientist, Experimentation at Stitch Fix (USA)

📅 Upcoming events

A running list of upcoming events.

🎁12 May: Accelerating Innovation with AB Testing (virtual, get $500 off)
22 May: Experimentation Culture Awards (virtual)
🎁4-5 Jun: Experimentation Elite (Birmingham, UK, get 10% off: EXELITEJOBS10)

🎁Also, don't miss GMS 2025 on June 18, 2025, in Frankfurt! As Europe's #1 growth conference, it's the perfect platform to learn, network, and grow. Use my promo code KEVIN100 to get €100 off your ticket and be part of this event! For more details, visit the official website: www.growthmindedsuperheroes.com