Experimental Mind #261

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Mar 31, 2025

Hi folks! Enjoy this week’s edition of interesting reads.

➡️ 4 Hybrid A/B testing myths that could be holding you back
Hybrid A/B testing combines client-side and server-side testing, which enables comprehensive evaluation across design and backend features. But many teams hesitate due to misconceptions, missing out on the value.
Link to article from Sitespect

Thanks to Convert and Sitespect for their continued support.

🔎 Interesting things you might have missed

Google’s experiment on the value of European news content
Google did an experiment removing European news content for 1% of users in eight EU countries to assess its impact. The results indicated no significant change in Google's ad revenue and only a 0.8% decrease in usage, suggesting minimal effect from the absence of news content. Link | Report (PDF)

____

Experimentation Program ROI calculator
Juan Cruz Giusto developed a handy experimentation program ROI calculator, in Google Spreadsheets. Link

____

Outdated distinction: innovation vs development
The traditional separation between innovation and development is becoming obsolete in modern product companies. He emphasizes that, with the adoption of continuous deployment, innovation should be integrated into the development process to foster continuous improvement and adaptation. Link

____

Give your promising A/B tests a second chance
Jakub Linowski from GoodUI offers an expert look at you testing backlog, test design from recent experiments, high-quality test designs, test analysis, and a personalised training session. Link

____

The hidden costs of OKRs: when goal-setting becomes a burden
Like any framework, OKRs are not a silver bullet. When applied as one without careful consideration to the unique needs of every aspect of your organization, you may find yourself in a situation where OKRs aren’t an asset at all, but rather a burden for your teams to create and implement. Link

____

Last week’s most clicked item:

Using Causal Inference for Measuring Marketing Impact
How BBC Studios uses causal inference techniques, particularly geo-holdouts and synthetic control methods, to measure the true impact of marketing campaigns. By comparing treated and control regions, they isolate marketing effects from external factors, enabling more accurate decision-making on ad spend and strategy. Link

🎧 Podcast of the week

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

💬 My Experimentation Career Journey

Today’s interview is with Wing Yee Tang, Senior CRO specialist at DPG Media.

I often try to reflect on different situations, to assess whether I could have done things differently. — Wing Yee Tang

Read the full interview with Wing Yee Tang

🚀 Job opportunities, supported by Eppo

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

CRO Specialist at All4Running (Alkmaar, Netherlands)
Experimentation Program Specialist IV at Uber (Chicago, USA)
CRO-Manager at Internet Up (Munich, Germany)
Website Conversion Rate Optimisation (CRO) Manager at Team Internet (London, United Kingdom)
CRO Specialist at Booking Experts (Enschede, Netherlands)

📅 Upcoming events

A running list of upcoming events.

🔥Deadline 31 Mar: case entry Experimentation Culture Awards
3 Apr: EXL Meetup (Seattle, USA)
🎁12 May: Accelerating Innovation with AB Testing (virtual, get $500 off)
🎁4-5 Jun: Experimentation Elite (Birmingham, UK, get 10% off: EXELITEJOBS10)

🎁Also, don't miss GMS 2025 on June 18, 2025, in Frankfurt! As Europe's #1 growth conference, it's the perfect platform to learn, network, and grow. Use my promo code KEVIN100 to get €100 off your ticket and be part of this event! For more details, visit the official website: www.growthmindedsuperheroes.com