Experimental Mind #247

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Dec 09, 2024

Someone recently asked me: "If you started your newsletter today, would you choose Substack?" Short answer is yes, read here why.

In today’s edition: some predictions, a poll, articles about novel experimentation techniques, a podcast, an interview, events and more.

➡️ Top 5 A/B Testing Trends for 2025
A/B testing in 2025 will emphasize agility, privacy, and advanced technologies. These trends aim to help businesses create more user-centric products and services through adaptive, privacy-conscious experimentation.
Link to article from Sitespect

Thanks to Convert, Sitespect, Zoho PageSense and Eppo for their support.

❔Question of the week

It’s December, time for reflection and lists. Lot’s of lists.

🔎 Interesting things you might have missed

Measuring Product Impact Without A/B Testing
How Discord used the synthetic control method to measure the impact of their voice messages feature. Synthetic controls are a powerful addition to your causal inference toolkit when traditional A/B tests aren't feasible—due to logistical or ethical constraints. Link

____

[Paper] Contrastive representations of high-dimensional, structured treatments
In this paper, Spotify researchers tackle the challenge of estimating causal effects for high-dimensional, structured treatments like text, video, or audio, which traditional methods struggle to analyse. By developing a novel contrastive approach that learns to identify and focus on causal factors while discarding non-causal information, they demonstrate an innovative method for more accurate causal effect estimation across synthetic and real-world datasets. Link

____

Female Speakers in Growth and Experimentation
Organising an event and looking for a speaker? There is this great list full of great female speakers. Use it. Link

____

Last week’s most clicked item:

Four Levels of Data Proficiency
Successful companies create a "culture of truth-seeking" by systematically using data to understand customer behavior, making informed decisions, and continuously testing and validating their assumptions. Link

🎧 Podcast of the week

There is new podcast in the experimentation space: growth-minded SUPERHEROES. It’s hosted by André Morys and Julia Rumpf from konversionsKRAFT. First three episodes already dropped: Slobodan (Sani) Manić, Lukas and Vermeer and (the one I chose to share this week) Stewart Ehoff. Stewart shares some lessons from the experimentation journey at RS.

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

💬 My Experimentation Career Journey

Today’s interview is with Arslan Nasir. He is a CRO Developer and open to work.

In the next 5–10 years, I see experimentation becoming more predictive and integrated with personalization strategies, while the fundamental principles of user-centric design and data-driven decision-making will remain constant. — Arslan Nasir

Read the full interview with Arslan