Experimental Mind #259

Your weekly overview of interesting reads, events and jobs for the experimental mind.

Mar 17, 2025

Hi folks!

In today’s edition articles on website surveys, multi-armed bandits, why product is so hard and why ‘doing your best’ is not enough anymore. And as always: one podcast to listen to, 5 jobs to explore, a career interview, a couple of events, a quote to think about and a cartoon that made me smile. Have a good week!

➡️ Some of the best qualitative research that almost no one does
Most website surveys fail due to poor execution, but “TriFecta Intent Mapping” improves insights by asking why users visit, if they succeed, and how to imporve their experience.
Link to article from Convert

Thanks to Convert and Sitespect for their support.

🔎 Interesting things you might have missed

Test Plan: Multi-Armed Bandits for Product Recommendations
Ben Labay is working on a study guide to make Contextual Multi-Arm Bandit (CMAB) testing easier to grasp, stressing the need to understand the setup instead of just relying on automation. He believes A/B testing will shift toward model-based approaches like CMAB, which adapts in real-time, optimizes recommendations, and reacts faster to changing user behavior. You can give feedback. Post / Doc

____

Why “your best” isn’t enough and what to do instead
In today's fast-paced environment, only delivering quality work isn't sufficient; success requires active promotion, iteration, and impact-driven efforts. Moreover, viewing work as "best" implies completion, whereas continuous improvement is essential in business. Link

____

Why is product so hard?
Building successful products is challenging due to inherent uncertainties, complexities, and constant changes, as well as divergent needs and misalignment among stakeholders. Also, organisational culture significantly influences a team's ability to adapt and innovate effectively. Read Itamar Gilad’s solutions. Link

____

How Covid changed everything
Fascinating overview of the effect of Covid in 30 charts, like this one:

____

Last week’s most clicked item:

Duolingo’s approach to experimentation: The Green Machine
A nice quote in Duolingo’s latest shareholder letter on the results from 2024:

2024 was an extraordinary year for Duolingo. People often ask me, “What’s the secret behind Duolingo’s success?” The truth is, there’s no secret—just our mission and a process. We call the process "The Green Machine." It’s our approach to experimentation, and it’s the engine behind everything we do, from product development to viral marketing. To see how we operate in more detail, I welcome you all to read the Duolingo Handbook, which we just published a few weeks ago. — Luis von Ahn (CEO and Co-Founder Duolingo)

Link

🎧 Podcast of the week

If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.

💬 My Experimentation Career Journey

Today’s interview is with Xeï Hulshoff, CRO specialist at My Jewellery.

Find a position where you can work together with more experienced specialists, this way you can learn faster than trying to figure out everything by yourself.
— Xeï Hulshoff

Read the full interview with Xeï

🚀 Job opportunities, supported by Eppo

Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:

Director of CRO at Directive (Canada)
Web Experimentation Lead at Capita (United Kingdom)
CRO Specialist at 310 Nutrition (Philippines)
Head of Product Experimentation at Welltech (Multiple locations)
CRO Specialist at Penn Foster Group (USA)

📅 Upcoming events

A running list of upcoming events.

18 Mar: Conversion Boost (Copenhagen, Denmark)
19 Mar: Experimentation London Meet-up #4 (London, United Kingdom)
19 Mar: CRAP Talks #28 (London, United Kingdom)
19 Mar: EXL Meetup (Hamburg, Germany)
🆕26 Mar: ProductTank: The Art of Experimentation (London, UK and virtual)

🔥Deadlines

20 Mar: case entry Experimentation Elite Awards
31 Mar: case entry Experimentation Culture Awards

🎁Also, don't miss GMS 2025 on June 18, 2025, in Frankfurt! As Europe's #1 growth conference, it's the perfect platform to learn, network, and grow. Use my promo code KEVIN100 to get €100 off your ticket and be part of this event! For more details, visit the official website: www.growthmindedsuperheroes.com

💬 Quote to think about

“Some failure in life is inevitable. It is impossible to live without failing at something, unless you live so cautiously that you might as well not have lived at all – in which case, you fail by default.”
— J.K. Rowling