Experimental Mind #244
Your weekly overview of interesting reads, events and jobs for the experimental mind.
Hi folks!
Next weekend I’ll be at Conversion Hotel conference on the island of Texel, Netherlands. Let me know if you’ll be joining.
This edition articles about: QA, user research, metrics, passwords and more.
➡️ QA-ing Client-Side & Server-Side Experiments
QA in client- and server-side experiments ensures accurate, reliable results. 12 experts weigh in on how to streamline QA for flawless execution in a full-stack approach.
Link to article from Convert
Thanks to Convert, Sitespect, Zoho PageSense and Eppo for their support.
🔎 Interesting things you might have missed
Unbundling user research
User research becomes more and more integrated into product practices. Dave Hora shares his perspective on the democratisation of user research. And what this means for the user researcher and product teams. Link
____
[Paper] It's About Time: What A/B Test Metrics Estimate
This paper from the Spotify experimentation team compares cumulative and windowed metrics in online A/B tests, highlighting trade-offs in complexity, speed, and statistical power. The choice depends on factors like treatment timing and experiment duration. Link
____
Top 200 most common passwords
NordPass reviewed and analysed a 2.5TB database extracted from various publicly available sources. They segmented the data by country. What do you think is the most common password in your country? Link
____
Ten 10 points for assessing the value of a metric
John Cutler put together this great list of points to think about when formulating metrics. Link
____
Enterprise Scan
Handy list of questions for any project or idea you are working on in your organisation. This simple tool can help improve understanding and alignment. Link
____
Last week’s most clicked item:
Advanced Topics in Practical AB Testing
Ronny Kohavi has a new course: Advanced Topics in Practical AB Testing. Ten topics in two days of 2.5 hours each. Warning: prior knowledge assumed. Link
🎧 Podcast of the week
I’m a bit behind with my podcast feed. Only catched up with this one. It’s good.
If you like this episode, make sure to subscribe to the podcast. Or discover other podcasts via the Experimental Mind curated podcast feed.
🚀 Job opportunities
Find 100+ open roles on ExperimentationJobs.com. This week’s featured roles:
Senior Product Manager, Feature Flags and AI Experimentation at Eppo (USA)
Senior Product Manager, CRO at Blinkist (Berlin, Germany)
Senior Consultant CRO at eClerx (Amsterdam, Netherlands)
CRO Consultant at REO (USA or Canada)
Senior Strategy Consultant (Experimentation) at Creative CX (United Kingdom)
📅 Upcoming events
A running list of upcoming events.
20 Nov: Experimentation Meetup (Berlin, Germany)
22-24 Nov: Conversion Hotel (Texel, Netherlands)
Meet me there. Final batch of tickets now available.22-25 Nov: Society for Judgement and Decision Making (New York, USA)
28 Nov: Asia Pacific Experimentation Summit (virtual)
2 Dec: Accelerating Innovation with A/B Testing (virtual)
$500 discount for Experimental Mind readers3 Dec: CRAP Talks BTN #1 - CRAPmas Spectacular! (Brighton, UK
10 Dec: Experimentation Elite (London, United Kingdom)
16 & 18 Dec: Advanced Topics in Practical AB Testing (virtual)
$100 discount for Experimental Mind readers19-20 Dec: Conference on Field Experiments in Strategy (Fontainebleau, France)
Check out more experimentation conferences & events in 2024
😃 Something that made me smile
We all know we need to learn new skills, but this how it often feels.
📣 How was this edition?
Let me know:
If you have made it to here and you are enjoying this newsletter, I would love to read a short testimonial from you.
Read what others are saying and add your own.
Have a great week — and keep experimenting.
Thanks, Kevin