The best of 2023
Your overview of interesting reads, events and jobs for the experimental mind: #198
Happy New Year!
In 2023 I published 25 newsletters, one every other week.
I checked my bookmarks and out around 1.000 bookmarks I selected 172 articles and other links to share with you. In this newsletter I have picked the 8 most popular ones.
The number of subscribers grew with almost 40% to 1.682 today.
I’m thinking of going back to a weekly schedule. Curious what you think of that.
You can help me by filling this super short survey.
🤔 How can I make this newsletter more useful for you?
Here’s a super short, two-question survey. Your feedback would mean a lot. I read all the responses and will take your advice on board for the new year!
🔎 Most popular articles from last year
Replacing an A/B Test with GPT
Interesting idea by Will Kurt: instead of running an A/B test, let an AI model predict the best version. Biggest problem I see: will that version work for your company and for your customers?Meaningful metrics: How data sharpened the focus of product teams
Interesting approach by Duolingo to segment users and use that to instrument their growth strategy. (also weird to learn I am currently a Dormant User in Duolingo’s db).Reflections Conversion Hotel 2023 conference
Last weekend I have been to the Conversion Hotel conference. This was already the tenth edition of this event and I attended six of those, with one as a speaker. Here I will share my personal take aways and some general reflections.
Jonathan Fulton wrote an A/B Testing 101: What I wish I knew about AB testing when I started my career. He covers a bit of everything: randomisation, feature-flagging, metrics, statistics, etc.
The Spotify experimentation team wrote a post that can help in Choosing Sequential Testing Framework.
DoorDash is using a metrics layer to standardise and scale experimentation.
Paper: Statistical Challenges in Online Controlled Experiments
Running experiments at scale has presented challenges that required new statistical methodologies. This paper reviews these challenges and discusses the practice and culture of online experimentation, as well as its statistical literature.Bookmark it, it will come in handy. (I used it last week to make a point)
🚀 Job opportunities
Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week’s featured roles:
Principal Product Manager – Product Analytics & Experimentation at Amazon (Seattle, USA)
Senior Data Engineer – Experimentation Platform at Zalando (Berlin, Germany)
Product Manager – Growth and Conversion at Aira (Stockholm, Sweden)
Staff Engineer, Billing Growth Engineering at Stripe (USA or Canada)
Senior Experimentation Program Manager at HP (Multiple locations, APAC)
Are you hiring? Submit here
Also, make sure to follow Experimentation Jobs on LinkedIn.
📅 Upcoming events
29 Jan-2 Feb: SUPERWEEK (Hungary)
I am still in the process of creating a new list of planned events in 2024. Let me know if you are hosting something and I will make sure to include it.
💬 Quote of the week
Something to think about when you making plans for this years.
All life is an experiment. The more experiments you make the better.
—Ralph Waldo Emerson
😃 This made me smile
👍 Thanks for reading
If you are enjoying the Experimental Mind newsletter please share this email with a friend or colleague. Looking for inspiration how to best do that? Take a look at what others are saying. Have a great week — and keep experimenting.