Experimental Mind: Misconceptions, Portfolio Thinking, Echo Chambers, Four-Day Workweek...

🔎 What I've been reading last week

Georgi Georgiev takes down four common myths around A/B testing:

1. Business experiments need not be scientifically rigorous

2. Statistical rigor means testing with a 95% confidence threshold

3. Proper statistics means an A/B test has to be evaluated just once

4. Statistically rigorous A/B tests are impractical

Having an answer on these questions/myths is important.

Sounds like someone with an experimental mind:

When we’re not certain of the right answer, the best approach is to have a portfolio, a range of bets that reward us with resilience and significant upside.

There is lot's of talk about echo chambers, but how big is this problem actually? This piece examines what social science says about echo chambers, filter bubbles and online polarisation more widely.

In summary, the work reviewed here suggests echo chambers are much less widespread than is commonly assumed, finds no support for the filter bubble hypothesis and offers a very mixed picture on polarisation and the role of news and media use in contributing to polarisation.

I love these kind of experiments:

Workers at more than 30 UK businesses are taking part in the pilot, which will see the working week reduced to four days. They will be asked to do the same amount of work as before, and for up to 35 hours per week, but this will be split over four days not five. For the same amount of pay.

💡 Learn directly from Ronny Kohavi

This is a unique opportunity to learn directly from Ronny Kohavi. The second cohort of 'Accelerating innovation with A/B testing' starts January 31st.

You have only a couple days to enroll

🚀 Job opportunities

New opportunities keep opening up. Experimentation is a fascinating field to work in. Checkout the 29 open roles.

These are the latest featured roles:

• 🆕 Data Scientist Experimentation at Just East (London)

• 🆕 Digital Experimentation Manager at Philips (Amsterdam)

• Head of Product Experimentation at Zalando (Berlin)

• Experimentation Lead Software engineer at Vista (remote EU)

• CRO Lead at New Balance (Amsterdam, the Netherlands

WIth so many companies investing in experimentation capabilities and building teams, it's a challenge to find the right people. Posting your open roles to the job board is a great way to get your amazing job in front of the right people. It also helps to support the Experimental Mind newsletter.

📅 Upcoming events

This is a running list of upcoming events:

• Accelerating Innovation with A/B Testing (Jan 31-Feb 10, online)

• ‍SUPERWEEK (Jan 31-Feb 4, Hungary)

• Quantum Leap (Feb 7-9, Austin, TX, US)

• MeasureCamp North America (Feb 26, online)

I've started to create the full overview of events for 2022, let me know which ones I should include.

💬 Quote of the week

"Without an experiment, our actions are based on hope instead of evidence." — Cassie Kozyrkov

😉 Fun of the week

We all know the Build Measure Learn loop. Unfortunately is often mostly Build, a little bit of measure and even less learning ... (via David Bland).

🙏 Thanks for reading

If you're enjoying the Experimental Mind newsletter, you can buy me a coffee or beer (in case you rather use Paypal, you can send it to 'kevin@experimentalmind.com'. Thanks to the 20 people who have already done it. Much appreciated.

Or simply share this newsletter with a friend or colleague (just like Luka did). Point them here to subscribe.

Have a great week — and keep experimenting.

Kevin