Experimental Mind: Learning, Netflix, Ethics, Principles, GA, Vaccines, Jobs...

Jan 17, 2022

🔎 What I've been reading last week

Many companies run too few experiments and learn ineffectively. Itamar Gilad explains the underlying causes and what you can do fix them.

Many companies that say they want to experiment, don’t really want to experiment. They want the mechanics, but are unwilling to adopt the underlying philosophy. They desire the benefits, without any of the (perceived) costs. They aspire to be modern and scientific, but cling on to old product development approaches.

This is blog post 6 of a 7 part series on how Netflix uses experimentation to run their business. I highly recommend you dive in this post (and the others).

The Dutch Association of Psychologists (Nederlands Instituut van Psychologen, NIP) has defined five simplified ethical principles for anyone involved in influencing and changing behaviour:

From policy makers to marketers, from advertisers to process designers. Sometimes influencing behaviour has the opposite effect. There are almost always side effects. Influencing behaviour is complex, with the risk that it is harmful to people and our society. These principles reduce the chance that you act unethically, or cause (unintentionally) negative mental and behavioural effects.

We talk to Craig Sullivan about the explosion of mass digital experimentation, the ethical implications, and the solutions we should adopt.

This could have big consequences. If one country in the EU comes to this conclusion, others will follow. Of course, Google quickly came with a statement.

In this post, Georgi Georgiev explains what practitioners of online experimentation can learn and generalize from the COVID-19 vaccine trials.

A full year has passed since results from the first clinical trials of COVID-19 vaccines became available. Vast swaths of follow-up observational data are now in circulation which enables some conclusions about the strengths and weaknesses of those trials.

🚀 Job opportunities

The job board now boosts 23 open roles.

And here are the latest featured roles:

🆕 Head of Product Economics & Experimentation at Zalando (Berlin)
🆕 Experimentation Lead Software engineer at Vista (remote EU)
CRO Lead at New Balance (Amsterdam, the Netherlands
[Dutch] Conversie Manager at Online Dialogue (Utrecht, Netherlands)
Principal Analyst, Experimentation at Vista (remote EU)
Senior Experimentation Strategist at Speero (remote)
Product Analytics & Data Lead at Stitch Money (remote NL/Cape Town)

Is your company hiring? Post your open roles to the job board. Readers of this newsletter can post for free, simply use the same email address.

📅 Upcoming events

This is a running list of upcoming events:

Accelerating Innovation with A/B Testing (Jan 31-Feb 10, online)
‍SUPERWEEK (Jan 31-Feb 4, Hungary)
Quantum Leap (Feb 7-9, Austin, TX, US)
MeasureCamp North America (Feb 26, online)

Do you know of an event that should be on this list? Let me know.

💬 Quote of the week

"Experimentation can't be solved bottom up. If the leadership of an organization doesn't care about scientific measurement, no one will." — Chad Sanderson

So many people are trying to get people to care about experimentation, without the necessary support from senior leaders in the company.

😉 Fun of the week

Tell Me Why (credit Courtney Dunn-Snede)

🙏 Thanks for reading

If you're enjoying the Experimental Mind newsletter, you can buy me a coffee or beer (in case you rather use Paypal, you can send it to 'kevin@experimentalmind.com'. Thanks to the 19 people who have already done it. Much appreciated.

Or simply share this email with a friend or colleague. Point them here to subscribe.

Have a great week — and keep experimenting.

Kevin

Experimental Mind