Experimental Mind: stop confusing correlation with causation

Nov 15, 2021

🔎 What I've been reading this week

We must learn to analyze data and assess causal claims — a skill that is increasingly important for business and government leaders. One way to accomplish this is by emphasizing the value of experiments in organizations.

Ben Labay shared his framework for defining and differentiating between CRO, CXO, and experimentation:

"... this can be our industry’s new language. So whether you’re in marketing, product, sales, or ‘growth’, or you’re the CEO or COO, you have a common language, understanding, and expectations relevant to the flavor of the discipline you implement."

In this blogpost the Microsoft experimentation team explains several signs of data quality issues in A/B testing and how to monitor those.

Master the art of A/B testing with Ronny Kohavi. He will be hosting a series of 5 two hour long sessions, especially for executives who want to make their organizations data-driven.

Ronny Kohavi roasts Optimizely's (and Adobe) documentation. They are misinterpreting p-values.

🚀 Job opportunities

Are you looking for a new opportunity? These are this weeks featured jobs:

Product Manager, Growth & Retention at Vista (remote)
Product Manager, Experimentation at Adyen (Amsterdam, Netherlands)
Full Stack Developer Experimentation at Booking.com (Amsterdam, NL)

Or have a look at the 16 other open roles on the job board.

Are you hiring? Post your open roles to the job board. Readers of this newsletters can post for free, by using this coupon code at the last step: SUBSCRIBER200

📅 Upcoming events

This is a running list of upcoming events:

CJAM Session with Dana di Tomaso & Collin Tate Crowell (Nov 18)
The Conference formerly known as Conversion Hotel (Nov 19-21, NL)
CJAM Session with Nir Eyal & Hazjier Pourkhalkhali (Nov 23, online)
Design The Next Step (Nov 25, 2021)
🆕 Panel session on Behavioural Data Science (Nov 30, online)
CJAM Session with Terry Pierelli & Erin Weigel (Dec 2, online)
🆕 Etsy Experimentation Virtual Event (Dec 7-8, online)

💬 Quote of the week

"You never change things by ﬁghting the existing reality. To change something, build a new model that makes the existing model obsolete." — R. Buckminster Fuller

😉 Fun of the week

Visit Iceland mocks Mark Zuckerberg about his Metaverse and presents the 'Icelandverse':

The Icelandverse is unlike any other open-world experience with “-verse” in name, because it’s real. Plus, you don’t need a funny-looking VR headset.

🔁 Any feedback?

Don't forget to share your feedback, it takes less than a minute.

🙏 Thanks for reading

If you're enjoying the Experimental Mind newsletter you can buy me a coffee or beer. Thanks to the 10 people who already did it. Much appreciated.

And share this email with a friend or colleague. I would love this community of experimenters to grow. You can point them here to subscribe.

Have a great week — and keep experimenting.

Kevin

Experimental Mind