Behavioral Insights: Brick, Werner, CRO, Stats, Ethics, FDA, Skewed.
Hi there! You're receiving this email because you signed up for the weekly newsletter by Kevin Anderson at the intersection of experimentation, analytics and behavioral science. Thank you for being here.
Experimental Mind.
👋 Hi,
2021 is already halfway over. Can you believe it's already Q3?
in this week's newsletter another set of articles, especially selected for you. Enjoy the reads and your week.
1. Building a culture of experimentation 🧱
Every time I hear about a team that did their first experiment, I proactively reach out. I will jump on a call and start asking question.
✨ Who triggered this experiment?
❤️ How did you experience the setup?
🙁 What was difficult or unclear?
🚀 What is holding you back from doing more experiments?
It helps me to understand what the driving forces are in getting a team started on their experimentation journey.
And it's direct input for things to improve in the onboarding flow.
This is how you build a culture of experimentation. One brick at a time.
[I wrote this on LinkedIn. Any thoughts? Jump in the comments]
2. Amazon's Culture of Experimentation 🔥
Werner Vogel, CTO at Amazon talked about many interesting things in this 30 minute interview. I always pay extra attention to how they foster their culture of experimentation.
"Remember an experiment is not an experiment if you already know the outcome. There needs to be a certain level of uncertainty that goes with doing experiments so we want to make sure at all times that we have a pipeline of experiments lined up ... and we also remove every possible barrier for this experiments to go into this pipeline." Watch the quote
3. Experimentation experts answer 9 tough CRO questions ❓
I especially liked Georgi Georgiev answer to the question "What’s the key to an effective Conversion Strategy?": “Quality of the data, analysis, hypothesis generation, prioritization, and statistical testing are all crucial. Get one wrong, and the rest will function much worse than optimal.” Read the article
4. A/B testing statistics 🤔
By a beginner (Nitin Balodi) for the beginners. Read the article
5. Reversals in psychology 📜
Psychology has in recent years been racking up reversals: in fact only 40-65% of its classic social results were (some what) replicated. Read the overview of reverted studies
6. Ethics in experimentation ❤️
David Bland distills his advice on ethics in experimentation down into one guiding statement: “Experiment with your customers, not on your customers.” Read the article
7. We should test AI the way the FDA tests medicines 💊
According to Carissa Véliz in this HBR article: "We should submit new algorithms to randomized controlled trials, similar to those the FDA supervises when approving new drugs." Read the article
8. Dutch court case against Facebook 🧑⚖️
Dutch Data Privacy Foundation (DPS) and the Consumentenbond (consumer protection association) are taking Facebook to court for "... its rampant collection of internet users’ data — arguing the company does not have a proper legal basis for the processing". The case will serve in October. Read more
😀 Fun of the week
We're skewed ...
As always, if you're enjoying this newsletter, I'd love it if you shared it with a friend or two. You can send them here to sign up.
I try to make it one of the best emails you get each week, and I hope you're enjoying it.
And should you come across anything interesting this week, send it my way! I love finding new things to read through members of this newsletter.
Until next week, keep experimenting.
— Kevin