Behavioral Insights: why people do not A/B test 💬

Behavioral Insights.

👋 Hi!

Welcome to week 8 and another Behavioral insights newsletter. I have selected 6 articles*|INTERESTED:Would you also like insights in Dutch?:Yes please / ja graag|* and some job opportunities*|END:INTERESTED|* to share with you.

1. Reasons why people do not A/B test 💬
Ronny Kohavi wrote down the reasons he often hears for why other people think running controlled experiments in software should be avoided:

1. "hurts agility"

2. "leads to incrementalism"

3. "leads to local minima; we need to do big launches"

4. "no one is looking at the big picture"

5. "Steve Jobs didn’t A/B test. Shakespeare and Picasso didn’t A/B test"

6. "Focus groups are enough to understand the value of ideas"

Read the article and find out why these reasons are wrong.

2. Online RCT's: lessons and extensions to medicine
This paper gives an overview of key scaling lessons learned in the technology field around online A/B testing and translate them to the field of medicine. Read the paper

3. What is an underpowered A/B test?
Georgi Georgiev explains the confusions, myths and reality around the concept of statistical power. Read the article

4. Chrome Extension flags SRM issues
A Sample Ratio Mismatch (SRM) test can be used to detect a wide variety of data quality issues around A/B tests. Lukas Vermeer made a Chrome Extension that automatically flags SRM issues in your own A/B testing tool. Read more

5. Naming conventions for digital analytics data
Article from Stefan Koning on the importance of having a standardized naming convention in place. "Digital analysts should not have to plough through all filters and tag management hacks before they can get started for what they are hired for: analyse!" Read the article

6. How to build an experimentation team
Jonny Longden explains the skills needed to build an experimentation function and buckets them in three categories. Read the article

*|INTERESTED:Would you also like insights in Dutch?:Yes please / ja graag|*Job opportunities in The Netherlands:

• VodafoneZiggo: CRO-Specialist / Webanalist

• ING: Technical Consultant

• NRC: Digitaal Analist

• Bijenkorf: Online Conversion Specialist

• ING: Technical Webanalist

• Ahold: Lead e-Commerce Analytics*|END:INTERESTED|*

Subscribe to a curated list of my favorite podcasts on Spotify or any other podcast player.

Have a great week!
— Kevin