Experimental Mind: Mind the gap

🔎 What I've been reading last week

A well, researched blog post by Davis Treybig on the gap between companies who truly get experimentation, and the rest.

... most companies can not, despite their best efforts, realistically achieve an experimentation program anywhere close to what the aforementioned companies have.

If you read one article this week, let it be this one.

In Ronny Kohavi's latest A/B testing class, three agencies were asked to present their views on the different tools. If you are looking into buying a new tool or building (parts of) an experimentation platform, you should definitely read the slides that were shared.

Build vs. Buy is a false dichotomy. You may buy some components and build others into a hybrid.

Merle van den Akker created this list of online resources to learn more about behavioural science. This is a follow-up of her books & papers post.

Blog post about Duolingo's experimentation tool for copy tests.

... there was a big payoff once the changes were made and the tool was built — within the first three months with Expurrimenter, we more than doubled our all-time volume of copy tests.

From one of my favorite blogs: Ness Labs.

Social proof can lead us to act in ways that do not align with our beliefs, and that morally may not be defensible. Learning when and how to use social proof to make decisions will allow you to benefit from the wisdom of others, without encountering the associated pitfalls.

Often misunderstood concept, so good reminder:

P-value is a probability value corresponding to the likelihood of obtaining a data value (‘test statistic’), which is at least as “extreme” as the actually observed data value (observed ‘test statistic’), under the assumption that Null Hypothesis is correct.

P-value provides an answer to: Assuming no-difference (i.e. Null Hypothesis), what is the likelihood of seeing the difference we are observing in the data.

Post by Rory Sutherland linking direct marketing, behavioral science and the importance looking at people's behavior.

David Ogilvy used to say that every copywriter should start off by working four years in direct marketing before they move into general advertising - because in direct marketing you find out what works.

Direct marketing is basically a vast series of behavioral science experiments performed at scale.

In every circumstance, we must not assume that what we think we know or is always true - we should start by asking, how do people behave?

Joe Goodman created this Experimentation Journal to keep track of all those personal experiments in Notion. Get your free copy here.

Ronny Kohavi reviewed the book 'Feature Management with LaunchDarkly', including what he liked and didn't like.

🚀 Job opportunities

DUTCH: Wil jij grote digitale platformen zoals Videoland, RTL Nieuws of Buienradar helpen optimaliseren? RTL is op zoek naar jou.

All featured roles:

• 🆕 Digital Analyst at RTL (Hilversum, NL)

• 🆕 CRO Specialist at RTL (Hilversum, NL)

• Product Analytics Team Lead at Wolt (Helsinki/Berlin)

• Experimentation Lead Software engineer at Vista (remote EU)

• Principal Analyst Experimentation at Vista (remote EU)

• Data Scientist Experimentation at Just East (London)

• Performance Marketing Specialist at Picnic (Amsterdam)

• Digital Experimentation Manager at Philips (Amsterdam)

• Head of Product Experimentation at Zalando (Berlin)

• CRO Lead at New Balance (Amsterdam, the Netherlands

Or checkout the other 28 open roles.

Hiring? Post your open roles to the job board (from €0 for you)

📅 Upcoming events

This is a running list of upcoming events:

• 🆕 Data Mishaps Night (Feb 24, virtual)

• MeasureCamp North America (Feb 26, virtual)

• Conversionboost (Mar 15, Copenhagen)

• Emerce Conversion (Apr 21, Amsterdam)

• Experimentation Elite (Apr 27, London)

See the full overview of events to find 15+ other events and conferences for you and your team.

💬 Quote of the week

From 100 ways to make your working life a little easier:

"Adopt an experimental mindset that aims to try new things in the pursuit of learning.” — James Chudley

😎 Cool thing of the week

In 2021, Cees Bassa's camera took a photo of the Dutch sky every 15 seconds.

This image combines the 2.1 million images into a year-long keogram, showing the length of the night change during the year, the motion of the Sun and Moon, and lots and lots of clouds.

Cees Bassa @cgbassa

During 2021, my #allsky camera took an photo of the sky from the Netherlands every 15 seconds. This image combines the 2.1 million images into a year-long keogram, showing the length of the night change during the year, the motion of the Sun and Moon, and lots and lots of clouds. https://t.co/F1map5tBpX

🥑 Thanks for reading

If you’re enjoying the Experimental Mind newsletter, buy me a coffee, beer or an avocado. Do you prefer to use Paypal? Send it to ‘kevin@experimentalmind.com’. Thanks to the 20+ people who have already done it. Much appreciated.

Or share this newsletter with a friend or colleague (just like Luka and Siddhartha did). You can point people to: getrevue.co/profile/kevinanderson

Have a great week — and keep experimenting.

Kevin

ps. this is a tool I recently discovered 👇

💌 A message from The Sample

The Sample is a newsletter discovery service. After you sign up, they pick a different newsletter to forward you each day. If you like it, you can hit "subscribe in 1 click". Great idea, great service.