Experimental Mind: Trustworthy Tests

😎 Summer Series

This is already the fifth edition of the Summer Series where we learn from experimenters how they are spending the summer time. Today we let John Ostrowski share his summer plans. Let’s go.

🔎 What I've been reading

Matteo Courthoud explains "What makes an observation unusual?"

Seventy percent of executives from digitally leading companies embrace an experimentation mindset.

eBay's experimentation platform runs hundreds of A/B tests on any given day. ... We discuss two automated test quality monitoring processes and methodologies, namely randomization validation using population stability index (PSI) and sample ratio mismatch (a.k.a. sample delta) detection using sequential analysis. The automated processes assist the experimentation platform to run high quality and trustworthy tests not only effectively on a large scale, but also efficiently by minimizing false positive monitoring alarms to experimenters.

Experiment Nation will be holding awards to name Experiment Nation's Best New Voice of 2022. Deadline October 14.

The nominee doesn't have to be new to the field, but they should be new to sharing their voice (via Social, Posts, Newsletters, videos, conferences, etc) within the past 2 years.

A new marketplace where speakers and event organizers find each other. Looking for a speaker about experimentation for example? Look no further. Also a great way for (potential) speakers to get discovered. I just created my one profile.

🚀 Job opportunities

Lead the experimentation program with a focus on evolving the current experimentation framework to increase test volume and velocity.

Check out the job board to find all other open roles. You can now also subscribe to @experimentationjobs on Instagram.

📅 Upcoming events

All upcoming events in the next months:

• 20-23 Sep: Nonprofit Innovation & Optimization Summit (Kansas)

• 20 Sep: Growthhackers Conference (virtual)

• 20-21 Sep: Marketing Insights Event (Utrecht, NL)

• 26-30 Sep: Experiment Nation Conference (virtual)

• 20-21 Oct: Conference on Digital Experimentation (Boston)

• 25-26 Oct: CXL Live (Austin, Texas)

See the full overview of events to find other events and conferences for you and your team.

💬 Quote of the week

"... we delivered over 50% growth in bookings and revenue from the prior year quarter, despite a challenging macroeconomic environment ... We see this growth as a result of our relentless focus on constant product improvement. Through continuous experimentation, we are driving organic growth, deeper user engagement, and stronger free-to-paid user conversion." — Luis von Ahn (CEO Duolingo)

🙏 Thank you for reading

If you’re enjoying the Experimental Mind newsletter please share this email with a friend or colleague. Looking for inspiration how to best do that? Take a look at what others are saying.

Have a great week — and keep experimenting.

Kevin