Experimental Mind: when the numbers do not add up

Now over to the newsletter with a selection of articles, jobs, events and quotes. Handpicked by me, so you don’t have to.

🔎 What I've been reading

Although, I don't think it's a good idea to track you experiments in Google Analytics (doing it properly is hard), you will find some good tips in this post about how to prevent data discrepancies between you analytics and experimentation tools: run A/A tests, use the same definitions in all your tools, create filters and watch out for those ad blockers.

Latest blog by my colleagues Joshua Tankard and Lukas Vermeer on how Vista is taking price change experiments beyond A/B and pre-post.

Paper by the Amazon Science team proposing methods how to deal with non-stationarity, such as the time-of-day effect and day-of-week effect.

Learn how to answer A/B testing questions and ace the next interview, with these prompts designed for product managers and data scientists.

Homerun is moving from a 5 to 4 day workweek, but runs a pilot first:

We're trying it out for nine months during which we’ll evaluate whether it’s truly improving the team’s well-being. We’re running dedicated 4DWW surveys every month with the whole team. If all goes to plan (even with the anticipated bumps in the road), our intention is to make the 4DWW permanent after the pilot.

Some observation by John Cutler on why so many teams are still in the feature factory mode. Does this sound familiar?

Utilization rates are so high that teams have barely a moment to experiment and respond to feedback. Even if they want to. It is a wicked cycle. At a certain point, just shipping prescriptive features is hard…which makes the problem even worse!

🚀 Job opportunities

I am working on a big improvement of the job board. Don't peek.

📅 Upcoming events

Join this great virtual event for free.

All upcoming events in the next months:

• ⏩ 25 June: MeasureCamp (Paris)

• 7 July: Experimentation Culture Awards (virtual)

• 4-6 Aug: Digital Elite Camp (Estonia)

• 22 Aug: Accelerating Innovation with A/B Testing (virtual)

See the full overview of events to find other events and conferences for you and your team.

💬 Quote of the week

"Experimentation can't be solved bottom up. If the leadership of an organization doesn't care about scientific measurement, no one will." — Chad Sanderson

😉 Fun of the week

You better be showing some insight, or else ... (from Timo Elliott).

☕ Thanks for reading

If you’re enjoying the Experimental Mind newsletter and want to say thank you? Buy me a coffee (or a beer). Or share this newsletter with a friend or colleague.

Have a great week — and keep experimenting.

Kevin