Forms, CRO Unbundling, Duolingo, Non-Inferiority, Hippo, Laura Forster, Tolerance...

Your overview of interesting reads, events and jobs for the experimental mind: #207

Mar 04, 2024

Hi folks!

Happy to share that Exatom joins as one of the partners of this newsletter:

Exatom is a Dutch/Belgian Form Analytics & Activation software provider that makes your existing web forms perform better. Exatom gives you microscopic understanding why visitors leave your web forms and how to fix it.
We are private-by-design, technology agnostic and very easy to deploy. Brands like Brooks Running, KIA Motors, LeasePlan and Delta Fiber use Exatom to optimize their online conversions. Learn more

➡️ Exatom form analytics boosts conversion rate for KIA Germany by 55%

Do you want to increase your form conversion rate? KIA was able to increase the form start, submit, autofill and resubmit rate. All with the help from Exaton.
Learn from this case study

Thanks to Convert, SiteSpect and Exatom for their support.

💡Three trends in the experimentation job market

Last week I shared some insights from two years of curating experimentation jobs. First of all there are three types of roles in the experimentation space: the Specialist, the Enabler and roles where experimentation is ‘simply’ part of the role.

I see three trends:

Unbundling of CRO / Experimentation Specialist role
With the growing adoption of experimentation, different tasks that normally only a CRO Specialist does, will be moved into other roles. CRO is not dead, it will be unbundled.
Less building. More integrating & driving adoption
With so many mature third party tools on the market, the role for experimentation enablers will be more on integrating these tools and less on building their own.
Experimentation ‘simply’ part of all roles
Experimentation skills will become tablestakes for people in growth, marketing, analytics, product management and engineering roles.

🔎 Interesting things you might have missed

How Duolingo runs experiments at scale

We’re on track to have our best year ever in terms of user growth, bookings, revenue, and profitability. As we’ve mentioned in previous shareholder letters, we have achieved this in large part because of our culture of experimentation and A/B testing. (Duolingo’s Q3 2023 Shareholder letter)

When the CEO writes that their growth is in large part because of experimentation in a letter to shareholders, it is interesting to learn how they approach experimentation. That brought me (again) to this article on Duolingo’s approach.

___

Beyond Superiority: Maximising Insights with Non-Inferiority Tests
Andrea Corvi shares where non-inferiority tests shine:

… some product changes follow a less strict path to production: even if a change isn't clearly superior, factors such as compliance, operational efficiency, or senior leadership's input can push for its implementation.

___

Dylan Lewis on the origin of the term HiP(P)O
In this interview Dylan (Head of Experimenatation at Atlassian) shares the origin story behind the term HiPPO (Highest Paid Person’s Opinion). Ronny Kohavi later documented it here.

💬 Laura Forster: My Experimentation Career Journey

Today’s interview is with Laura Forster, next to being Experimentation Specialist and Business Unit Lead at Up Reply, she is also an avid ultrarunner.

I’ve identified numerous parallels between physical running and managing an experimentation program. Both involve overcoming obstacles, adapting to sudden changes in conditions, and pushing personal limits to achieve success.
— Laura Forster

Read the full interview with Laura

🚀 Job opportunities

Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.

This week’s featured roles:

Conversion Rate Optimization (CRO) Specialist at ConversionWise (Remote, United Kingdom)
Analytics & CRO Manager at Svea Solar (Stockholm, Sweden)
Senior Experimentation Specialist at Preply (Barcelona, Spain)
Optimisation Lead at Just Eat Takeaway (London, United Kingdom)
Product Analytics Manager at 7-Eleven (Irving, USA)

Are you hiring? Submit here

Also, make sure to follow Experimentation Jobs on LinkedIn.

📅 Upcoming events

A running list of upcoming events.

9 Mar: MeasureCamp (Bologna, Italy)
19 Mar: Conversionboost (Copenhagen, Denmark
🏷️Use promo code ‘expMind50more’ for an extra 50 euro off.
23 Mar: MeasureCamp (Helsinki, Finland)
🎁8 Apr: Accelerating Innovation with A/B Testing with Ronny Kohavi (virtual, ~~$1,999~~ $1,499 USD)
13 Apr: MeasureCamp (New York, USA)
18 Apr: Emerce Conversion & Analytics (Amsterdam, NL)

Full list of experimentation conferences & events in 2024

👋 On June 19th I’ll be attending growth marketing SUMMIT, Europe's premier real life conference for digital growth, experimentation and optimization in Frankfurt, Germany. Use code ’experimental-mind‘ to get 10% off and let’s meet there.

💬 Quote to think about

Came across this one in a talk Jen-Hsun Huang gave at Stanford in 2011:

“Innovation requires a little bit of experimentation. Experimentation requires exploration. Exploration will result in failure. Unless you have a tolerance for failure you would never experiment. And if you don't ever experiment you would never innovate. If you don't innovate you don't succeed.”
— Jen-Hsun Huang, co-founder and CEO of NVIDIA (source)

😃 Something that made me smile

Is your company looking for a Product Marketing Manager? Meet Marta*.

Marta Puerto recently got laid off. Applying to jobs clearly did not work, so she made this great video to better showcase herself.

Experimental Mind

Discussion about this post