[Summer Series] Leihua Ye: My Experimentation Career Journey
Your overview of interesting reads, events and jobs for the experimental mind.
Hi, I just returned from my time in the Scottish Highlands. The newsletter will stay in holiday mode, but I can share this great story by Leihua Ye (Senior Data Scientist Experimentation at Walmart) about his career in experimentation so far.
*Uncover what makes your CX win or lose customers
Get the latest insights into EMEA customer preferences, pain points, and industry-specific trends, enabling you to make data-driven product decisions. Discover how experience loyalty is overtaking customer loyalty. Download the survey findings now!
💡Leihua Ye: My Experimentation Career Journey
The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Leihua Ye shares his journey. You can follow Leihua on LinkedIn, his blog and YouTube channel.
My name is Leihua Ye, and I enjoy nature, outdoor hiking, traveling, and reading good books. Besides, I hold a PhD degree from the University of California, Santa Barbara.
A fun fact about my educational background, I have degrees in three disciplines, including Humanity, Social Science, and STEM. Also, I obtained these degrees in three countries: China, the UK, and the US.
What is your current experimentation role and what do you do?
Currently, I’m a Senior Data Scientist at the Experimentation Team (called Expo) for Walmart. As the Fortune 1 company, Walmart has a huge need for trustworthy test results. Expo administers and delivers high-quality A/B testing data points for decision-making.
I wear two hats:
On the technical side, I provide technical support and integrate best practices in various fields – Big Data, Data Science, Causal Inference, and Reinforcement Learning – into our platform.
On the culture side, I’m an experimentation evangelist who preaches the value of experimentation.
Running tests is easy; running trustworthy tests is hard!
Experimentation is such an interdisciplinary field that requires domain knowledge from various subjects. For example, Online Experiments typically have millions of users, a typical big data setup. Doing any type of big data calculation and obtaining the results timely present a unique challenge to both Data Scientists and Data Engineers. As a Senior Data Scientist, I need to validate the statistical rigor and provide alternatives if the method does not work. This is just one simple example of the interdisciplinary nature of A/B testing. There are so many more use cases that need unique solutions.
As a Senior DS, I need to ensure the methodology is the “right” choice for the problem statement. The reason for the quotation marks is that being right is not equal to being fancy or perfect. There are so many new methodologies coming out every year, but only very few of them provide practical value. Putting on the DS hat, my thought process starts with the business problem and then moves back to the technical arsenal, trying to identify a good enough solution. It’s a fun process but not easy.
Referencing Microsoft’s Flywheel Theory (Fabijan et al., 2021), I see experimentation as a dynamic state: in order to make the flywheel spin, we need a strong culture of experimentation to facilitate collaboration, change how key decisions are made, and address concerns. Technical support alone is insufficient, and we need someone who understands customer pain points and helps them understand the value of experimentation.
How did you enter the experimentation space? What was your first experimentation related role?
I received a ton of training in quantitative methodology from my PhD program. While trying to locate an industry job after graduation, I applied for different quantitative positions, including UX researcher, Quantitative Financial Analyst, Machine Learning Researcher, and Data Science. Unfortunately, this strategy was too generic and broad and did not work out well. So, I re-examined my strongest strengths and identified the experimentation space as the best fit, considering my background in Experimentation and Causal Inference. After deciding the direction, I spent thousands of hours reading and researching how different companies build up their experimentation platforms. Up until now, reading has become my daily routine.
This is my first experimentation related job in the industry, but I used a lot of statistical and causal inference methods in my PhD years. Landing the first industry job is particularly challenging for PhD candidates. The available number of positions is limited, but the candidate pool is large. My initial strategy was not effective and struggled for a while. Later on, I found a few strategies that helped me land the first job:
Create content online and build up your network. I spent hundreds of hours reading/researching technical articles/blog posts and talking to other DS in the field.
Practice your interview skills. Like any other skills, interview skills are obtainable after deliberate practice. Technical interviews have several components, and practice each component extensively.
Interviewing is a game of numbers: success rate = 1 offer / the total number of interviews. You only need 1 offer but need dozens of interview opportunities to succeed.
How did you start to learn experimentation?
My PhD program is a great start. By its completion, I have been using the same statistical tools for 5+ years. Also, reading blog posts/research papers/talking to other experimenters help me build a very solid foundation in the field. It’s like Learning by Reading.
After joining Expo, I got the opportunity to be more hands-on and implement a lot of cool methods. It’s kind of like Learning by Doing.
In the full interview you will also learn what Leihua does to keep up with the ever-changing industry. He also shares his recommendations to someone who is looking to join the experimentation industry.
🚀 Job opportunities
Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week's featured roles:
🆕 Growth Analyst at Indicia (Madrid/Barcelona, Spain)
🆕 Sr Product Manager, Experimentation Analytics at Optimizely (EU)
🆕 Engineering Manager Experimentation at Spotify (Netherlands)
🆕 Head of Experimentation at United Nations (Maputo, Mozambique)
🆕 Senior Data Analyst – Experimentation at BBC (United Kingdom)
Are you hiring? Submit here
👍 Thanks for reading
If you are enjoying the Experimental Mind newsletter please share this email with a friend or colleague. Looking for inspiration how to best do that? Take a look at what others are saying. Have a great week — and keep experimenting.