Xaxis AI Marketplace Case Study
Xaxis provides media buying artificial intelligence (AI) technology for B2B clients. In 2021, Xaxis submitted a project proposal for a marketplace concept. The marketplace is an e-commerce B2B business network and exchange of custom digital algorithmic solutions that are used to measure the success of media campaigns.
Project Background
Summary Details
- My Role: Lead UX Researcher and Recruiter
- Research Type: Remote Generative Research (Discovery)
- Company: Xaxis (now known as GroupM Nexus)
- Company Type: B2B Corporate
- Industry: Media Company, Artificial Intelligence
- Team: Head of Product, Product Manager and
2 UX/UI Designers - Tools: FigJam, Delve, Airtable, Microsoft Teams, Google Slides, Screencastify, YouTube
💣 The Problem
Both Sides
Business Problem: Xaxis lost money and revenue because they were not able to deliver custom solutions requested by their clients when it was in demand.
User Problem: User requests for custom algorithm solutions were not being met because Xaxis didn't have the technical capability in the existing web platform, Copilot. This led to unhappy clients.
💡 Xaxis Project Proposal
To attempt to resolve the business and user problems mentioned, Xaxis pitched a project proposal for a digial marketplace concept to the parent company, WPP.
The marketplace was to enable Xaxis to meet client requests for custom alogrithm campaign solutions that they were limited or unable to do within the current web platform, Copilot. The marketplace would include a self-maker tool section for their data scientists to have the technology to create custom algorithms more efficiently, replacing the burdensome hacks in Copilot that they were dealing with at the time.
Another reason for the marketplace was to launch a digital business networking community between buyers and sellers (the algorithm makers). The marketplace proposal concept was thought to provide business exposure for seller algorithm providers while providing the buyers with their custom campaign needs. (Mutually beneficial)
WPP approved the project proposal and gave Xaxis 💵 $8 million💰 to fund the project in 2021!
The Dilemma 1 Year Later
The Marketplace project was put on hold due to the pandemic and 1 year had passed since Xaxis won the budget money.
By 2022, Xaxis had to decide between two options below:
My Role
Where I fit in the Project
As the only UX researcher, my role for this project was to be like a detective 🕵🏾♀️ to investigate whether the Marketplace concept was still relevant, wanted or needed by users the 1 year after the project proposal.
As a contractor, Xaxis is my client for this project. (customer) My research aimed to collect information to help Xaxis make a major company strategic decision for the million-dollar budget and to help Xaxis' user base.
Do users still need the marketplace?
Does a marketplace solve this user problem?
Project Challenges and Constraints
What I faced when starting the project
- Time and budget constraints.
- Technical jargon and unknown industry language.
- Vagueness, confusion and gaps of information.
- Multiple delays for scheduling research sessions because so many people were on vacation or out on PTO.
- Required going through layers of contacts during recruitment to finally reach the needed research participant.
🤔 "What is the marketplace for?"
🤔 "What is a programmatic algorithm solution in the media buying industry?"
🤔 "What is a DSP?"
More questions in the process section belowProject Process
My Approach
Info Gathering for Clarity
Before conducting user research, it was crucial to get context and clarity to understand the complexity behind the marketplace concept. For clarity, I initiated workshop and team brainstorming sessions with the project lead, the Head of Product, the Product Manager, and the designers.
Create a Research Plan
The problem and the users were vague at the start of the project. The Marketplace was also a concept that was not live in the market because it was in the exploratory stage so I decided to aim for Discovery Research (Generative). I created a research plan with objectives, methods and goals to help strategize and roadmap for the project.
Conduct Research
I recruited and interviewed international participants that would be buyers or sellers in the marketplace. My research was also an introduction for Xaxis to understand their users and personas.
Research Report
My research was presented to all top level executives, including Head of Product. My user research recommendations directly impacted the product strategy and business.
Information Gathering Stage
Clarity and Context
To successfully recruit the right participants and use the correct research methods, I first needed to ask specific questions to collect essential information that was missing at the start of the project. I led an early discovery workshop and brainstorming sessions with the project team to get clarity and context.
At the start of the project, I needed to know 🤔:- Who are the users of the Marketplace?
- What problem are we trying to solve with the Marketplace? (Both the User and Business side)
- How does each user type interact with each other in the marketplace? (This question helps to connect the dots for the whole picture)
FigJam Key Information Gathering Sessions with the Head of Product, Associate Director and two UX/UI Designers
FigJam Discovery Team Workshop Brainstorming (Problem Statement, Assumptions Mapping Exercise and Business Goals) with the Head of Product, Product Manager and two UX/UI Designers.
The identified user types below ⬇️ based on team assumptions at the start of the project.
I created the ⬇️ diagram below as a result of meeting with the Head of Product to help 🔨 break down the complexity of the marketplace concept and to understand how each user interacts in the marketplace.
I also used this diagram as an introduction to help new research participants understand what the Marketplace was and how they fit in it so they could answers questions about it.
Research Planning Stage
Research Methodology
Why Methods Were Selected
- Contextual inquiry was chosen to directly observe how data scientists create algorithms and use software tools, providing deeper insights into their workflows and pain points.
- The recorded contextual inquiry provided visuals to help the design team brainstorm a Marketplace UI to replace the algorithm workaround hacks that data scientists struggled with.
- It also served as an educational tool for the product team to understand data scientists' workflows, needs, and pain points.
- User interviews were chosen for research participants who did not need to share their screens to demonstrate their workflow.
- User interviews provided valuable insights into users' wants, needs, work pain points, and whether the marketplace addressed their challenges.
- Early Concept testing was selected because I wanted to test the provided design prototype during the early stage to see if was intuitive to users and to collect their feedback.
🎤 📹 Conducting Research Stage
Originally, the product team thought there were 6 different user types. However, user research 💡 revealed that were actually 4 users types after talking to real users during research because the Account Services user type combines multiple roles into one.
As the solo researcher, I conducted 9 recorded research sessions on Microsoft Teams. Most sessions were user interviews and a few were contextual inquiries.
6 Sellers
👨💻 6 Data Scientists
on all corporate levels
3 Buyers
👩💼 2 Traders
👨💼 1 Account Services
9 Users Total in Qualitative Research
I did work with some colleagues to identify people to interview or contacts that would know who to interview because the company is HUGE. Then I directly reached out to them and scheduled sessions.
I requested that the Data Scientists that specifically created custom algorithm code to prepare a demo of their workflow during the contextual inquiry session so that I could see their current tools and painpoints to determine how the product team could possibly improve or resolve their issues through the Marketplace.
The contextual inquiry allowed me to emphathize with data scientists (seller user types) because I was able to directly see the current tedious limitations and workarounds they had to endure to manually create custom code.
🔎 Data Analysis and Synthesis Summary
UX Deliverables
I produced UX Deliverables based on the qualitative research data.
- Foundational Research Slide Deck - To educate the organization about the benefits of UX Research.
- 📄 Marketplace Research Report - Note: The report file cannot be shared to protect private information at the client's request. However, anonymized research snippets are included in the results below.
- Personas - Were crafted based on insights from real research participants I interviewed.
Personas
Data Scientist: The seller who creates and sells algorithm products within the marketplace.
Account Services: The buyer who orders algorithm products on behalf of clients within the marketplace.
Trader: Another buyer type who gets approval from the media planner
🔑 Key Insights and Findings
My strategy was to ask targeted questions that would uncover specific information to determine whether the marketplace remained relevant and necessary for users during my research sessions.
For example, my interview question: 🤔"How often do your clients request custom algorithm solutions now?"
Buyer User Response
Account Services persona
To be honest, that had come up a lot more in the past than it does right now. We had some clients in the past that really liked to get fancy with the way they were measuring success.
Unfortunately because of the economy and COVID, a lot of those conversations had to take a bit of a back seat because test budgets aren't as freely given to us when the economy is where it is, and you know they're not as willing to test a new tactic to test a new strategy.
VP of Account Services
(Media Planner Buyer User Type)
Seller User Response
Context: For Sellers that will make the algorthm product for the Marketplace, I asked about specific work processes that were relevant. I wanted to know whether the Marketplace could resolve any existing workflow pain points that were a result of the current restrictions and limitations in Copilot because the Marketplace was supposed to be the upgraded code work environment.🤔 Interview Question: When you output the data back into the DSP for optimization, what are some of your biggest challenges in the current process right now?
" There is a lot of manual work that needs to be done for custom code because of how the URL UTM parameters are quoted will vary for each different software client.
We cannot test more than two algorithms at the same time. Even though it's copy and pasting for model building, there is still some manual work that is being done. "
Data Scientists in Singapore
"I don't have any or many pain points at all. If we build anything custom, we can always rely on custom tree and it's not like we need to update it every day. There's already an API for that in Xander anyway.
So basically because there's an API for everything, the manual work that we have to do is is pretty minimal and certainly from my side.
Data Scientists in London, UK
🗒️ Data Interpretation Conclusion: The workflow pain points depends on how the data scientist individually works and what tools they decide to use. What is a problem for one user, is not a problem for the other user so the Marketplace might not be a solution for ALL data scientists.
Results and Impact
The Outcome
Based on my research insights, the Xaxis leadership team decided to pivot away from the marketplace project and use the million-dollar budget for better use.
💵 $7 Million Saved! 🎉
A Slack message from the Head of Product who requested that I
obfuscate sensitive data.
Reasons why leadership decided not to move forward with the Marketplace concept:
-
The buyer user type in the research revealed that there was no longer any demand for custom algorithm solutions from their clients.
The Marketplace no longer had business value because clients were no longer requesting it. Leadership felt that it wasn't worth the risk of wasting both time and money to build a complex E-commerce marketplace.
- The Xaxis 🧑🏽💻 engineering team began developing 💡 another solution in a different project that resolved the workflow issues that a data scientist mentioned during research for the marketplace. Leadership felt this confirmed a lack of need and value for the sellers because their technical pain points will be resolved without the marketplace.
Impact
⭐ Business Financial Impact: I saved Xaxis $7 million by deprioritizing the proposed Marketplace solution via my research.
⭐ Organization and Operation Impact: I established a rigorous research process for my team to understand a research-first approach to evaluate a product concept and to learn about their users.
Reflections
💡 What I Learned
- I was amazed to learn how research insights can drive significant impact by guiding critical business decisions, even when the project concept doesn't launch or move forward. This experience taught me that my research is about empowering informed decisions. Thanks to my insights, Xaxis can now invest those millions into building solutions their users truly need and want.
- This project was also a reminder of how powerful asking the right research questions can be to collect the needed insights.
What I Would Do Differently
- If I had more time, I would have aimed for an equal amount of participants for both user types, sellers and buyers for better balance.
- I would have done more research methods. My research plan had more methods in the other stages, but I was cut short for time and only got to do stage 1 with the research methods mentioned.