Now that collective awareness of the potential value of Big Data has risen, fueled by Artificial Intelligence, teams working on these types of products are gaining momentum, and many have integrated Product Managers (PMs). Why this choice, and how have PMs adapted to this specific field ?
Data is everywhere : whether it's used to improve decision-making, automate processes, or even become a new lever for monetization, this topic has far exceeded the confines of IT to become a key element in corporate strategy. Some say that the advent of ChatGPT was a catalyst. It brought to the forefront an entity that was quietly bringing about a revolution behind the scenes, particularly with the development of Machine Learning algorithms. Understanding the business significance of these areas, organizations needed to properly reinforce themselves and saw in Product Management the missing piece allowing them to unlock value potential that was untapped until then.
🤔 Why Product Management has become essential in the data field
Having a Product Manager (PM) to manage one's data product is no longer an option. They address three major issues.
#1 A need to connect data with Business
Many organizations have seen Agility and then Product Management break down the existing silos between Tech and Business to create better products.
A similar tearing down of barriers has gradually become necessary between data and Business. The root of this need is a common problem : two entities that do not speak the same language, that do not share the same objectives, and, lastly, that do not move forward together. The Business side always wants more, and to keep going faster. On the other hand, data is focused on product development. It is subject to constraints (both capacity and technical ones) and thus becomes a hindrance to the ambitions of the Business.
Faced with this situation, the PM plays a crucial role as a mediator to bring these two sides together and unite everyone around a common roadmap, both ambitious and realistic. Marc Skwarski, Thiga PM, experienced this at Invivo, a large agriculture group : "Some do not understand the complexity inherent in new Data subjects. As a result, I spend a lot of my time evangelizing, simplifying, and providing visibility on progress."
Similarly, Pauline Minary, a data PM at Leboncoin, recently discussed the importance of evangelizing stakeholders : "Uncertainty is truly a crucial element, which makes a real difference with a traditional product. With Machine Learning, you cannot know before you start how well it will work or what performance will be achieved. It's necessary to share this reality with stakeholders and insist on the idea that Machine Learning is not magic with 100% accuracy."
Having a PM capable of playing this evangelist role is key for efficient organization, as they are seen as a facilitator able to communicate with both parties and therefore make informed decisions.
#2 The urgency of meeting the innovation challenge
With the arrival of ChatGPT, AI was on everyone's lips when defining their 2024 roadmap. Although interest in Big Data and Machine Learning is not new, ChatGPT has built real hype.
For strategic reasons, the interest of the Business for these topics is increasing, and there's brainstorming everywhere to find the right use case. Victor Billette de Villemeur, former Data PM at L'Oréal, shares his point of view on the current situation : "The trendy nature of data - and especially AI - leads sponsors to launch projects just because it looks good to shareholders or the hierarchy (when it's not a marketing element as well). Having a PM is a way to focus once again on value and always start from a need to be addressed, while finding with their teams the best way to do it." Spoiler alert : AI is not always the solution.
Yet, one must acknowledge that AI and Big Data are powerful drivers of innovation. And who better than Product Managers to lead these initiatives ? Julien Ngboda, Lead PM at Thiga, testifies to the contribution of Product Management to Decathlon's data Factory : "The data Factory used to be seen as a service center where business units would typically go to fetch information or express a need. The challenge for the PMs who joined this data Factory was to bring proactivity in identifying current and future needs." These needs might involve evolving a dashboard or an existing product, but they can also lead to the creation of a new product.
One of the revolutions we are currently experiencing is primarily about the accessibility of these models, especially the Generative AI ones. There's no longer a need for armies of data scientists to integrate features with AI. This glass ceiling, which has been shattered, has fostered innovation, and countless organizations have created their own hackathons to find a practical case related to their business where AI would be a catalyst.
#3 A capacity to drive value
The primary mission of Product Managers is to maximize the value delivered by the product. Their ability to make decisions based on impact and allocated effort is crucial in a field like data, where it's easy to get lost in the quest for the perfect model.
"The hardest thing for Data PMs is to decide whether to go into production or not. Traditional PMs just have to test... It's quite binary."
Zacharie Treister, coach at Thiga
Marc tells us about his experience at Invivo : "Upon arriving, my number one challenge was to simplify things to deliver the expected results as quickly as possible. Technical profiles tend to complicate everything and aim for perfection. We, as PMs, come in with a ROI-focused vision and a Test & Learn culture."
“Should I stop or continue ?” Here's the kind of issues that might seem trivial for traditional PMs, but are much more engaging and not as straightforward to handle in the data domain. Zacharie Treister, a coach at Thiga who has worked on data products at Deezer and Qwant, testifies : “The hardest thing for Data PMs is to decide whether to put into production or not. Traditional PMs just have to test... It's quite binary. Data PMs have to define performance thresholds for the algorithm. From what threshold do we consider the result to be so incorrect that it becomes harmful ? From what threshold do we consider having a truly beneficial result ? And what do we do in the gray area ? For instance, if I have a 65% accurate result but it saves me 2 hours a day, even if I manually check the incorrect outcomes, it might still be worth it.”
These questions of "performance potential" and "achievable potential for improvement" are recurring issues that almost never fade away. Like a savvy investor operating with limited development capabilities and budget, PMs must fight against the trap of over-quality... And in the data realm, the temptation is very strong.
⚒️ How the PM adapts to the data field
I have never been a Data PM, but I have asked myself : "What to do when entering such a context and what are the job's specificities?" Today, thanks to insights from my fellow Data PMs, I identify three main differences.
#1 A Discovery Process Almost Exclusively Based on Data
Applied to data products, the fundamental principles of discovery and Design Thinking remain relevant. Marc shares the inception of a topic addressed with Machine Learning at SeLoger : "We tackled the issue of a workaround that some professionals use to boost their listings for free. Our first mission was to check the extent of the problem using data. The numbers were astonishing ! We concluded that there was a real problem to address."
Haoitif Dellaoui, a Thiga PM currently assigned to La Centrale on the automatic fraud detection tool, shares her approach to continuously identifying new improvement opportunities : "Every month, together with the customer service team, we review the scams that occurred as well as those falsely detected. We try to see if there is a possibility for model evolution and if there are any management rules that could be applied. Sometimes, we realize that the issue stems from the interface and the way data is entered, which can both be misleading. If we only focused on our scope, we would just be putting a bandage on a wooden leg."
Whether you’re seeking improvements like Haoitif or tackling a big issue like Marc, what clearly changes is the way discovery is conducted. "A large part of your discovery is done through data, and not with user interviews as it is more traditionally done," Zacharie explains.
Having identified the problem is good, but is it possible to have a model that fulfills this need ? To answer this question, two variables must be considered: on one hand, the algorithm itself and its potential for scalability; on the other hand, the quantity and quality of the data. Regarding the available data, it's one of the elements that is important to dwell on in discovery. Zacharie can attest to their importance : "At Jellysmack, the business came with a problem. On paper, it was simple. Then we looked at the available data and realized we didn't have the right ones. We asked the business to start by recording the necessary information. Discovery is also used to validate these prerequisites before moving on to delivery."
What also changes is the involvement of Data Scientists and Data Analysts. Given the importance of data in the discovery phase and their expertise in the matter, they are almost as involved and crucial in the discovery phase as they are in the delivery phase.
#2 Working with Different Profiles
Unless you have a background as a Data Scientist, Data Engineer, or Data Analyst, adapting to a data team requires a period of adjustment. Marc Skwarski, PM at Thiga, recollects : "In the beginning, I used to listen, to ask questions, and whenever I heard a term I didn't know, my first instinct was to look up its definition on the internet."
"Data is no exception : to succeed, one must return to the very essence of Agility and Lean by placing humans at the heart of the interactions between PMs and their team.”
Haoitif Dellaoui, Thiga PM
On their part, PMs must also bring a new culture and perspective to data teams. Victor tells what he has observed : "Since they tend to have more of a 'researcher' profile, some can lack pragmatism and economic rationality. It's up to PMs to bring this aspect to them."
Often motivated by technical challenges, PMs sometimes need to temper their enthusiasm with a constant focus on ROI (Return On Investment). What’s a classic example of this ? Believing that AI is the solution to all problems. The first instinct of a PM and a data team should be to consider whether it's possible to develop a management rule (more efficient and quicker to implement) to address the desired use case before resorting to AI.
In this quest for mutual understanding, it's important for everyone to make an effort to meet halfway. Haoitif shares : "Data is no exception : to succeed, one must return to the very essence of Agility and Lean methods by placing the human element at the heart of interactions between PMs and their team. By understanding the complexity of each other's roles, personal interests, and worlds, and by being willing to adapt for valid reasons the way we operate, a truly winning dynamic can be created."
#3 A deeply uncertain ROI
Working on data products implies operating over a longer period without being certain of the outcome, which can be frustrating for PMs.
Releasing into production every day or every two weeks is utopian (for now), except in "fine-tuning" phases of a model. Developing an algorithm and training a model takes time, but this doesn't preclude adopting an incremental approach. For Marc, "there is an intention to break down topics as finely as possible to try to control complexity. Extremely useful for traditional products, this becomes crucial in fields like data." Falling into a tunnel effect is a risk that can highlight the value of structured methodologies like Scrum or Shape Up, or even SAFe.
As for the result, it's always difficult to predict in advance. Zacharie tells us more : "With front-end development, the duration can vary but we know we will get a result. With a Machine Learning algorithm, regardless of the time spent, the result may sometimes be unattainable for various reasons (data quality and quantity, algorithms, etc.)."
Reducing this uncertainty that goes along with the industrialization of a model involves adopting a "Test & Learn" approach, which is dear to PMs. Marc shares how it went with his product at SeLoger : "We started with one type of property, then trained our model and tested it in one geographic area. The approach that was conducted before applying the model to all our listings was truly an iterative and incremental one."
Given their inherently high uncertainty, data science topics should be approached as R&D subjects. Forcing a rhythm can help reduce this uncertainty while allowing to be agile and to readjust plans. Charles De Gaulle said, "Researchers who search are easily found. It's the researchers who find that we're still searching for.." Let's ensure that this iterativity can lead all data teams to fall into the latter category, thanks to PMs.
In conclusion, products based on Big Data or Machine Learning have demonstrated the value of Product Management, which brings a ROI-focused vision and creates cohesion with the business. To achieve this, Product Management has adapted to a domain where expertise and uncertainty reign. We are now at a point in history where the demand for Data PMs is skyrocketing, with new challenges like the integration of Generative AI that will require even more adaptation but offer great opportunities for innovation. The spotlight is on you. Ladies and Gentlemen Data PMs, inspire us !
To learn more about our job: download our book Agile Product Management
