Discussion in conferences, webinars and articles about the future of market research often highlights the implications for research agencies and the efficiencies generated by new technology. These efficiencies include automating and standardising research, agile research, VR and AI applications. All of which enables us to do research projects faster and cheaper.
Another phenomenon which can drive speed is to use what we already know. This has led to the enormous rise of Insight Engines like Market Logic, Lucy, Sharpr and Stravito. External sources are used for this. The focus is on traditional data providers like Nielsen Retail Panel, GfK household panel, Euromonitor, Mintel etc. Over the last years, we’ve seen several Data Marketplaces emerge. In these Marketplaces enormous amount of external data sources is available.
We’ll explain how these external sources can be exploited for consumer understanding and what it means for the capability of consumer insights departments.
Data Marketplaces are two-sided platforms intended to match data sellers with buyers and to facilitate and manage data exchanges and transactions. Data Marketplaces have a general purpose. And that is to trade any kind of data. This is different to more niche Data Marketplaces which focus on certain industries or certain types of data. Recently Personal Data Marketplaces have been popping up, which allow consumers to get paid for sharing their data on a consent-managed basis.
Overall, Data Marketplaces enable easy access to third-party data sources and let organisations monetise their own data assets.
As noted above, a lot of external sources are available. So why is the consumer insights department not exploiting these sources to understand consumers and generate actionable insights as much as they could? Especially at a time when interview fatigue is increasing. Wouldn’t it be better to limit interviewing to qualitative deep dives and use as many existing data sources as possible?
Several reasons can be given:
But we think the main reason external data sources aren’t used as much as they should be is that their usage requires creative thinking as solutions for business issues. It requires, on the one side, out-of-the-box thinking and, on the other side, the capability to make connections to business situations. Two examples can highlight this:
Both examples need a different thinking framework. This capability is, in general, underestimated. Especially because when people see the solutions, they think that it’s logical and simple. But to come up with these solutions requires a capability which is hardly experienced amongst Consumer Insights Managers – being able to judge the quality of external sources. This is vital as e external sources might not cover the full picture.
Typically, a Data Marketplace charges a commission fee for the service of connecting the provider to the buyer. Buyers can pay in several ways like one-off data purchase with no rights to updates to the dataset, ongoing data subscription and a usage-based data license.
To attract traffic, Data Marketplaces offer free data sources. This is open-source data made available by authorities or public institutions or also outdated samples of their products. This means that buyers can play around with them and get to know how useful the whole data product could be for their purposes before buying it.
Buyers receive data in several ways. This can be via an API, or through a specific software or software platform as well Dataset bulk download.
The Landscape is scattered and complex. It requires expertise to find your way in this data jungle!
Companies can appoint a so-called Data Scout. A Data Scout understands the business issues but also has a Consumer Insights Manager track record. They’re knowledgeable and can assess whether external sources contain sufficient information for the business issue or if (additional) market research is still needed. They also understand Data Science. This means she can steer data scientists for specific projects and able to translate the results into business recommendations. Data Scouts are capable of developing relationships with Data Marketplaces and know how to connect external-data streams with other data sources while being an out-of-the-box thinker, as explained at the beginning of this article.
We recommend that companies start to experiment with how to harness information from external sources. Companies can start with a small project that requires external data and involve one external dedicated agency led by a Consumer Insights Manager with a data scientist. It will create awareness and knowledge of Data Marketplaces in the company and for which kind of solutions these can be exploited. It will also allow companies to evaluate the need and benefit of appointing a Data Scout on a structural basis. Companies taking the lead in harnessing external sources will be able to react faster but also obtain a richer and more cost-effective Consumer Understanding.