There has always been tension in the data management world between operational and analytical data. And the tension...
is increasing along with the number of data sources being tapped by organizations, as evidenced by the growing complexity of master data management processes -- particularly ones involving customer data.
Learning how to categorize operational data for analytical consistency is the realm of master data management (MDM). More than a decade ago, software tools emerged to serve as hubs for product, supplier, location and customer MDM initiatives aimed at establishing some order over the data generated by different transaction processing systems.
Such systems are all about processing business transactions, such as sales orders. As companies automated their operational processes, each system worked fine in isolation. However, a transaction carries with it lots of linked contextual data.
If you go into a supermarket and buy a can of cola, a considerable amount of associated data is produced -- for example, which product was purchased, what it cost, whether it was on special offer and if it was bought with cash or a card.
A marketer wants to see the abstract level of data above the individual transactions: Which products are selling better in individual stores and regions, and at what time of year? Are promotions working? How sensitive to price are the customers in different locations? The answers to such questions have a big impact on marketing and sales strategies.
For example, a particular loaf of bread at my local Sainsbury's supermarket costs exactly £1, but the identical loaf at the more convenient Sainsbury's Local convenience store at the end of my road costs £1.40, despite these stores being just six-tenths of a mile apart. Given the razor-thin margins under which grocers operate, being able to charge 40% more for the identical product in a slightly different location is clearly a valuable thing to understand.
MDM's job: Inspire confidence in data
However, in order to accurately answer broad questions about customer behavior, marketers must have confidence in the quality of data and how it's categorized. For example, similar products need to be categorized in the same way so true comparisons can be made.
A can of cola may be classified as a carbonated drink by one regional operating unit. If another unit, with its own transaction processing system, classifies cola in another way -- labeling it as fizzy, say -- that makes it hard for someone in the central office to see the overall picture of carbonated drink sales.
In MDM hubs, data drawn from multiple transaction systems is aggregated, and any inconsistencies are removed. In the cola example, carbonated and fizzy would be matched as being equivalent. This higher quality and validated master data, a so-called golden copy, enables meaningful data analysis to be done without confusion.
It has become conventional wisdom that this approach -- aggregating master data from its various source systems and blending and validating it in order to produce a clean copy -- is the way to go for customer MDM programs and the like. Recently, though, that view of the MDM world has come under strain.
Achieving the desired master data nirvana has always been hard, but with good data governance processes and enough effort and political determination, it could be done. At least, it could be done provided the organization was actually in control of the master data in question. That generally was the case in the early days of MDM, as the main master data for an enterprise could usually be found in its ERP, supply chain and customer relationship management systems.
However, things have become more complicated. In addition to third-party reference data from companies like Dun & Bradstreet or Experian, companies now have to handle a myriad of new data sources that may have master data buried within them.
In the case of a customer MDM effort, relevant data on customer interactions may be captured via phone, email, an e-commerce website, web chats, surveys or even in person. Customers may also post their opinions of a company and its brands on external websites or social networks -- sometimes forcefully, as United Airlines recently discovered.
Master data gets harder to pin down
This plethora of new data sources means that master data is no longer easily corralled within a corporate MDM hub. Customer data, in particular, lurks in new places -- some outside the firewall -- and yet, it still needs to be monitored and managed.
Even internally, data lakes now exist alongside many enterprise data warehouses, but making sense of all the data pooled under their surface can be tough. It's hard to tie a social media handle back to a customer number, and techniques like machine learning and new forms of pattern recognition may be required in order to extract meaning from the murky waters of a data lake.
Somehow, the disguised master data within these new sources needs to be matched to the golden copy master data so analytics teams can make more accurate assessments of customer sentiment, propensity to churn and customer lifetime value -- to name three key metrics enabled by effective customer MDM. New software is emerging in this area, but there's also an opportunity for existing MDM vendors to adapt their tools to be able to interpret such data and amalgamate it with what they manage now.
This flood of new operational data has the potential to turn master data management on its head as enterprises work to update their MDM strategies and systems to accommodate the incoming information. But it's worth the struggle: Organizations that successfully navigate the increased MDM complexity will have a significant competitive advantage over their business rivals.
Steps to take in setting up an MDM program, from consultant Anne Marie Smith
Mobile apps increase the need to jointly manage metadata and master data
Read a book excerpt on identifying key master data across an organization