The master data management industry has been around for nearly 15 years. Analysts bandy about wildly different...
figures as to its size, but it's certainly a multibillion-dollar industry with a compound annual growth rate of at least 9%. What's interesting is how the market is developing and how that fits into general models of industry growth.
Like other fields, master data management software went through an initial excitement phase that was marked by dreamy marketing promises and the frantic acquisition of pioneers by the industry giants.
Then there was a stage when the reality dawned that getting large companies to change the way they managed shared data was hard, depending as it did as much on modifying human behavior as solving technology issues. Getting a large enterprise to agree on a single golden version of customer or product data despite a plethora of competing requirements and internal political rivalries proved an elusive goal.
I witnessed some wildly ambitious projects sink under their own weight when they tried to impose global standardization of master data management (MDM) on business units that weren't ready for it, or that were downright hostile to the idea due to their entrenched ways of working.
Master data management software's new phase
This cold reality of disillusionment with master data was, to an extent, rescued by the rise of data governance, whereby businesses came to consider ownership of data a corporate asset rather than a task foisted on them by corporate IT departments. This development neatly complemented MDM technology and enabled it to be deployed more successfully, as the initiatives were now aligned with the business side.
Today's master data projects are often smaller in scope, but they are still driven by business need, such as handling multichannel retailing or governance-driven tasks in the finance and pharmaceutical industries. This change has seen the birth of new master data management software vendors, often focused either on a particular vertical sector or on specific business problems, such as improved customer experiences or the need to handle customer privacy in the light of tougher regulations.
Recent developments in MDM technology, from both new and established vendors, have been partly innovative and partly reactive. For example, just about every vendor now offers cloud deployment in one form or another as companies slowly but steadily move more of their processing to cloud platforms.
A tougher nut to crack has been adapting to the rise of data lakes, many of which resemble "Hotel California" of the 1977 Eagles song, where data can check in any time it likes but never leave. Putting some order into this mess may involve reindexing data from the lakes into data hubs, and potentially deduplicating data in these hubs so it is in a state from which it can be queried successfully.
Linking such hubs up to existing data warehouses via a hub of hubs may sound familiar. Indeed, it might sound remarkably like an MDM problem. In this way, master data solutions can potentially offer a lifeline to enterprises, enabling them to climb out of the murky data lakes that they are often stuck in and start to get some actual value from them.
The current state of MDM
Modern master data management software often offers more elaborate features, such as graph databases, to analyze the extended relationships data can have. In the B2C world, this may involve looking at not just a specific customer, but also at the relationship that customer has to others -- e.g., family members.
In the B2B world, this involves ensuring that you have a picture of not just a particular supplier or trading counterparty, but also where that company sits within a web of linked companies. Assessing the risk of trading with a particular company is all very well, but if it turns out to have a parent company with an entirely different name, then you need to factor that into your counterparty risk models.
Plenty of companies discovered this, to their detriment, when Enron folded spectacularly in 2001, taking with it a series of not obviously linked companies. Mapping the web of dependencies, whether in corporate hierarchies or with consumers, is a problem well-suited to modern master data software.
The MDM industry continues to evolve to handle new business problems, as well as underlying infrastructure changes such as cloud computing and big data. It has matured beyond the phase of wild-eyed excitement when master data management software was going to solve all enterprise data management challenges in boil the ocean, grandiose projects.
MDM is now moving to a more modest and sustainable phase in which it is applied to tackle more specific, immediate business challenges supported by data governance processes that are driven by business leaders rather than technologists. It is a welcome transition, and one in which the focus is shifting to delivering business value rather than putting in a new layer of corporate technical infrastructure.