ktsdesign - Fotolia

Ten years on, master data management software market comes of age

It has been 10 years since the master data management industry emerged. MDM technology is maturing, and project success rates are on the rise -- but there's still a long way to go.

By my reckoning, the master data management (MDM) industry is 10 years old this year. The term MDM started to come into common parlance in July 2004 after the launch of SAP MDM. A few early pioneers, such as Data Foundations, DWL and Kalido, had introduced master data management software before then, but mid-2004 is when the MDM market as such really came into being.

MDM was born of the realization that the ERP movement of the 1990s had profited some software vendors -- as well as the systems integrators trying to implement ERP systems -- but it hadn't solved the underlying problem of inconsistent master data across enterprises. In corporations of all sizes, multiple applications were using their own versions of customer, product, location, asset and other data that, in principle, should be a shared common asset. Even after ERP systems were implemented, other business applications -- customer relationship management, supply chain, human resources and marketing systems, for example -- still held their own master data. In addition, ERP systems usually had multiple separate instances. Recently, a major oil company I had worked with admitted to having more than 600 significant applications, just one of which was SAP, its "standard" ERP system.

MDM tools and processes promised to improve the situation by bringing key shared master data together in a central hub, either by physically copying the data or by providing pointers back to transaction systems. The goal was to have a golden copy of data that was consistent across the enterprise and that everyone could rely on. As often is the case with technology trends, we saw some initial excitement. Following SAP's lead, other giant vendors began muscling in on the act. IBM bought DWL, Oracle introduced a series of products, and Microsoft purchased Stratature, a small MDM vendor.

More master data to manage

Along with more vendors entering the market came a dawning realization that customer and product data wasn't all that needed to be managed. Asset data, chart of account data, personnel data, marketing data, supplier and location information -- it all needed to be sorted out, too. After a brief tussle, the multi-domain MDM tag largely won the argument, and vendors of all stripes began claiming that you could store any kind of data in their MDM hubs, whether or not the hubs were actually designed for that purpose.

In reality, different types of data have different demands. Customer data is relatively simple in structure, but in a consumer-oriented business, it can be huge in volume; address duplication is a nightmare problem that has only been dented by an industry of data quality tools. Product data, although typically lower in volume, is much more complex, with elaborate hierarchies to be managed. And the core master data is often stored in barely structured text files, all of which need software for parsing and classification.

Despite the inherent issues, MDM vendors and systems integrators began selling the idea of master data as a major infrastructure component for linking shared data across business units and geographies, and across many data domains. Such projects, ambitious in scale for global corporations, often floundered, due not only to technical issues, but also to political ones. Business managers didn't like to change the way they were doing things; they were happy in principle to standardize, as long as that meant everyone else adopted whatever they were currently doing. In response to some high-profile MDM project failures, the concept of data governance was born, or at least rebadged. Companies were encouraged to set up committees of business users who would be responsible for setting standards on such things as international product codes and global account structures, and to assign data stewards to help identify data issues and ensure that users adhere to the internal standards.

In the meantime, data integration vendors decided they wanted some of the MDM action, and a further round of acquisitions occurred: Tibco bought Velosel, Informatica bought Siperian, Software AG bought Data Foundations. Some vendors made multiple purchases: IBM bought Trigo and Initiate in addition to DWL, and Informatica added Heiler to its MDM software lineup. More startups, such as Orchestra Networks, also continued appearing. Some vendors, such as VisionWare with its focus on local government, specialized in a specific vertical market segment. Others, like Stibo with its eye on product data, stayed focused on a particular data domain.

MDM project success rates on the rise

So, what does the master data management software market look like today? The advent of data governance has improved success rates on MDM projects. Surveys conducted by The Information Difference, my consulting company, showed that by mid-2011, 82% of MDM initiatives were rated by participants as successful, compared to 25% in 2008. Also, the proportion reporting that MDM was broadly established in their organization almost doubled, to 23%. However, the sheer scale of the problem to be tackled could be seen in the unchanging average number of systems generating master data. According to the surveys, in 2008, the median number of systems generating master data was 15; by mid-2013, that hadn't changed.

During the past year, the MDM software industry became a billion-dollar business. (Systems integration associated with MDM is four times that figure.) Despite all the acquisitions, new vendors are still appearing. Recent examples include Pitney Bowes, Semarchy, InfoTrellis and Verdantis. Clearly, the MDM industry is far from mature. Recent discussions I've had suggest a significant upturn of interest in MDM, driven by ERP consolidation projects that haven't consolidated that much, and by compliance and risk management initiatives in some industries.

So happy 10th birthday to you, MDM. You've moved past infancy and begun to mature. There's still a long way to go, but you're growing up, and you have a promising future as you approach your teenage years.

About the author:
Andy Hayler is co-founder and CEO of London-based consulting company The Information Difference Ltd. and a frequent keynote speaker at conferences on master data management, data governance, and data quality. He also reviews restaurants and blogs about food on the website
Andy Hayler's Restaurant Guide.

Email us at [email protected] and follow us on Twitter: @sDataManagement.

Next Steps

Get tips from consultant Anne Marie Smith on priming the MDM project pump

Find out why MDM programs need a business focus to succeed

Read our guide to building and managing an effective data governance program

Dig Deeper on MDM best practices