Big data and master data management seem destined for a future together, but for now their relationship remains murky and unresolved.
At first blush, the two seem like a mismatched pair. Big data environments typically encompass large volumes of text and other forms of unstructured data from a variety of sources, such as social networks, Web server logs and sensors. Enterprise master data management (MDM) initiatives are meant to create a trusted source of highly structured transaction data throughout an organization.
But even with that apparent disconnect, data management analysts see a prospective link, with MDM processes having a role to play in aggregating useful information from pools of big data and then matching it against the master data in a company's transaction systems.
"The world of big data is a world of unknowns, and you need to somehow anchor it to the stuff you do know and trust -- that's the relationship between big data and master data," said Ted Friedman, an analyst at consulting and market research company Gartner Inc.
All eyes on the customers
The primary goal, according to Friedman and other analysts, is to provide a more complete view of customers for marketing, sales and customer service purposes. For example, customer service reps might be able to use a combination of big data and master data to reach out to customers to address complaints they've posted on Twitter or other social media sites. Marketing and sales teams could get a better idea of people's interests and affiliations to aid in the sales process.
It isn't a case of putting raw streams of social network data through an MDM filter with data consistency and governance in mind; trying to govern external data is clearly a dead end. The more likely approach is to mine sets of big data for valuable nuggets of information using sentiment analysis software and other analytics tools and then deploy MDM-fueled data matching capabilities against the results.
More on enterprise master data management
Read about MDM design and deployment options in this book excerpt
Learn about MDM best practices from expert Andy Hayler
See how MDM drove Scotiabank's mainframe modernization effort
"Applying MDM against raw, unstructured content is of questionable value," said Evan Levy, vice president of business consulting at analytics and data management software vendor SAS Institute Inc. "But distilling that content down to have some sort of meaning and then linking it to a specific customer -- that's where MDM can come into play."
Companies are starting to see -- or picture, at least -- a connection between enterprise MDM and big data, according to survey results published in November 2012 by The Information Difference Ltd., an MDM consulting and research company in London. Only 17% of the 209 corporate respondents in North America, Europe and Asia said they expected big data applications to generate new master data in their organizations. But 59% said they thought MDM hubs and big data systems could be linked together for business uses, including the ability to use master data to automatically detect customer names in sets of big data.
MDM and big data disconnects
Just because there appears to be a future for the two doesn't mean the pairing is free of complications. For one thing, big data applications often are developed with a disposable mindset, spun up quickly to support specific analytics tasks and then taken down. "The architecture and business practices for big data are ephemeral -- it just comes and goes pretty fast, and that goes against the grain of MDM," said Aaron Zornes, chief research officer at The MDM Institute, a consultancy in Burlingame, Calif.
Zornes also pointed to potential disconnects between the IT staffers on MDM teams and the data scientists and other analytics professionals who use big data. "The data scientists have been elevated by the press to rock star status, and culturally they don't like the rest of IT," he said. "They see IT as the old farts that they don't want to deal with."
In addition, it might be only a matter of time before enterprise master data management and big data technologies can be easily hooked up to one another -- but that time isn't here just yet, according to Kelle O'Neal, founder and managing partner at consultancy First San Francisco Partners.
"It's still fairly new, and everything is evolving and maturing," O'Neal said. "Vendors are creating ways to automate the process so [companies can] have better faith in social media data as it pertains to a customer or product. But for now, it's going to take some manual intervention."
Beth Stackpole is a freelance writer who has been covering the intersection of technology and business for more than 25 years.
This was first published in August 2013