Editors Joe Hebert and Ed Burns discuss the gulf between Hadoop hype and adoption in this BizApps Today video report on the 2014 TDWI Executive Summit and World Conference in Boston -- the first in a regular series of videos that will be produced by SearchDataManagement and sister sites on news and trends related to business applications, information management and application development.
Going beyond the hype and getting real about big data in general and Hadoop in particular was a big discussion topic at the TDWI event. Philip Russom, an analyst at The Data Warehousing Institute, which organized the conference, said viable Hadoop use cases are most common "if you want to do the data lake thing, where you're just hurling lots of data into it." But, he added, TDWI surveys still show Hadoop adoption rates in the single or low double digits. That maps to the results of a new TechTarget Inc. survey in which 13% of 901 respondents said their organizations had fully or partially deployed Hadoop clusters, while another 12% said systems were under development.
Many attendees, primarily from healthcare, financial services and insurance companies, said they're still trying to figure out how to use the open source distributed processing framework in their data management and analytics environments. "The challenge we're finding with Hadoop is really not so much with the technology," said Ryan Fenner, an enterprise data solutions architect at MUFG Union Bank in San Francisco. "It's finding the appropriate use case." And that's a must before deciding to deploy Hadoop, said Rick Sherman, founder of consultancy Athena IT Solutions: "Just don't do it to do it -- do it because there's a reason."
Burns, who covered the conference, told BizApps Today host Hebert that Hadoop is best suited for processing large data sets -- but for traditional enterprise uses, he added, "a lot of businesses just don't have that much data to justify it." Watch the three-minute video to hear more analysis from Burns on what's holding back enterprise Hadoop applications and deployments.