Joshua Resnick - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Enterprise Hadoop promises to bring big changes

Long-time IT watcher Tom Davenport sees potential in the latest big data go-around, with enterprise Hadoop part of the mix.

Hadoop has made plenty of headlines in recent years, but some people are still looking for more signs that enterprise Hadoop has happened. That odd juxtaposition of Hadoop hopes and enterprise pragmatism is not lost on Tom Davenport.

"I am the token East Coast representative," he joked to the westward-leaning crowd as he took the stage recently at Hadoop Summit 2014. He noted that while he was wearing jeans, he also had on a sports jacket.

It is useful to talk about the cultural divide that exists between online powerhouses from Silicon Valley and the rest of the business world, he told the Hadoop crowd, because the two camps can gain a better understanding of each other's strengths and weaknesses.

He described his own path from skeptic to believer. He said his thinking on Hadoop and big data changed as he worked on a recent book. While lamenting the issues with the term big data, the author and Babson College IT management professor said there is truly something new afoot, and its opportunities reach beyond a few Web players.

"There is a continuous, rapid flow of data in many companies today," he said. Enterprise leaders would do well to study it, and to prepare for a new era of faster decision-making.

The process of bringing Hadoop and big data to the enterprise will require a change in thinking, however. Davenport said today's big data is less about creating reports for senior execs and more about creating new services for customers.

Big Data @ Work

While working on Big Data @ Work: Dispelling the Myths, Uncovering the Opportunities, Davenport learned that organizations have to be ready to work with the new tools. Big data will change managers' jobs and usher in new approaches to analytics, he told SearchDataManagement.

Big Data @ Work cover

The book features case studies of big data usage at Internet upstarts such as LinkedIn, Kyruus and Recorded Future. Additionally, it takes a look at big data trends in long-established companies like UPS, Macy's, Bank of America, Procter & Gamble and Citigroup.

Such organizations are processing big data workloads to lower costs and reach better and faster decisions, but they increasingly have an eye on new data-based products and services. In Big Data @ Work, Davenport catalogs industries that could be substantially transformed:

  • Every industry that moves things;
  • Every industry that sells to consumers;
  • Every industry that employs machinery;
  • Every industry that sells or uses content;
  • Every industry that provides service;
  • Every industry that has physical facilities; and
  • Every industry that involves money.

That covers a lot of ground, no doubt. Clearly, it is not just about big volumes of data. Davenport is reluctant to use the words big data because the term doesn’t aptly describe the wide variability of the data that is now being created in organizations, on the Web and via the Internet of Things.

As well, enterprise Hadoop is not always in the mix. But it often is. "For me, Hadoop is becoming an umbrella term, a sort of shorthand for a whole set of open source software tools to work on data," Davenport said.

Small data meets big data

According to Davenport, a handful of established firms take the best of traditional small data analytics and put them together with new big data analytics, which in his book, he refers to as "Analytics 3.0."

What makes this new -- and worthy of the Analytics 3.0 banner -- is the unusual speed of data gathering and decision making that we now encounter. Machine learning can facilitate better decision making and help reduce problems, which is why Davenport expects it to become more essential in the future.

The nature of business management is changing right along with the tools of data management, making it a time of substantial change. Enterprise Hadoop may call for changes in the way people do business and make decisions, but many are just getting around to thinking about the big data paradigm.

Davenport's analysis suggests that enterprise managers should be the ones thinking, learning and preparing to move forward on it the most, or they may risk obsolescence.

Jack Vaughan is SearchDataManagement's news and site editor. Email him at, and follow us on Twitter: @sDataManagement.

Next Steps

See where big data architecture is going

Read more about Hadoop Summit 2014

Go behind big data's hype with Tom Davenport

Dig Deeper on Hadoop framework

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.