Big data is well into the IT hype cycle -- to the point where even some vendors and consultants looking to capitalize on deployments of big data software and hardware are getting tired of the term. At the 2013 edition of the Pacific Northwest BI Summit, an annual gathering of about 20 vendor executives and technology consultants in Grants Pass, Ore., fun was had at the expense of the moniker big data during a session on the topic. For example, Shawn Rogers of consulting and research company Enterprise Management Associates said that the classic three V's definition -- volume, velocity and variety -- "has been beaten to death" at this point. "It just defines big data as an analyst hobby," Rogers said, to general approval.
But there are real big data projects going on out there, in lots of companies. And by now, efforts to deploy big data technologies such as Hadoop and NoSQL databases may be putting your organization through the ringer. If so, it's high time to rinse out some of the hype and look at big data management and analytics applications with a more considered eye.
There is value to be had; at least, that's the expectation. The Pacific Northwest BI Summit attendees weren't down on the potential benefits of using big data -- just on the term itself. And in a reader survey on business intelligence, analytics and data warehousing topics conducted earlier this year by TechTarget, publisher of the online magazine Business Information, interest levels in big data analytics were relatively high, and high-minded. Forty-one percent of 540 respondents said they had active programs or planned to add one in the next 12 months. And the primary goals of those respondents primarily revolved around using big data to drive new business: A combined 66% cited gaining competitive advantages, better understanding customers or increasing revenue. By comparison, 27% opted for improving organizational efficiency.
The three articles in our special issue of Business Information on deploying big data tools offer insight and advice to help point the way forward. First we look at the capabilities, and limitations, of Hadoop. Next we report on the relationship between big data and in-memory analytics tools -- and issues to consider before joining them at the hip. And we close with tips on making Hadoop work in corporate applications from a panel of IT and BI professionals who spoke at the Hadoop Summit 2013.