Business Information

Technology insights for the data-driven enterprise


Manage Learn to apply best practices and optimize your operations.

Internal big data skills trump outside consulting help

Big data skills are in short supply in many companies. But some IT managers say it’s more fruitful to develop them in-house than to rely on outside help.

Big data projects rely on advanced skill sets that are in short supply in many organizations. That might be a big opening for consultants offering outside help. For example, research organization Wikibon estimates that worldwide professional services revenue from big data deployments will vault from $3.87 billion last year to $15.38 billion in 2017.

But ensuring that your organization has ample big data talent is important to the success of big data applications, said Neeraj Kumar, vice president of enterprise architecture at Cardinal Health, a distributor of pharmaceuticals and medical products in Dublin, Ohio, that's in the early stages of working with Hadoop. "Innovation does not happen via consultants," he said. "It happens when you have skin in the game, when you have people working with [the data] who understand the business and understand the problems."

Kumar sees bringing in consultants to help train internal staffers on big data management and analytics technologies as a valid way to get started on projects. After that, though, "you need people in-house who can take the project forward," he said.

That's the approach taken by Inc. Paddy Hannon, its vice president of architecture, said the online publisher of automobile pricing data and other car-shopping information hasn't used outside consultants on a Hadoop deployment that began in 2011. Instead, a team of Java developers learned how to use a slew of new open source technologies to drive the Santa Monica, Calif., company's efforts to tap Hadoop and related big data tools as an alternative to an existing relational data warehouse that was running out of steam on scalability.

In a June blog post, Edmunds Chief Information Officer Philip Potloff wrote that the early months of the initiative were slow going, as team members mastered the intricacies of the Hadoop Distributed File System, MapReduce, HBase, Oozie, Hive and Pig -- all tools in the Hadoop ecosystem. But he said the company made the switch to the new Hadoop-based data warehouse system last February and now is supporting all of its reporting operations with information from the new setup.

Article 8 of 11

Dig Deeper on Data quality techniques and best practices

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

As per the current market, Big data is the demand of the hour. It is expected to increase five fold times in the upcoming years up till 2017. The big data management and the analytics needs to be managed in an effective way as to take care of the huge amount of data. So, software’s are really needed for managing data.

Get More Business Information

Access to all of our back issues View All