At the 2014 Pacific Northwest BI Summit in Grants Pass, Ore., consultant Colin White co-led a roundtable discussion on managing big data in the cloud. White, president and founder of BI Research in Ashland, Ore., kicked off the session by saying that the cloud presents IT managers looking to deploy big data platforms with the same data management issues as on-premises systems do. At the heart, he said, it's simply an alternative deployment vehicle -- nothing magical.
White elaborated on that point in a video interview recorded at the conference. He recommended that organizations weighing big data cloud deployments consider them "from an end-to-end viewpoint" before deciding to move forward. "We have the technology and, yes, we can get up and running faster [in the cloud]," White said. "But there's still a certain amount of work to do in setting those systems up."
For example, data integration work often needs to be done between different cloud platforms and between cloud and on-premises systems. A variety of startup vendors have introduced managed services that free users from having to deploy and manage cloud-based Hadoop clusters and NoSQL database systems themselves, White noted. But the services are relatively new, and most of the vendors still have small customer bases.
White said one of the primary uses he's seeing for big data systems in the public cloud is processing and managing data that's already outside the corporate firewall. That might include sensor data being captured from the Internet of Things, or data from external information services providers. "Companies are looking at the cloud really as a cost-effective way of accumulating that data," he said. Often, though, they then move filtered subsets of the data into on-premises BI and analytics systems instead of analyzing it in the cloud.
Watch the five-minute video to hear more of what White had to say about setting up and running cloud-based big data platforms.