How are traditional data warehousing jobs being affected by all the attention that Hadoop and big data technologies are getting?
I think traditional data warehousing is a still good career path to pursue -- although big data is probably better. Extract, transform and load (ETL) work is already becoming commoditized, and more and more continues to be moved offshore. Also, reporting tools are becoming very business-user-friendly, and some don't require as much support from the business intelligence and data warehouse team as they used to. As a result, I do think traditional data warehousing jobs are going to evolve -- more so at larger companies, or companies whose business is very data-centric.
New tools, technologies and platforms enter the business intelligence and analytics space frequently. With data coming from so many more sources than 10 years ago, and in more of an unstructured fashion, organizations are going to use more big data technologies. Most of the new projects and efforts we see with larger companies involve some aspect of big data. The size of a company and the amount of data it processes will have an effect on how many big data technologies it implements. Smaller companies will probably hang on to traditional data warehouse architectures for some time.
We still see openings for traditional data warehousing jobs, and I suspect we will for the next few years. I wouldn't recommend just focusing on any one specific tool, without understanding the broader landscape. For example, make sure you learn and understand SQL if you are going the traditional data warehousing route. In the short term, being an expert in one specific technology can be lucrative -- however, new tools and technologies will come along to replace or, at the very least, improve a "hot technology."
The traditional data warehousing areas such as master data management, metadata management, data governance and data quality will continue to be mainstays in data warehousing. Many will say these areas become even more important with the increase in data volumes. The two camps when it comes to backgrounds for those seeking big data jobs are to take either the traditional data warehousing path, with ETL and SQL, or to climb up the Java development ranks. We see clients who have differing options. However, both camps will agree that it's necessary to make sure they are working with the appropriate data.
Hands-on experience is invaluable for data management jobs
Big data and data warehousing: A guide