What trends or big changes are you seeing this year with data warehousing and database management system (DBMS) technology?
A good (and challenging!) question.
- The continuing rise of the data warehouse appliance is one of the big data warehouse trends. Many companies have complex transactional systems (human resources, finance, etc.). A data warehouse appliance can provide rapid analytics based on the data in these systems. There often is no effort to integrate such data with that in other transactional systems, but data warehouse appliances do offer a rapid solution to a real problem that many companies face.
- Of even greater interest is the increasing attention being paid to semi-structured data. This is essentially a no-brainer: We’re all becoming interested in social media data, but semi-structured data is found in a great number of other forms – x-ray images in hospitals, emails in business, etc. We need to think about how we store such content economically. We may well see a trend to manage the storage and querying of semi-structured data with technologies such as Hadoop and MapReduce, with the metadata produced by them being stored and manipulated in more conventional technologies.
This was first published in May 2011