This article originally appeared on the BeyeNETWORK.
Data warehousing has been around since the mid 1980s. Early data warehouses were created as a means of satisfying the information needs of the marketing, sales, management and accounting departments. This meant that integrated, historical and granular data was needed. With a data warehouse foundation, the data was flexible and able to be reshaped by different internal organizations without losing its reconcilability or integrity.
Today, in corporations and organizations doing large scale information processing, data warehousing is a standard part of the information infrastructure.
As data warehouses matured, it became apparent that the original design of data warehouses did not address some commonly occurring situations. These include:
- Data warehouses often held huge amounts of data.
- As data began to accumulate over time, there was a noticeable difference between the probability of access of data in different parts of the same table within the data warehouse.
- There was a need for unstructured data.
- There was a need for metadata as a standard part of the data warehouse infrastructure.
It is time to define the second generation of data warehousing. This article introduces DW2.0, an architecture for the next generation of data warehousing. (The full definition of this architecture is available. Click here for free access.)
Some of the new and interesting features of DW2.0 include:
- Acknowledgement of the different life stages of data within the data warehouse environment. Data is entered, edited, integrated and stored in the data warehouse; ages in a near-line sector; and then is archived.
- Acknowledgement of the need to bring both structured and unstructured data into the data warehouse. DW2.0 acknowledges that both structured data and unstructured data belong in a data warehouse and that a bridge must be built between the two environments for integrated decision making.
- Metadata is an essential part of the data warehouse environment. Local and enterprise metadata, and subtypes of metadata – business and technical – are intimately and interactively related in the DW2.0 environment.
Other important differences between first- and second-generation data warehouses as well as more information about the architecture for the next generation of data warehousing – DW2.0 – can be found at http://www.inmoncif.com/.