This article originally appeared on the BeyeNETWORK.
In the early days of metadata, nearly everything was mainframe centric. At that time, the mainframe and centralized processing dominated the marketplace. The types of systems that were being built then were transaction processing, for the most part. These systems were built by the centralized data processing organization.
The metadata that existed in that day and age was all technical metadata.
The first corporate experiences with metadata were those that centered on the data dictionary. The data dictionary experiment collapsed for a variety of reasons including:
- the entire dictionary had to be built before the data dictionary was useful,
- the data dictionary was passive, not active,
- the maintenance of keeping the dictionary up to date was gruesome (and unrealistic),
- the technology supporting the data dictionary was old, and
- the people who knew what belonged in the data dictionary had moved on to other jobs or other companies.
As noble as the idea of a data dictionary was, the reality of its implementation was just plain ugly.
In a few years came the repository. In many regards, the repository was a definite step upward from the data dictionary. However, the repository was still a massive effort and required large funding. Given the lack of enthusiasm for the support of metadata, it is no surprise that building repositories was a slow and arduous process.
But the idea of a repository is still a very good one. At the heart of the repository is the tacit acknowledgement that there is a need for global management of metadata. If the organization is going to avoid the problem of the Tower of Babel, it is necessary to have a global repository of metadata.
The irony is that there is no shortage of metadata. There is metadata everywhere. There are directories, universes, ETL tools, DBMS directories and control parameters. It is just that the metadata is all local. Each vendor, from Oracle to DB2 to Teradata to Business Objects, has its own rendition of metadata. Saying that there is a need for metadata is like saying that there is a need for language – and saying that there are different renditions of metadata is like saying that Oracle speaks metadata in Chinese, DB2 speaks metadata in Russian, Teradata speaks metadata in French, and Business Objects speaks metadata in Swahili. There may be a language of metadata, but it certainly is not common. There really is a Tower of Babel when it comes to the vendor implementation of metadata.
The problem is that all the necessary things are done at the local – i.e., the vendor/product level. Metadata is created, used and maintained within the realm of a single product.
This leads to the insight that there really are two levels of metadata that are needed – local metadata and global metadata. Local metadata is metadata that is collected and used within the confines of the vendor’s product. Global metadata is where all vendors contribute to the enterprise understanding of metadata – where there is interchange among divisions and departments.