News Stay informed about the latest enterprise technology news and product updates.

Completeness of metadata

For a seemingly innocent description of data, the different types of metadata are seemingly endless.

This article originally appeared on the BeyeNETWORK.

At first, it seems like a good idea. What better way is there to start the day than to have a complete selection or library of metadata. But the deeper you go, the more you realize that there is no such thing as completeness of metadata.

Oh, sure. There are the familiar tables, attributes, and indexes. Everybody knows about them. But when you look closely, there are physical attributes, foreign keys, m:n relationships, 1:n relationships, and 1:1 relationships. Then, there are the number of rows in a table, the number of bytes in a row and variable length rows. And while you are at it, there are average and maximum row lengths, unused blocks and reserved row space. And there are date fields, encoded fields, and parent/child relationships. Not to mention networked relationships, inverted files, and sparse indexes. Don’t forget entities, relationships and subtypes. And what about nontraditional cardinality and definitions? Data item sets, anyone? Or mapping and transformation rules? Aggregation? Summarization? Summarization algorithms? And the list goes on.

For a seemingly innocent description of data, the different types of metadata are endless, or so it seems.

Then what happens when the size of data and metadata double overnight? And as if this weren’t enough, then there is the issue of the data changing. By the time you have a good solid definition of metadata, it has changed, and you have to go back to the beginning and start all over again.

So maybe this notion of a complete and accurate description of metadata is just a good theory. Maybe it is not possible to actually create a complete description of metadata.

Perhaps we should be looking at necessary metadata, not complete metadata. Maybe there is just too much metadata that keeps changing. Maybe people keep adding systems so that we will never be finished.

Instead of looking for and documenting ALL metadata, maybe it is simply rational to look for necessary metadata. By looking for necessary metadata, the task of gathering and managing metadata becomes doable. Looking for and documenting all metadata in a project – especially an ever-expanding project – is like a heat sink. It becomes a place where you can spend huge amounts of resources and huge amounts of time and never be finished.


Bill InmonBill Inmon

Bill is universally recognized as the father of the data warehouse. He has more than 36 years of database technology management experience and data warehouse design expertise. He has published more than 40 books and 1,000 articles on data warehousing and data management, and his books have been translated into nine languages. He is known globally for his data warehouse development seminars and has been a keynote speaker for many major computing associations. Bill can be reached at 303-681-6772.

Dig Deeper on MDM best practices

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.