Please remember you don't necessarily "cleanse" all data that is determined to be less than par quality. Most of...
the time, you simply report on the data exception and find out your expected bounds are too tight or the exception was truly a business exception.
The best way I've found to determine strategically what data should be cleansed is to look at the business impact of not recognizing the less-than-ideal data quality and leaving it in place. If that business impact is greater than the effort to raise the awareness of the exception or cleanse the data, then the data certainly should be cleansed. Certainly there is some dirty data that will affect the business to a lesser degree than would the cost of fixing it. I have found that most data warehouse programs should take the important first step towards investigating data quality and establishing a data quality program: creating a framework for addressing quality violations.
Dig Deeper on Data quality techniques and best practices
Related Q&A from William McKnight
Are business intelligence certifications worth it? Get certification advice from our business intelligence expert, William McKnight. Continue Reading
There are loads of business intelligence tools out there on the market today. Our business intelligence expert provides some valuable resources for ... Continue Reading
Get examples of how data mining is used in vertical industries, such as retail, manufacturing, healthcare, financial and telecommunications. Continue Reading
Have a question for an expert?
Please add a title for your question
Get answers from a TechTarget expert on whatever's puzzling you.