Home > Ask the Data management / BI Experts > Data governance and quality Questions & Answers > Data quality management for data warehouses
Ask The Data Management Expert: Questions & Answers
EMAIL THIS

Data quality management for data warehouses

David Loshin EXPERT RESPONSE FROM: David Loshin

Pose a Question
Other Data Management Categories
Meet all Data Management Experts
Become an Expert for this site


Enterprise IT tips and expert advice
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google


>
QUESTION POSED ON: 10 June 2008
We are discussing the option of populating our data warehouse tables with less than quality data. We need to make recommendations to the (business project) sponsors. We need to balance the risk of credibility of having bad data with gaining knowledge of how bad things are by having people access the information. What would you recommend -- populate with bad data or not populate at all?

>
This is a little bit of a conundrum. If I understand the question correctly, you have two challenges. The first is that your team is not ready to populate the data warehouse because the quality of the data is not acceptable for the defined business objectives. The second is that you are unable to effectively articulate to the business users the degree to which the data does not meet data quality thresholds. You also have positioned the challenge in terms of "risk" -- risking credibility (presumably of the data warehouse).

By looking at these aspects, though, I suspect that there are some deeper issues involved. For example, by whose criteria are you determining the quality of the data? If the data warehouse team is making that assessment with respect to the needs of the business project sponsors, that implies that they have been engaged in defining the business data expectations, requirements, critical dimensions of data quality, measures and metrics for monitoring the quality of the data. In that case, there whould be no risk of credibility, since (1) the data loaded into the warehouse is not "owned" by the data warehouse team, but is just collected from other sources upstream and then determined to not meet the objectively defined standards, and (2), the business sponsors would be the ones who have contributed their data quality expectations, and those rules are not "owned" by the data warehouse team either. And, in that case, you do not have to answer the question about whether the data warehouse should be populated -- that should be up to the business project sponsors, since the levels of quality would already be reported to them based on their own set of criteria.

However, this is clearly not the fact, which means a few things. First, it is not clear what the definition is in this context for data quality acceptability, but it is not derived or owned by the business users. Second, it appears that there may be an opportunity to improve the data requirements analysis process in a way to capture the data quality dimensions, rules and metrics in a way that allows you to characterize the candidate data sources for suitability prior to data warehouse population time. Third, there is an opportunity to better communicate the value of high quality data deflecting the "risk" away from the data warehousing team and back to the business clients: are they willing to accept the risk of making bad decisions or not having accurate reporting due to poor upstream data quality?

More data quality management resources

More data warehouse resources


Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google



RELATED CONTENT
Data quality mgmt. best practices
Information assurance: Dependability and security of networked information systems
What is high quality information?
Data quality and governance management quiz
Thirteen causes of enterprise data quality problems
How to maintain data quality and provide high quality information management and analysis
Data quality management pitfalls: Three common mistakes to avoid
How to develop and maintain an enterprise data quality management strategy, with Larry English
Five steps for weaving data quality management into your enterprise data integration processes
Data quality management: Problems and horror stories
Data quality management: Follow the doctor's orders

Data governance and quality
Are there data governance plans, templates or standard procedures?
Should we buy data quality management tools or focus on policies?
Resolving data ownership issues with external funders, organizations
Where to find new academic resources on data quality best practices
How to estimate customer data cleansing costs
What is high quality information?
Data quality management tools: Where to get unbiased information
Roles and salaries of data quality and governance analysts
The role of data governance in unstructured data
How to maintain data quality and provide high quality information management and analysis

Data quality techniques and best practices
Poor data quality costing companies millions of dollars annually
Are there data governance plans, templates or standard procedures?
Should we buy data quality management tools or focus on policies?
Where to find new academic resources on data quality best practices
How to improve data quality on a tight budget -- a guide
Data quality management tips and best practices
Peachtree Data uses SAP BOBJ Data Services to clean up mailing lists
Data quality software, including dashboards for non-IT users, gaining traction
IBM acquires data discovery vendor Exeros
Data Quality Management Software Product Directory

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
data  (SearchDataManagement.com)
data governance  (SearchDataManagement.com)
data quality  (SearchDataManagement.com)
data scrubbing  (SearchDataManagement.com)
fixed data  (SearchDataManagement.com)
raw data  (SearchDataManagement.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Search and Browse the Expert Answer Center
Search and browse more than 25,000 question and answer pairs from more than 250 TechTarget industry experts.
Browse our Expert Advice

About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Site Map




All Rights Reserved, Copyright 2005 - 2009, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts