How to improve data quality on a tight budget -- a guide

Listen to a podcast and read a Q/A with author and data quality expert Arkady Maydanchik on how a data quality assessment can help identify data quality problems and find solutions. Get tips, advice and best practices information from data management experts and excerpts of data quality books.

During uncertain economic times, there is a certain amount of belt-tightening expected across the board, and the IT department is not immune to this. Yet before you grab the knife to start slashing the budget, it is worth considering that reducing the investment in any program or infrastructure that supports the organization's business needs is a measure that will not only diminish needed agility during poor economic times but will also slow the organization's competitiveness when times start to get better.

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Data quality techniques and best practices Understanding five major enterprise information management benefits Gartner: Open source data quality software focuses on data profiling Poor data quality costing companies millions of dollars annually Are there data governance plans, templates or standard procedures? Should we buy data quality management tools or focus on policies? Where to find new academic resources on data quality best practices Data quality management tips and best practices Peachtree Data uses SAP BOBJ Data Services to clean up mailing lists Data quality software, including dashboards for non-IT users, gaining traction IBM acquires data discovery vendor Exeros Data quality management software Gartner: Open source data quality software focuses on data profiling Informatica hopes to bridge the business/IT divide with latest release DataFlux demonstration of product data quality software Poor data quality costing companies millions of dollars annually Should we buy data quality management tools or focus on policies? Data quality management tips and best practices Peachtree Data uses SAP BOBJ Data Services to clean up mailing lists SearchDataManagement.com product directory library Data quality software, including dashboards for non-IT users, gaining traction IBM acquires data discovery vendor Exeros Data management tutorials Seven secrets to business intelligence (BI) success How to design project management plan templates for large projects Handling project management issues for a proprietary ownership conflict Should we use traditional or agile software development processes? Next steps for business intelligence: Video Improving ODBC application performance and coding Selecting ODBC functions for optimized SQL statements Guidelines for managing data updates to optimize ODBC performance Is it better to have a centralized or decentralized master data structure? Can we leverage existing data quality tools for an MDM program?

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary data  (SearchDataManagement.com) data governance  (SearchDataManagement.com) data quality  (SearchDataManagement.com) data scrubbing  (SearchDataManagement.com) fixed data  (SearchDataManagement.com) raw data  (SearchDataManagement.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

Often, data quality management is seen as a good practice, but most organizations do not have the discipline to integrate its value proposition holistically across the organizational value drivers, whether they are focused on revenue growth or operational cost containment. Therefore, a recession actually provides an excellent opportunity to assess two aspects of the relationship between data quality and the business. Companies can directly connect high-quality data to the organization's value drivers, weighted by the perception of existing economic trends.

Determining data quality's impact on business processes

The first aspect is identifying specific business processes that will be positively affected by high-quality data. Data quality may affect different business processes in different ways. A data quality analysis should incorporate a business impact assessment to identify and prioritize risks. Those business impacts associated with bad data can be categorized within four general categories for assessing either the negative impacts suffered or the potential new opportunities for increased value resulting from improved data quality:

• Revenue growth, incorporating financial impacts such as decreased sales, higher cost to acquire new customers, and customer retention.
• Cost reduction, such as increased operating costs, reduction or delays in cash flow, and additional unnecessary charges.
• Risk management and confidence management, such as credit assessment, investment risks, competitive risk, capital investment and/or development, fraud and leakage, compliance risks, decreased organizational trust, low confidence in forecasting, inconsistent operational and management reporting, delayed or improper decisions, decreased customer, employee, or supplier satisfaction, or lowered market satisfaction.
• Productivity impacts such as increased workloads, decreased throughput, increased processing time, or decreased end-product quality.

Assessing the business impacts associated with data means working with the business consumers to understand their information needs and the corresponding data quality expectations. One can elicit information about the business impacts associated with data quality by asking probing questions such as these:

1. What importance does data have in achieving the organization's business objectives?
2. What data is critical to your business processes?
3. How confident are you in the accuracy of your data?
4. What changes to the data can improve business process performance?
5. In which aspects of data improvement should the company be investing, and in what time frame?

Any significant data issues that will affect the business are likely to be revealed during this process, and this provides you with the basis for further researching documented business issues and connecting them to any type of data flaw. It will provide a connection between data quality improvement and a potential for increase in value. At the same time, this provides an opportunity to reinforce conformance with business data quality expectations by validating data quality rules and the corresponding thresholds for acceptability.

This leads into the second aspect of data quality management: monitoring the level of efficiency of the data governance and data stewardship activities. As a data quality program matures, the management of issues transitions from a reactive environment to a proactive one, and this can be scored in relation to continuous monitoring of the quality of data. In the optimal environment, the data stewards allocate time to address the most critical issues as they are identified early in the processing streams. Less efficient organizations have stewards reacting to issues at their manifestation point, at which time these issues may have already caused significant business repercussions.

The importance of designing a data quality scorecard

Therefore, organizations that inspect, monitor and measure the performance of data quality initiatives on an ongoing basis, across all processing streams, can then populate a data quality scorecard reflecting the effectiveness of the program and the efficiency of its staff. Together, these two aspects reflect the value of the program and the way that it has been implemented. Focusing on both of these aspects provides a number of valuable benefits:

• It can demonstrate the value proposition for maintaining the effort, even in the face of economic stress.
• It can provide long-term justification for continued funding and growth of data quality management as the recession ends.
• It can help identify additional areas with an acute data quality improvement need that can help support the organization's survival during a recession.
• It will demonstrate an example of proactive value management to other organizations.

On the other hand, it may turn out that this data quality assessment will show that the organization does not get a reasonable return on its data quality management investment. In this case, it provides an opportunity to reduce operating costs associated with the areas of missed expectations. While this is unlikely, it does demonstrate a level of accountability that should pervade all management activities. It is more likely, however, that this process will only strengthen the view that a data quality management program is fundamental to the ultimate success of the business.