How to estimate customer data cleansing costs

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Data governance and quality Are there data governance plans, templates or standard procedures? Should we buy data quality management tools or focus on policies? Resolving data ownership issues with external funders, organizations Where to find new academic resources on data quality best practices Data quality management for data warehouses What is high quality information? Data quality management tools: Where to get unbiased information Roles and salaries of data quality and governance analysts The role of data governance in unstructured data How to maintain data quality and provide high quality information management and analysis Data quality techniques and best practices Understanding five major enterprise information management benefits Gartner: Open source data quality software focuses on data profiling Poor data quality costing companies millions of dollars annually Are there data governance plans, templates or standard procedures? Should we buy data quality management tools or focus on policies? Where to find new academic resources on data quality best practices How to improve data quality on a tight budget -- a guide Data quality management tips and best practices Peachtree Data uses SAP BOBJ Data Services to clean up mailing lists Data quality software, including dashboards for non-IT users, gaining traction Data stewardship Poor data quality costing companies millions of dollars annually Resolving data ownership issues with external funders, organizations The importance of metadata management in EIM Keys to planning an enterprise information management (EIM) initiative Data quality software, including dashboards for non-IT users, gaining traction IBM acquires data discovery vendor Exeros Cleansing data after an acquisition often calls for data quality software Customer data integration helps credit union personalize customer experience Exec explains IBM's Information On Demand (IOD) initiative Business intelligence, data mining maximize direct marketing for Intellidyn

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary data  (SearchDataManagement.com) data governance  (SearchDataManagement.com) data quality  (SearchDataManagement.com) data scrubbing  (SearchDataManagement.com) fixed data  (SearchDataManagement.com) raw data  (SearchDataManagement.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

The challenge with this question is that underlying its simplicity lie many latent questions whose answers are needed before any kind of customer data cleansing costs analysis can be considered. For example, what data elements constitute the customer record? How many records are there? What are the criteria for declaring a record "clean"? What types of customer data are there? Individuals or organizations? How old are the records? Are they in a single table or scattered across many data assets? What approaches are to be taken for cleansing? There may be studies performed by vendors on the average cost, but I suspect that beneath this question lurks a number of other more important ones.

To start thinking about the cost of cleansing, consider this example, with residential customer data consisting of first name, last name and telephone number. One can determine if a single record is "correct" using this algorithm: Call the telephone number, ask to speak with the person whose name shares the record with the telephone number. If the person comes to the phone, ask if all the values are accurate, and correct those that are not. If there is no one there by that name, the record is incorrect. However, at this point what can be done to correct it? Either the name is not correct or the number is not correct. The next step in cleaning requires additional information, and if none is available, then the algorithm ends.

Simplistic? Yes. Accurate? Yes. Cost effective? Depends on the number of records, staff members and telephones. Scalable? Not really. There are alternatives, but reliance on different approaches start to impact those key considerations. Automated solutions may be more scalable, more costly, less accurate, more complex, require more expertise, etc.

It may be better to challenge the question, then, and turn it into a different sort of beast by suggesting that these questions be answered first and then look at the different alternatives and their corresponding costs:

The level of effort that is reasonable to spend on customer data cleansing must be less than the value of the accrued business benefits, and this provides an upper limit to what could be budgeted for the process.