Internal data quality standards not just a game for business users

Ensuring that business users follow corporate standards when entering and updating data is vital to the success of data quality improvement efforts.

Fingers first point to organizational leadership as a key driving force in the effort to improve data quality. But the day-to-day process of ensuring that business users adhere to internal data quality standards can ultimately affect the success of a quality improvement initiative as much as any data quality charter or enterprise data governance program can.

Missing or incorrect data, duplicate entries, misidentified information, undocumented relationships between data elements -- they’re just some of the problems that tarnish data every day in enterprises worldwide, often because of mistakes made by business users. Added together, all of those little data quality problems can cause big issues in business processes and result in significant losses of both money and employee productivity, according to analysts.

As the potential impact of data errors has become more widely recognized, interest in enforcing corporate data quality standards has “gone from complete neglect to the realization that something has to be done about this,” said Lyn Robison, an analyst at Gartner Inc. in Stamford, Conn.

On the surface, the challenges inherent in ensuring that business users consistently employ internal quality standards may appear to be counterintuitive. Most data entry activities aren’t mathematically complex. It’s like a golfer stroking a 4-foot putt, not trying to hit a 250-yard fairway shot that needs to curve over a lake in order to land safely on the green. But those short putts don’t always drop into the hole, and Robison and other analysts said that mistakes in entering and updating data are all too common in fast-paced business environments.

Getting business users to follow agreed-upon internal standards is the core element of a data quality initiative, said William McKnight, president of McKnight Consulting Group in Plano, Texas. There’s a lot of work to be done first, though: “Coming up with the data quality standards and putting them in practice is the hard part,” he added.

Teaming up on data quality standards
And unlike golf, data quality is a team sport. A good game plan starts with regular meetings to thrash out data quality definitions and rules with all of the players, including data stewards and other representatives from business units and departments. “Getting business users involved early in the process of creating standards is always a good idea,” McKnight said. “They’re likely to challenge the rules if they weren’t part of the process of determining them.”

What’s critical at these meetings is recognizing and appreciating the various points of view, he added. IT and business professionals tend to look at data from different perspectives and with different sets of priorities; upending entrenched habits on both sides is difficult. In the past, IT handled the data quality process unilaterally in many organizations. Now, McKnight advised, it’s often necessary to relax political will in the name of reaching a collective outcome.

“The most common impediment to formulating and implementing data quality standards is getting a consensus across the board,” he said. And, he added, that’s where the elusive talent of effective leadership is most needed to help guide organizations through the “tough judgments” that have to be made on standards while still getting everyone to agree on them.

In the end, though, business users likely will be expected to accept accountability for the quality of data. Robison recommends making adherence to data quality standards part of job descriptions and the performance review process. “That is always the best incentive,” he said.

The carrot is that taking a little extra care with data often results in tangible business benefits. So if business users can avoid negative consequences and help boost business performance by embracing internal standards, why might they not consistently follow the rules?

Bad apples versus busy bees
The answer, analysts say, can be divided into bad and understandable reasons. The bad ones include some of the usual workplace suspects: resistance to change, ignorance, pettiness, laziness and departmental isolationism. On the other hand, there are the realities of busy schedules and deadlines that need to be met.

“The enforcement of business rules often conflicts with the desire to get the job done,” said David Loshin, president of consultancy Knowledge Integrity Inc. in Silver Spring, Md. Hence, he added, it’s important to define what constitutes sufficient data quality even if the information isn’t completely error-free.

Ted Friedman, another Gartner analyst, suggested that the process of ensuring compliance with data quality standards should start at the beginning, with controls that enforce the rules when users enter or update data.

To help keep flawed information out of corporate systems, Friedman said, nonconforming data should be rejected -- automatically if possible, or through manual efforts if not. Either way, the data entry process should be monitored, he said. At the same time, Friedman added, organizations should evaluate possible changes to business processes to relieve bottlenecks that can cause data errors. And investing in ongoing data quality training can further assist efforts to get business users to toe the quality line, he said.

Roger du Mars is a freelance writer based in Redmond, Wash. He has written for publications such as Time, USA Today and The Boston Globe, and he previously was the Seoul, South Korea, bureau chief of Asiaweek and the South China Morning Post.

Dig Deeper on Data quality techniques and best practices