What are your recommendations for instituting effective policies and processes for governing sets of big data?
Big data has many definitions, but all of them come down to these main points: It consists of a high volume of material, it comes from many different sources, it comes in a variety of formats, it arrives at high speeds and it requires a combination of analytical or other actions to be performed against it. But at heart, it's still some form of data or content, though slightly different than what has been seen in the past at most organizations. And because it is a form of data or content, business-critical big data needs to be included in data governance processes.
Remember that not all data must be governed. Only data that is of critical importance to an organization's success (involved in decision making, for example) should be governed. For most companies, that translates to about 25% to 30% of all the data that is captured.
What governance best practices apply to big data? The same best practices that apply to standard data governance programs, enlarged to handle the particular aspects of big data:
- Take an enterprise approach to big data governance. All data governance programs should start with a strategic view and be implemented iteratively. Governance of big data is no different.
- Balance the people, processes and technologies involved in big data applications to ensure that they're aligned with the rest of the data governance program. Big data is just another part of enterprise data governance, not a separate program.
- Appoint business data stewards for the areas of your company that are using big data and ensure that they receive the same training as other data stewards do, with special focus on big data deemed necessary due to the technology in use at your organization.
- Include the value of big data governance in the business case for overall data governance.
- Ensure that the metrics that measure the success of your data governance program include those related to big data management capabilities.
- Offer incentives for participating in the data governance program to all parts of the business using big data to encourage full participation from those areas.
- Create data governance policies and standards that include sets of big data and the associated metadata, or that are specific to them, depending on the situation.
Dig deeper on Big data management
Related Q&A from Anne Marie Smith, Ph.D.
When implementing an MDM initiative, businesses should take steps to incorporate effective data quality management practices from the outset.continue reading
Data governance programs often rely on the part-time data steward -- an employee from a business unit who also serves as a data expert.continue reading
Enlist senior-level champions from both business and IT to guide a data governance program and achieve harmony between the two sides.continue reading
Have a question for an expert?
Please add a title for your question
Get answers from a TechTarget expert on whatever's puzzling you.