The lines between data quality and data governance continued to blur in 2009, while data integration took on renewed importance. Find out what 2010 has in store for those markets with these expert forecasts.
Principal analyst at Cambridge, Mass.-based Forrester Research, covering data integration, data quality, data governance, and master data management (MDM)
- Data governance best practices and supporting technologies will dominate the data quality discussion. Expect to hear loud, competing messages from all the leading data quality and MDM vendors on how they will be delivering differentiated data governance capabilities. These data quality and MDM vendors recognize what data quality pros have known for years: Data quality is not a technology problem but an organizational issue, where the critical roles and responsibilities for delivering trusted data do not exist. Large enterprise investments in data governance strategies drive software vendors to provide tooling and best practices to support data governance efforts.
- Real-time data quality services gain momentum. Increased adoption of SOA and Web service architectures across enterprise silos opens the door for data quality technology to escape from its user-imposed downstream, batch-oriented prison. Expect increased adoption and more compelling use cases across industries for real-time data quality services to validate and cleanse data as it's captured in upstream applications.
- Data integration and data quality will continue to explore "the cloud" – but be patient. Cloud-based IT infrastructure is no passing fad, but don't expect enterprise data integration and data quality platforms to suddenly transform into hosted solutions. Integration issues, security, and cross-enterprise complexity will keep these technologies grounded at least for the next two to three years. That said, expect a significant increase in the usage of integration technology that enables "cloud to on-premise" integration. More specifically, data synchronization with software-as-a-service (SaaS) apps like Salesforce.com and on-premise data warehouses and inventory management systems, for example, will continue to drive vendor investment and user experimentation in cloud-integration capabilities.
- Enterprise integration strategies will combine data integration and application integration competencies into a combined shared service organization. The lines between data integration technologies -- such as ETL, change data capture and data replication -- and application integration middleware such as ESB and B2B gateways have begun to blur. Data integration vendors continue their push into middleware by enabling more real-time integration functionality, and it's no longer obvious to enterprise architects which integration technology should support their complex integration requirements. To mitigate, expect CIOs to consolidate their skilled data and application integration resources into a cross-enterprise shared services organization that will deliver faster, more economical integration solutions to support cross-enterprise needs.
Founder and chief research officer for the San Francisco-based analyst firm the MDM Institute and chairman of the MDM summit conferences
- During 2010, most enterprises will struggle with cross-enterprise master data governance scope as they initially focus on customer, vendor or product governance. Enterprise-level data governance programs that include entire master data lifecycles will be mandated as a core phase 1 deliverable of large-scale MDM projects. By the end of 2010, integrated master data governance (not data steward consoles, a.k.a. downstream or "passive data governance") will be a top five criterion for MDM hub evaluations.
- Through 2011, major systems integrators and boutique consultancies will focus on productizing data governance frameworks while MDM software providers struggle to integrate upstream data governance process with hub technologies as "active data governance." Concurrently, global 5000 enterprises' IT organizations will struggle to deploy cross-line of business data governance, let alone globally distributed/federated decision rights for governance, as they mobilize to evolve from levels 1 and 2 data governance maturity to levels 3 and 4.
- By 2012-13, vendor MDM platforms will finally move from "passive-aggressive data governance" mode (oversell, under-deliver) to "active data governance." Concurrently, use of social media frameworks such as wikis will assist in evolving the semantics of a unified and shared business vocabulary at the department, line-of-business, and enterprise levels.
Various Gartner Inc. analysts covering data management, including Ted Friedman, Mark Beyer, Eric Thoo, Donald Feinberg, and Andreas Bitterer
- Through 2012, only 25% of organizations will proactively and comprehensively include data quality processes and competencies in their data integration work. Business drivers such as the imperative for faster time-to-market and the agility to change business processes and models are forcing organizations to manage their data assets differently. Simplification of processes and of IT infrastructure are necessary to achieve transparency, and transparency requires a consistent and complete view of the data that represents the performance and operation of the business. But as organizations increase their investment in data integration capabilities, far too few recognize the connection with data quality. Their perception is that they need worry only about the "plumbing" of how data gets from here to there. Data integration initiatives have focused heavily on the mechanics of data access and delivery but have ignored the fact that robust delivery capabilities will only increase the availability and speed of access to data that is often incomplete, inconsistent and inaccurate.
- Through 2012, organizations that fail to introduce data management and integration architectural components to their existing data warehouse will be unable to support in-line operational analytics. Data warehousing practices have matured. Adopters can now capitalize on the data management and integration lessons learned during the past 15 years to enhance information management initiatives throughout their organizations. The global financial challenges dictate that, now more than ever, companies need the competitive advantages and survival capabilities of analytical data stores (notably data warehouses, data marts and operational data stores) because the cost of not supporting business analysis might be outright business failure. However, even though data warehousing has a relatively long history for an IT solution (the term "data warehouse" was coined in the late 1980s), many organizations that are mature in the implementation, operation and use of data warehouses have only just begun to use them as infrastructure to support more aggressive use of performance management, business activity monitoring, in-line operational analytics and other operational areas where analytics can influence tactical decisions so that they reflect strategic goals.
- Through 2013, 35% of data management initiatives will include alternative tools in open source, cloud or SaaS-based models as part of a comprehensive range of deployment options. Contemporary business pressures for speed, agility, competitive differentiation and compliance, among others factors, are driving organizations to rethink the way they look at and handle data management. Emerging delivery approaches are providing alternatives to various aspects of the data management discipline, such as data quality, data integration and DBMSs. Alternative approaches using open source, SaaS and cloud-based software technologies are creating radical shifts in technology providers' business models and the ways in which consumers deploy technology. Data management activities in enterprises are slowly diversifying their deployment approaches and incorporating technology components implemented using a nontraditional IT delivery model.
- By the end of 2012, the discrete markets for data integration and data quality tools will have converged into one. Following the trend for providers and other participants in the data integration and data quality tools arena to combine products or make them seamlessly interoperable, data quality has increasingly become a function of data integration. In addition, the movement toward consolidation of technology offerings in these data management domains is being driven by buyer demands. Many end users already use transformation logic and business rules from their data integration platform to mimic data quality functions such as matching, cleansing and standardization, instead of using distinct data quality tools. Data quality functionality will increasingly become a required feature in various data integration efforts.