data fabric

What is a data fabric?

A data fabric is an architecture and software offering a unified collection of data assets, databases and database architectures within an enterprise. Data fabrics can be confined to an application, used to collect distributed data and can extend to all enterprise data. Data fabric is the implementation of general data virtualization principles.

What is the purpose of a data fabric?

The purpose of a data fabric is to create a unified view of the associated data to facilitate application access to information, regardless of the data's location, database association or structure. It is also used to simplify analysis, often with AI and machine learning. As such, data fabrics are becoming a primary tool in converting raw data into business intelligence.

Data fabrics can also facilitate application development by creating a common model for accessing information, which is a departure from the application and database silos already common. This same harmonization can improve operational efficiency. At the line organization level, they can provide better information access. At the IT level, data fabrics improve efficiency by creating a single layer where data access is managed across all resources.

A data fabric is a recent innovation in enterprise data management and digital transformation. The most general application of data fabrics is the simplification of database access, which is made complicated by the wide variety of applications, data models, formats and distributed data assets found in a typical enterprise.

Business and technical benefits of a data fabric

Data fabrics offer both line operational benefits and benefits to the IT organization, including the following:

• Break down data silos. Modern databases are usually associated with applications or groups of applications. Databases also tend to grow as applications are added to the enterprise inventory. This often results in silos of data with different structures and formats. Data fabrics improve the ability to gain insight into the full range of enterprise information and use the collected data to improve operational efficiency and empower workers.
• Unite databases spread over a large area. Data fabrics can make sure that their location differences don't form a barrier to access. They simplify application development by harmonizing different data access APIs. They can be used either to optimize specific application data use without making data less accessible to other applications, or to unify data that's already become siloed.
• Act as a single way to access information in both the cloud and data center. The number of applications hosted in or partially in the public cloud has increased. Data fabrics can improve application portability and cloudbursting or backup of cloud components in the data center.

Data fabric implementation challenges

The greatest challenge in implementing data fabric solutions is the wide variety of databases, data management policies and storage locations found in most enterprises. A fabric solution must obviously be able to harmonize all these differences. If not, application silos and data silos will persist and the sum of information available in the data fabric will be limited.

Addressing this challenge must start with creating a unified platform as the foundation of a data fabric. Multiple platforms will add to the silo problem and reduce the operational efficiency benefits considerably. This means that if data fabric technology is initially applied to a specialized set of data, or to an operating unit or subsidiary, the technology must be extendable to the company at large, and that extension should be the goal.

Harmonization and unification through virtualization always create a risk, and that's true of data fabrics. For example, location-independence means that applications that access information via a data fabric are insulated from knowing where the data is located. This can create serious performance implications. In cloud computing, it can create high data transfer charges if data is moved regularly across the hybrid or multi-cloud boundary.

Other challenges include different access mechanisms found among the various databases and the difference in APIs and query languages. A good fabric strategy must support a common access/query mechanism. At the same time, it cannot exclude the use of specialized APIs or query languages, or current applications wouldn't be able to run. Thus, the fabric concept must reach the goal of harmonizing on the fabric access/query technology as applications are added or modified.

Data fabric uses and examples

The most common use of a data fabric is the virtual/logical collection of geographically diverse data assets to facilitate complete access and analysis. In this application, the data fabric is typically used for centralized business management. The distributed line operations that collect and use the data regularly are still supported through their traditional applications and data access/query interfaces. This is particularly valuable for organizations that have regional or national segmentation of their activities, but require central management and coordination.

A second common use is the creation of a unified data model for a company following a merger and acquisition. In these situations, it's almost certain that the database and data management policies of the previously independent organizations will be different, making collection of information across organizational boundaries difficult. A data fabric can resolve this by creating a unified view of data. This allows the combined entity to gradually harmonize on a single (virtual) data model if desired, but at the best pace for operational efficiency while sustaining profits and sales.

Data fabric software market

The market for data fabric software has been estimated at $1.1 billion in 2020, with a projected growth to $3.7 billion annually by 2026. However, it must be noted that the boundaries of the data fabric market are still hazy and have yet to be defined.

Popular data fabric software includes the following:

• Denodo
• Cinchy
• Atlan
• IBM Cloud Pak for Data
• Talend
• Tibco Software
• SAP Data Intelligence
• NetApp
• K2View Data Fabric