A data mesh and data fabric can work together to create a cutting-edge solution to a normally overwhelmingly complex challenge.
Data mesh and data fabric are often seen as two opposite approaches pitted against each other. While data fabric focuses on the technologies required to support metadata-driven use cases across hybrid and multi-cloud environments, data mesh focuses less on technology and instead takes a people and process-centric view. However, it’s also true that data mesh and data fabric are two equally effective, yet different, architectures that complement each other in what we refer to as a meshy fabric approach.
Data as a Powerful Product
It helps to consider data as a product. Data is a powerful asset, and with proper classifications, descriptions, and quality can also be packaged as a product. Think of your domain experts or data producers as shopkeepers. They’ll be the ones who welcome people into the world of data and guide them toward best-in-class usage.
When viewed through this lens, data can provide real ROI, with data producers acting as shopkeepers. Data leaders can then understand how to make data more accessible and understandable to their consumers, and consider other useful contexts for consumers to utilize their data. To be successful, companies must fully embrace a culture of decision-making centered around their data.
Data mesh inverts the common model of having a centralized team who manages and transforms data for wider consumption. A data mesh architecture calls for responsibilities to be distributed among the people closest to the data. A data fabric, on the other hand, focuses on the technologies required to support metadata-driven use cases across hybrid and multi-cloud environments. Once data producers and consumers recognize their new symbiotic relationship, an organization can set up a data catalog to do the heavy lifting to create a meshy fabric. By combining data mesh and data fabric, organizations can overcome bottlenecks and disconnections that are typical of data lakes and data warehouses so that data engineers don’t have to play middlemen between data producers and consumers.
See also: Driving Faster Insights with a Data Mesh
How Data Catalogs Support a Meshy Fabric
Data leaders can use the data catalog to establish governance principles that clarify best practices for data usage across the enterprise. Leaders can build policies regarding data into the catalog, either from an internal perspective or an external and a compliance-focused perspective. Through a data governance program, companies can establish a set glossary of terms that are standardized across the entire business to make finding definitions easier for users. As a result, the data catalog acts as a key foundation of a painless data governance program that can support the entire organization and increase efficiency.
A best-in-class data catalog will ingest metadata automatically using metadata connectors and apply behavioral intelligence to augment technical metadata with usage, popularity, lineage, classification, naming, and other contextual information. This helps an organization prepare and showcase its most valuable data products for wider consumption.
As an example, one Fortune 500 financial services organization leveraged a data catalog to build a strong data culture and bring data into the hands of teams across the firm. This financial services organization embarked on an ambitious digital transformation project using best-in-class, cloud-native technology. With this tech, they built data mesh domains that ensured data consistency across the firm, decentralized data management to create a leaner organization, and transformed the customer experience across all channels. This included retiring their old data lake system and moving to the cloud as the front door to their data, enabling data governance and self-service.
This entire process showcases how the marriage of data mesh and data fabric into a meshy fabric combines the best of decentralized autonomy. This allows a business to move at maximum speed with the right oversight and risk mitigation. Additionally, data consumers now have a reliable source of well-defined data products with owners who are directly accountable.
For many companies, it’s not necessarily a choice of one or the other but a combination of both. Data mesh and data fabric work together to create a cutting-edge solution to a normally overwhelmingly complex challenge. Data mesh and data fabric can provide a solution for enhanced decision-making to increase the overall success by allowing companies to have control and autonomy over their data. These are investments worth making, and enterprises that commit to these tools can realize outsized gains for their business.
