Organizations are looking for new ways to gain greater efficiency, flexibility, and performance from their data infrastructures. This is why data mesh is one of the most exciting new developments in the technology arena, particularly in data science and analytics. It promises a completely modernized revision of the traditional data infrastructure.
What is Data Mesh?
Developed by Zhamak Dehghani at Thoughtworks, data mesh is a decentralized approach to organizing a data infrastructure. Rather than structuring a data infrastructure around a central repository, a data mesh distributes the data evenly across the organization, and different specialized business teams share in managing the data.
Data mesh has four key principles:
- Domain-oriented data ownership and architecture
- Data as a product
- Self-serve data infrastructure as a platform
- Federated governance, for maximum interoperability
With data mesh, stakeholders in business domains can establish “data domains,” which store the data in their preferred ways, and facilitate self-service access to that data, in a way that meets their needs. Beyond that, they can make the data available throughout the organization in the form of “data products” that are trusted, because they adhere to an organization-wide data provisioning system. They are also reusable, to support myriad use cases without having to re-invent the same data products for multiple purposes.
The Need for Seamless Provisioning
Consider the “organization-wide data provisioning system” we just mentioned. This is where data virtualization comes in, a technology that serves as a key foundation of data mesh by enabling a seamless way to create, access, and govern data products.
Because data virtualization is established as an enterprise data-access layer above an organization’s disparate data sources, it also enables organizations to build any number of data models, including data products, without affecting the underlying source data, and much more quickly than using traditional data integration approaches. Also, with data virtualization, data products can be effortlessly reused. They can also be automatically listed in an organization-wide data catalog, for easy consumption, as with a data marketplace.
Because of the architecture of data virtualization, data virtualization also enables organizations to implement data governance protocols across the entire organization from a single point of control. It also enables organizations to define common entities that are represented consistently across all data products, for maximum interoperability. In this way, it enables the federated governance principle of data mesh.
The Need for Privacy
In addition, trusted, compliant data products need to be built on a strong foundation of privacy and security. In a distributed data environment, organizations need to be able to mask sensitive data, keep it private, or prevent unauthorized access.
Privitar is a company with deep experience in this area. Recently, Privitar and Denodo, makers of the Denodo Platform, a leading data virtualization platform, announced a partnership to help organizations ensure that their data stays safe in data mesh configurations. With this partnership, organizations will be able to leverage the Denodo Platform to create data products supported by Privitar policies, to automatically enable access to sensitive, governed data, in full compliance with current regulations.
Learn More on May 4th
To kick off this partnership, Privitar will present at Denodo’s Fast Data Strategy Virtual Summit AMERICAS, on Wednesday, May 4th. We hope you will join us; register today!
- Enterprise Data Unlocks Generative AI Potential with Vertex AI + Denodo - April 8, 2024
- Data Mesh and the Need for Seamless Provisioning and Privacy - April 28, 2022