Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints
Date Issued
2024
Author(s)
DOI
10.5220/0012620200003687
Abstract
In the continuously evolving and growing landscape of Big Data, a key challenge lies in the transformation of a Data Lake into a Data Mesh structure. Unveiling a transformative approach through semantic data blueprints enables organizations to align with changing business needs swiftly and effortlessly. This paper delves into the intricacies of detecting and shaping Data Domains and Data Products within Data Lakes and proposes a standardized methodology that combines the principles of Data Blueprints with Data Meshes. Essentially, this work introduces an innovative standardization framework dedicated to generating Data Products through a mechanism of semantic enrichment of data residing in Data Lakes. This mechanism not only enables the creation readiness and business alignment of Data Domains, but also facilitates the extraction of actionable insights from software products and processes. The proposed approach is qualitatively assessed using a set of functional attributes and is compared against established data structures within storage architectures yielding very promising results.