Enhancing Interaction with Data Lakes Using Digital Twins and Semantic Blueprints

Abstract

Advanced analytical techniques and sophisticated decision-making strategies are imperative for handling extensive volumes of data. As the quantity, diversity, and speed of data increase, there is a growing lack of confidence in the analytics process and resulting decisions. Despite recent advancements, such as metadata mechanisms in Big Data Processing and Systems of Deep Insight, effectively managing the vast and varied data from diverse sources remains a complex and unresolved challenge. Aiming to enhance interaction with Data Lakes, this paper introduces a framework based on a specialized semantic enrichment mechanism centred around data blueprints. The proposed framework takes into account unique characteristics of the data, guiding the process of locating sources and retrieving data from Data Lakes. More importantly, it facilitates end-user interaction without the need for programming skills or database management techniques. This is performed using Digital Twin functionality which offers model-based simulations and data-driven decision support.