Modern Data Storage Architectures for Managing Big Data: The Role of Semantically Enrichment Mechanisms in Data Management and Security

Abstract

This PhD thesis moves in the broader area of Smart Data Processing (SDP) and Systems of Deep Insights (SDI) and focuses on Big Data storage and management, addressing significant challenges such as optimizing data access, security, and retrieval. It explores current approaches for efficiently managing data sources, their organization, and storage for seamless access and retrieval while addressing challenges related to data integrity, privacy, and access control. A key contribution of this research is the development of a semantically enriched Data Lake framework, which enhances data structuring, accessibility, and governance by leveraging metadata-driven semantic data blueprints (SDB) supporting also process mining. Empirical findings demonstrate that Data Mesh architectures significantly outperform traditional Data Lakes, offering improved scalability, flexibility, and decision-making agility. The thesis demonstrates how transitioning from centralized Data Lakes to decentralized, semantically enriched Data Meshes enables enhanced data discoverability, real-time insights, and secure cross-organizational collaboration. The application of the aforementioned concepts in a smart manufacturing environment showcases how metadata-driven Data Meshes streamline operational efficiency, improve data traceability, and facilitate decentralized access control mechanisms. The integration of Blockchain technology and Non-Fungible Tokens (NFTs) further strengthens data ownership, integrity, and secures access management in Data Lakes and Data Meshes. Through experimental evaluation using real-world industrial data, research conducted highlights the effectiveness of the proposed framework in optimizing data workflows, reducing processing delays and enhancing security. This research provides valuable methodologies for enterprises seeking to harness the power of Big Data, fostering a more intelligent, secure, and adaptive data management paradigm.