Journal of Technology and Informatics
Systematic Literature Review: Modern Data Stack Architecture Layers and Technologies for Data Warehousing
DOI:
https://doi.org/10.37802/Keywords:
Modern Data Stack, Data Warehouse, Cloud Architecture Layers, Systematic Literature ReviewAbstract
This systematic literature review (SLR) examines modern data stack architectures for data warehousing, focusing on their architectural layers and the tools used within each layer. Following SLR procedures, scholarly articles published between 2020 and 2025 were retrieved from IEEE Xplore and Google Scholar and screened using predefined inclusion and exclusion criteria, resulting in 35 primary studies. The review addresses three research questions: (1) which architectural layers predominate in modern data stack frameworks, (2) which tools and technologies are commonly applied within each layer, and (3) which trends shape the evolution of these architectures. The findings show a clear convergence toward modular, cloud-native architectures consisting of six layers: data ingestion, storage, transformation, orchestration, serving, and governance. Four major trends were identified: the shift from Extract, Transform, Load (ETL) to Extract, Load, Transform (ELT), the emergence of lakehouse architectures, the growing adoption of managed and serverless services, and the expansion of real-time analytics. This study contributes a layer-by-layer technology mapping that offers a structured reference for practitioners, researchers, and organizations in evaluating and designing cloud-based data warehouse platforms.
Downloads
References
[1] A. Eweje and F. Ohaegbu, “Advances in Modern Data Stack Architectures for Scalable Data Integration and Business Intelligence,” International Journal of Multidisciplinary Research and Growth Evaluation, vol. 2, no. 5, pp. 538–550, 2021. DOI: 10.54660/.ijmrge.2021.2.5.538-550
[2] T. T. Bukhari, O. Oladimeji, E. D. Etim, and J. O. Ajayi, “Cloud-Native Business Intelligence Transformation: Migrating Legacy Systems to Modern Analytics Stacks for Scalable Decision-Making,” International Journal of Scientific Research in Humanities and Social Sciences, vol. 1, no. 2, pp. 744–762, 2024. DOI: 10.32628/ijsrssh242763
[3] K. M. Reddy, “Cloud Data Warehousing: A Transformative Technology Across Industries - A Comprehensive Analysis of Implementation and Impact,” European Modern Studies Journal, vol. 9, no. 4, pp. 508–514, 2025. DOI: 10.59573/emsj.9(4).2025.49
[4] S. K. Park and J. Kim, “Design of Vessel Data Lakehouse with Big Data and AI Analysis Technology for Vessel Monitoring System,” Electronics, vol. 12, no. 8, p. 1943, 2023. DOI: 10.3390/electronics12081943
[5] L. AlOud and O. Alharbi, “A Systematic Literature Review of Distributed Data Warehouse Architectures,” International Journal of Technology Innovation and Management (IJTIM), vol. 5, no. 1, 2025. DOI: 10.54489/ijtim.v5i2.567
[6] I. R. Mukhlis and R. Santoso, “Perancangan Basis Data Perpustakaan Universitas Menggunakan MySQL dengan Physical Data Model dan Entity Relationship Diagram,” Journal of Technology and Informatics (JoTI), vol. 4, no. 2, pp. 81–87, 2023. DOI: 10.37802/joti.v4i2.330
[7] A. R. Aulia, S. Saputra, A. D. Titandy, M. Zacky, and A. Arwani, “Pengaruh Big Data dalam Akuntansi Syariah: Analisis Prediktif dan Pengambilan Keputusan Strategis,” Journal of Technology and Informatics (JoTI), vol. 6, no. 1, pp. 17–21, 2024. DOI: 10.37802/joti.v6i1.704
[8] D. Anoshin, D. Foshin, R. Storchak, and X. Ireton, Azure Data Factory Cookbook: Build and Manage ETL and ELT Pipelines with Microsoft Azure’s Serverless Data Integration Service. Birmingham, UK: Packt Publishing, 2020.
[9] B. Guntupalli, “The Evolution of ETL: From Informatica to Modern Cloud Tools,” International Journal of AI, BigData, Computational and Management Studies, vol. 2, no. 2, pp. 66–75, 2021. DOI: 10.63282/3050-9416.ijaibdcms-v2i2p108
[10] P. Chauhan and M. Sood, “Big Data: Present and Future,” IEEE Computer, vol. 54, no. 4, pp. 59–65, Apr. 2021. DOI: 10.1109/MC.2021.3057442
[11] S. K. Somayajula, “Demystifying Modern Data Warehousing: From Traditional to Cloud-Native Solutions,” Journal of Cloud Data Management, vol. 11, no. 1, pp. 348–362, 2025. DOI: 10.32628/CSEIT25111235
[12] M. Armbrust, A. Ghodsi, R. Xin, and M. Zaharia, “Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics,” in Proc. 11th Conference on Innovative Data Systems Research (CIDR), 2021. [Online]. Available: http://cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf
[13] R. Thalpati, Practical Lakehouse Architecture: Designing and Implementing Modern Data Platforms at Scale. Shelter Island, NY, USA: Manning Publications, 2024.
[14] A. V. Chaudhari and P. A. Charate, “Optimizing Data Lakehouse Architectures for Scalable Real-Time Analytics,” International Journal of Scientific Research in Science, Engineering and Technology, vol. 12, no. 2, pp. 809–822, 2025. DOI: 10.32628/IJSRSET25122198
[15] D. Naphade, “The Evolution and Modernization of Data Pipeline Architectures,” European Journal of Computer Science and Information Technology, vol. 13, no. 6, pp. 42–53, 2025. DOI: 10.37745/ejcsit.2013/vol13n64253
[16] D. Foshin, T. Chernyshova, D. Anoshin, and X. Ireton, Azure Data Factory Cookbook: Build ETL, Hybrid ETL, and ELT Pipelines using ADF, Synapse Analytics, Fabric and Databricks. Birmingham, UK: Packt Publishing, 2024.
[17] B. Mohammed, “A Systematic Taxonomy of ETL Activities for Modern Data Pipelines,” International Journal of Advanced Multidisciplinary Research and Studies, vol. 5, no. 2, pp. 106–111, 2025. DOI: 10.62225/2583049X.2025.5.2.3816
[18] M. V. Talakh, T. O. Ushenko, O. V. Kaduk, and M. Y. Maksimovich, “Balancing Efficiency and Accuracy: Incremental Learning as a Key to Big Data Processing,” Optoelectronic Information-Power Technologies, vol. 48, no. 2, pp. 45–57, 2024. DOI: 10.31649/1681-7893-2024-48-2-45-57
[19] M. Nora, A. Shaikh, and R. Pradhan, “Managing Data Pipeline with Apache Airflow,” International Journal of Advanced Research in Science, Communication and Technology, vol. 3, no. 2, pp. 244–250, 2023. DOI: 10.48175/IJARSCT-12134
[20] V. N. K. Kundavaram, “Event-Driven Data Pipelines: A Cloud-Based Approach to Real-Time Data Processing,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 10, no. 6, pp. 364–369, 2024. DOI: 10.32628/CSEIT24106183
[21] P. K. Vattumilli, “Metadata-Driven ETL Pipelines: A Framework for Scalable Data Integration Architecture,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 10, no. 6, pp. 1799–1807, 2024. DOI: 10.32628/CSEIT241061224
[22] D. Bansal, “Enterprise Data Engineering: Architecting Modern Data Warehouses for Business Success,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 11, no. 2, pp. 3266–3277, 2025. DOI: 10.32628/CSEIT251112348
[23] B. Dash and S. Swayamsiddha, “Reverse ETL for Improved Scalability, Observability, and Performance of Modern Operational Analytics - A Comparative Review,” in Proc. 2022 OITS International Conference on Information Technology (OCIT), 2022, pp. 491–494. DOI: 10.1109/OCIT56763.2022.00097
[24] A. Dibouliya and V. Jotwani, “The Transformational Impact of Modern Data Warehousing on the Banking System,” International Journal for Research in Applied Science and Engineering Technology, vol. 11, no. 6, pp. 1862–1869, 2023. DOI: 10.22214/ijraset.2023.53968
[25] P. S. Dhoni, “From Data to Decisions: Enhancing Retail with AI and Machine Learning,” IRE Journals, vol. 5, no. 1, pp. 38–51, 2024. DOI: 10.47941/ijce.1660
[26] J. Qu and J. Wang, “Real-time Data Warehousing in the Big Data Environment: A Comprehensive Review of Implementation in the Internet Industry,” Applied and Computational Engineering, vol. 88, no. 1, pp. 105–114, 2024. DOI: 10.54254/2755-2721/88/20241643
[27] J. C. Ogeawuchi, A. C. Uzoka, A. A. Abayomi, and O. A. Agboola, “Innovations in Data Modeling and Transformation for Scalable Business Intelligence on Modern Cloud Platforms,” IRE Journals, vol. 5, no. 5, pp. 406–431, 2021.
[28] V. Garg, “Data Cataloging as a Catalyst for Effective Data Governance and Improved Decision-Making,” International Scientific Journal of Engineering and Management, vol. 3, no. 12, pp. 1–6, 2024. DOI: 10.55041/ISJEM0387
[29] Z. Ali, “AI-Ready Data Infrastructure: A Review of Zero-ETL, Declarative Pipelines, and Data Observability,” Cognizance Journal of Multidisciplinary Studies, vol. 5, no. 8, pp. 486–507, 2025. DOI: 10.47760/cognizance.2025.v05i08.019
[30] G. Govindarajan, “Data Quality and Automation in Modern Data Ecosystems,” International Journal of Advances in Engineering and Management, vol. 7, no. 4, pp. 228–239, 2025. DOI: 10.35629/5252-0704228239
[31] J. Jayabalan and D. Ramachandran, “Optimizing Enterprise Intelligence: A Strategic Framework for Integrating Salesforce with Modern Cloud-Based Data Warehouses for Real-Time Unified Analytics,” International Research Journal on Advanced Engineering Hub, vol. 3, no. 6, pp. 3010–3016, 2025. DOI: 10.47392/IRJAEH.2025.0443
[32] O. F. Enow, A. T. Ofoedu, E. Y. Gbabo, and P. E. Chima, “Advances in Real-Time Data Ingestion Strategies Using Fivetran, Rudderstack, and Open-Source ELT Tools,” Journal of Frontiers in Multidisciplinary Research, vol. 3, no. 1, pp. 500–516, 2022. DOI: 10.54660/.JFMR.2022.3.1.500-516
[33] A. Cahyadi and I. G. S. Rahayuda, “Data Processing Using Change Data Capture for Real-Time Data Synchronization,” Jurnal Elektronik Ilmu Komputer Udayana, vol. 12, no. 4, pp. 743–754, 2024.
[34] P. Dubey, “The Data Lakehouse: An Evolving Paradigm in Data Architecture,” International Journal of Computing and Engineering, vol. 7, no. 10, pp. 30–47, 2025. DOI: 10.47941/ijce.2958
[35] L. Alva, “AI-Augmented Real-Time Retail Analytics with Spark and Databricks,” World Journal of Advanced Engineering Technology and Sciences, vol. 15, no. 2, pp. 1024–1033, 2025. DOI: 10.30574/wjaets.2025.15.2.0631
[36] A. M. Kashimkar and Sudeepta, “The Role of Data Warehousing in Revolutionizing Banking and User Experience,” International Journal for Multidisciplinary Research, 2023. DOI: 10.36948/ijfmr.2023.v05i06.8890
[37] M. S. Dahibhate, “Enhancing Enterprise Data Orchestration Using Azure Data Factory,” International Scientific Journal of Engineering and Management, vol. 4, no. 5, pp. 1–9, 2025. DOI: 10.55041/ISJEM03578
[38] S. Gourishetti, “Performance Optimization in Distributed SQL Environments: A Comprehensive Analysis of Presto Query Engine,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 10, no. 6, pp. 241–253, 2024. DOI: 10.32628/CSEIT24106173
[39] V. Tundjungsari, K. Ernawati, and N. Mutia, “Public Health Surveillance to Promote Clean and Healthy Life Behaviours Using Big Data Approach (An Indonesian Case Study),” in Cyber-physical Systems and Digital Twins (REV 2019), M. Auer and K. Ram B., Eds., Lecture Notes in Networks and Systems, vol. 80. Cham, Switzerland: Springer, 2020. DOI: 10.1007/978-3-030-23162-0_69
