Architecture of Connected Intelligence Systems: AI, Big Data, Cloud and Digital Platforms
DOI:
https://doi.org/10.63503/aisem.49Keywords:
Connected Intelligence, AI Architecture, Big Data Architecture, Cloud Computing, Digital Platforms, System Architecture, MLOps, Data LakeAbstract
The modern digital world is becoming a landscape of Connected Intelligence Systems – complex systems built on Artificial Intelligence (AI), Big Data analytics, Cloud Computing, and powerful Digital Platforms that provide functionality previously unknown across various industries. Such systems that provide real-time insights, robotic decision making and an incredibly customised user experience are the key to present-day innovation. Such systems have an architectural basis, and this chapter discusses it in detail. We explain the synergy of AI, providing the mental capability to learn and make a decision; the Big Data technologies, which are scaling-up of the necessary amounts of heterogeneous data to train and feed AI models; the Cloud Computing, which is the scalable and on-demand infrastructure to the data and AI models, and the Digital Platforms, which are the integrated environments and interfaces to users and applications. In the chapter, the definition of the basic architectural components, i.e., data ingestion pipelines, scalable storage systems, distributed structure of processing, AI/ML operationalisation (MLOps), API management, and microservices, is provided. We discuss various architectural patterns, their structures, and detailed diagrams, such as data lakes, data stream processing architectures, AI-as-a-Service, and hybrid/multi-cloud deployments. Additionally, we cover such aspects of architecture as scaling, security, data control, cost optimization, latency, and interoperability. Future directions, such as the transition to edge intelligence, quantum-AI integration, serverless, and the need for resilient, ethical, and explainable design to navigate the expanding complexity and reach of Connected Intelligence Systems, are also described in the chapter. The aim of this analysis is to provide background insight for architects, engineers, developers, and decision-makers who may want to design, implement, and benefit from these mighty intelligent systems.
References
[1] Raju, N., Quazi, F., & Kondle, P. (2024). AI and Machine Learning in Digital Modernization Transforming Industries for the Future. International Journal of Global Innovations and Solutions (IJGIS). https://doi.org/10.21428/e90189c8.2bd57c3a
[2] Ramasamy, M. (2025). Transforming Industries: The Impact of AI-Driven Network Engineering and Cloud Infrastructure. European Journal of Computer Science and Information Technology, 13(26), 123–
[3] 131. https://doi.org/10.37745/ejcsit.2013/vol13n26123131
[4] Shandilya, S., Sangwan, S., & Yadav, R. (2025). Artificial Intelligence and Big Data. Edward Elgar. https://doi.org/10.4337/9781803925677
[5] Mahanty, R., & Mahanti, P. K. (2019). Unleashing Artificial Intelligence onto Big Data (pp. 2099–2114). Igi Global. https://doi.org/10.4018/978-1-5225-7501-6.ch108
[6] Thanh, L. T., Dat, N. Q., Hoang, V., & Hieu, T. H. H. (2024). The Important of Big Data in Machine Learning. Journal of Basic and Applied Research International, 30(6), 73–79. https://doi.org/10.56557/jobari/2024/v30i68952
[7] Adari, V. K. (2025). Architectural Frameworks for AI-Enhanced Cloud Systems in Large-Scale Enterprise Deployments Vijay Kumar Adari Cognizant Technology Solutions, USA. International Journal of Computer Technology and Electronics Communication, 8(6). https://doi.org/10.15680/ijctece.2025.0806024
[8] Anumula, S. R. (2023). Enterprise Architecture for Real-Time Intelligence in Distributed Environments. International Journal of Computer Technology and Electronics Communication,
[9] 6(4). https://doi.org/10.15680/ijctece.2023.0604009
[10] Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6), 599-616.
[11] Nielsen, F. (2016). The MapReduce Paradigm (pp. 147–160). Springer. https://doi.org/10.1007/978-3-319-21903-5_6
[12] Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107-113.
[13] Bajaber, F., Elshawi, R., Batarfi, O., Altalhi, A., Barnawi, A., & Sakr, S. (2016). Big Data 2.0 Processing Systems: Taxonomy and Open Challenges. Journal of Grid Computing, 14(3), 379–405. https://doi.org/10.1007/s10723-016-9371-1
[14] Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., ... & Zaharia, M. (2010). A view of cloud computing. Communications of the ACM, 53(4), 50-58.
[15] Arif, Z., & Zeebaree, S. (2024). Distributed Systems for Data-Intensive Computing in Cloud Environments: A Review of Big Data Analytics and Data Management. The Indonesian Journal of Computer Science,
[16] 13(2). https://doi.org/10.33022/ijcs.v13i2.3819
[17] Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., & Khan, S. U. (2015). The rise of "big data" on cloud computing: Review and open research issues. Information Systems, 47, 98-115.
[18] Newman, S. (2015). Building microservices: Designing fine-grained systems. O'Reilly Media.
[19] Burns, B., & Beda, J. (2019). Kubernetes: Up and running: Dive into the future of infrastructure. O'Reilly Media.
[20] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., ... & Young, M. (2015). Hidden technical debt in machine learning systems. In Advances in Neural Information Processing Systems (pp. 2503-2511).
[21] Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022). MLOps: A taxonomy and a methodology. IEEE Access, 10, 63606-63618.
[22] Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50-60.
[23] Chen, L., Zhang, H., & Wang, M. (2023). Connected intelligence systems: Architectural patterns for AI-driven digital transformation. IEEE Transactions on Systems, Man, and Cybernetics, 53(8), 4892-4905.
[24] Gubbi, J., Buyya, R., Marusic, S., & Palanisami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems, 29(7), 1645-1660.
[25] Katal, A., Wazid, M., & Goudar, R. H. (2013). Big data: Issues, challenges, tools and good practices. In 2013 Sixth International Conference on Contemporary Computing (pp. 404-409). IEEE.
[26] Marz, N., & Warren, J. (2015). Big Data: Principles and best practices of scalable real-time data systems. Manning Publications.
[27] Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., & Tzoumas, K. (2015). Apache Flink: Stream and batch processing in a single engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 36(4), 28-38.
[28] Inmon, W., Linstedt, D., & Levins, M. (2016). Data architecture: A primer for the data scientist. Academic Press.
[29] Chen, Y., Liu, X., & Rodriguez, P. (2020). AI-enhanced clinical decision support systems: Architecture and implementation patterns. Journal of Medical Systems, 44(12), 1-15.
[30] Kratzke, N., & Quint, P. C. (2017). Understanding cloud-native applications after 10 years of cloud computing-a systematic mapping study. Journal of Systems and Software, 126, 1-16.
[31] Skelton, M., & Pais, M. (2019). Team topologies: Organizing business and technology teams for fast flow. IT Revolution Press.
[32] Roberts, M., & Chapin, J. (2017). What is serverless? O'Reilly Media Online.
[33] Parker, G. G., Van Alstyne, M. W., & Choudary, S. P. (2016). Platform revolution: How networked markets are transforming the economy and how to make them work for you. W. W. Norton & Company.
[34] Jacobson, D., Brail, G., & Woods, D. (2011). APIs: A strategy guide. O'Reilly Media.
[35] Hohpe, G., & Woolf, B. (2004). Enterprise integration patterns: Designing, building, and deploying messaging solutions. Addison-Wesley Professional.
[36] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... & Zheng, X. (2016). TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org.
[37] Ahmed Khan, N. (2025). AI Infrastructure for Training and Inference. Technix International Journal for Engineering Research, 12(5). https://doi.org/10.56975/tijer.v12i5.158264
[38] Hyder, M., & Nafees, G. M. (2023). HARNESSING DEEP LEARNING AND NATURAL LANGUAGE
[39] PROCESSING (NLP). Noble Science. https://doi.org/10.52458/9789388996747.nsp2023.eb.ch-02
[40] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (pp. 4171-4186).
[41] Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., ... & Hassabis, D. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676), 354-359.
[42] Dhillon, V., Metcalf, D., & Hooper, M. (2025). The Era of Artificial Intelligence (pp. 1–52). Apress. https://doi.org/10.1007/979-8-8688-1402-0_1
[43] Dulam, A. (2025). Enhancing data processing with Apache spark: A technical deep dive. World Journal of Advanced Engineering Technology and Sciences, 15(3), 1279–1284. https://doi.org/10.30574/wjaets.2025.15.3.0910
[44] Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster computing with working sets. In Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing (pp. 10-10)
[45] Nalla, N. (2025). Real-Time Data Processing Pipelines: Enhancing Decision Intelligence with Apache Spark and Kafka. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 11(2), 227–231. https://doi.org/10.32628/cseit25112356
[46] Chennupati, S. (2025). Scalable Cloud Architectures for Real-Time AI: Dynamic Resource Allocation for Inference Optimization. Journal of Computer Science and Technology Studies, 7(3), 690–700. https://doi.org/10.32996/jcsts.2025.7.3.79
[47] Syed Sulaiman, M. R. (2025). Infrastructure Optimization for AI Workloads: A Holistic Approach to Cloud Performance. Journal of International Crisis and Risk Communication Research, 387–
[48] 395. https://doi.org/10.63278/jicrcr.vi.3376
[49] Ahmad Fawad, E. A. (2023). Efficient Workload Allocation and Scheduling Strategies for AI-Intensive Tasks in Cloud Infrastructures. Power System Technology, 47(4), 82–102. https://doi.org/10.52783/pst.160
[50] Natta, P. (2025). AI-Powered API Management: Intelligent traffic routing and adaptive scaling. World Journal of Advanced Research and Reviews, 26(1), 2758–2764. https://doi.org/10.30574/wjarr.2025.26.1.1353
[51] Jain, P. (2025). Identity and Access Management in the Cloud. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 11(2), 1528–
[52] 1535. https://doi.org/10.32628/cseit25112523
[53] Mostafa, A. M., Rushdy, E., Medhat, R., & Hanafy, A. (2023). An identity management scheme for cloud computing: Review, challenges, and future directions. Journal of Intelligent & Fuzzy Systems, 45(6), 11295–11317. https://doi.org/10.3233/jifs-231911
[54] Ashok, S. (2025). Efficient ETL workflows for big data: Handling massive datasets at scale. International Journal of Science and Research Archive, 14(2), 1567–1574. https://doi.org/10.30574/ijsra.2025.14.2.0531
[55] Kotla, S. (2025). Technical review: Transforming data into intelligence. World Journal of Advanced Engineering Technology and Sciences, 15(3), 864–871. https://doi.org/10.30574/wjaets.2025.15.3.0999
[56] Yadav, S. B., & Mewani, D. (2026). LEAIM: A Layered Enterprise Architecture Model for Scalable and Governed Integration of Artificial Intelligence in Distributed Systems. International Journal of Artificial Intelligence, Data Science and Machine Learning, 7, 175–182. https://doi.org/10.63282/3050-9262.ijaidsml-v7i1p130
[57] Johnson, R., Smith, A., & Williams, K. (2024). Quantum machine learning for connected intelligence systems. Nature Quantum Information, 10(1), 15-28.
[58] Quantum Computing Research Consortium. (2024). Hybrid quantum-classical architectures for large-scale AI systems. Science Advances, 10(8), eabk1234.
[59] Autonomous Systems Initiative. (2025). Self-healing distributed systems: Architecture patterns and implementation guidelines. Autonomous Computing Research Consortium.
[60] Spatial Intelligence Research Lab. (2025). Immersive computing platforms for enterprise intelligence systems. IEEE Computer Graphics and Applications, 45(2), 67-78.
[61] Neuromorphic Computing Alliance. (2024). Brain-inspired computing architectures for edge intelligence. Nature Electronics, 7(4), 289-302.
[62] Green Computing Initiative. (2025). Carbon-aware computing frameworks for sustainable AI systems. Nature Sustainability, 8(3), 234-247.
[63] Bio-Computing Research Institute. (2025). DNA storage systems for connected intelligence architectures. Nature Biotechnology, 43(2), 145-158.
