Structured Physics-Inspired Representations and dAta models for efficient Learning

Artificial Intelligence (AI) is set to revolutionize technology and society. While the fast improvement and adoption of AI comes with tremendous potential, it also brings significant challenges. The rise of so-called Large Language Models such as ChatGPT, which require tremendous computational resources, notably highlights the need for more efficient architectures to address sustainability and sovereignty concerns.In this context, the SPIRAL projectStructured Physics-Inspired Representations and dAta models for efficient Learningaims to create more interpretable, efficient, and targeted machine learning models by focusing on the role of structure. Indeed, despite advancements in Machine Learning research, the field has yet to fully understand how models process and build internal representations from structured data. SPIRAL seeks to close this gap by identifying how current architectures use structure in data and subsequently by developing better training protocols and tailored architectures, opening an alternative to the blind increase of model size and complexity.To achieve its goals, SPIRAL will take a dual, physics-inspired approach. First, it will establish an in-silico laboratory to explore the role of structure in data by developing a tunable model of synthetic structured data and conducting targeted experiments on modern machine learning architectures. Second, it will leverage insights from the statistical physics of disordered systems to understand the emergent structure in the solution landscape of asymmetric neural networks, ultimately proposing innovative architectures that make use of this internal structure to build meaningful internal representations of data.This interdisciplinary strategy, developed within the fast-growing Bocconi Institute for Data Science and Analytics, will provide new methods for building efficient models aligning with Europes goals of sustainability and technological independence.