From Spin Glasses to Transformers: A Statistical Physics Perspective on Machine Learning

Giorno 3 settembre 2026, con inizio alle ore 11:30, presso l'Aula T del DFA, su invito del Prof. V. C. Latora, il Dott. Fabrizio Boncoraglio (EPFL, Lausanne, Switzerland) terrà un seminario dal titolo From Spin Glasses to Transformers: A Statistical Physics Perspective on Machine Learning.

Tutte le persone interessate sono invitate a partecipare.

Abstract. Modern machine learning relies on models with enormous numbers of parameters trained on high-dimensional datasets. Understanding the collective phenomena that arise in this regime is a central theoretical challenge. Statistical physics was developed to explain how macroscopic properties emerge from many interacting degrees of freedom in the thermodynamic limit, making its concepts and tools naturally suited to modern machine learning. In this seminar, I will introduce this connection through the Ising model and the Sherrington-Kirkpatrick spin glass. These paradigmatic systems illustrate how complex energy landscapes arise and how they can be analyzed using mean-field ideas and techniques such as the replica method. The same formalism applies naturally to learning problems: model parameters play the role of interacting degrees of freedom, the loss function defines an energy landscape, and the training dataset acts as a source of quenched disorder. I will illustrate this correspondence through the perceptron, the simplest neural network, which already displays a rich phase diagram and sharp transitions between distinct learning regimes. Finally, I will turn to self-attention and transformers, the architectures at the core of today’s state-of-the-art large language models. I will discuss recent theoretical advances showing how analytically tractable high-dimensional models of simple transformers can characterize what is learned from data, the spectra of learned weight matrices, and the emergence of generalization curves and scaling laws.