The above picture is taken from Bruno Gavranović's position paper Categorical Deep Learning is an Algebraic Theory of All Architectures.
Towards a Predictive Science of Deep Learning
(See Structures of Learning, a GitHub repository where I document my research progress, including questions, constructions, and experiments)
I am interested in the mathematical foundations of AI, with the goal of contributing to mathematics that help us explain phenomena like emergence. My motivation is to contribute to theoretical work that aims to explain the behavior of learning systems through useful mathematical abstractions. The goal is for these abstractions to become not only descriptive and prescriptive, but eventually predictive: to lead to falsifiable predictions that can either provide new insight into deep learning or force us to refine the abstractions themselves, bringing them into closer alignment with the phenomena they are meant to describe. I think this would be great, not only in explaining currently mysterious observations, but developing safer and more aligned models.
We are currently in a steam engine era of deep learning: we can build powerful systems, but we lack a rigorous, falsifiable theory to explain why they work. I want to help in moving the field from heuristic engineering to a core part of science. In the same way that physics has many theories concerned with different phenomena, questions, and scales, I believe there will be multiple theories of deep learning. I am currently interested in describing generalization and understanding how the various components of deep learning (such as priors, architectures, training procedures, optimizers, and loss functions) relate to one another.
For many of these questions, category theory provides a particularly well-suited mathematical language, so I am spending much of my time investigating categorical approaches to deep learning. I am also interested in neuroalgebraic geometry and, eventually, in studying the connections between different emerging theories of deep learning themselves. This is yet another direction for which I believe category theory is especially well suited.
Related writing