School of Physics Colloquium

Speaker: Kamesh Krishnamurthy

 

Host: Audrey Sederberg

 

Abstract: I will give an overview of how tools from theoretical physics can be used to understand computations in neural networks. In the first part of the talk, I will focus on a fundamental question in neural computation: how do the microscopic mechanisms in neural networks influence their collective computation? In this context, I will present results on how a mechanism termed “gating” shapes computation in recurrent neural networks. Gating is not only ubiquitous in real neurons, but is also the central driver of performance gains in modern machine learning models. Among other benefits, I will show how gating robustly allows the generation of long timescales, and makes models more easily trainable by virtue of taming the gradients. Second, I will build on these insights about gating to address another salient issue in biophysics, neuroscience and machine learning: i.e. the challenges involved in implementing graded/continuous memories in without fine-tuning parameters. I will propose a general principle of "Frozen-Stabilisation”, which allows a wide variety of neural networks to self-organise to a critical state, allowing them to robustly implement continuous memory without the need for fine-tuning. This state also robustly exhibits a wide range of relaxation timescales – something that has been challenging to achieve theoretically. I will end the talk by laying out some broader problems in neural computation and machine learning upon which these versatile techniques from physics could be brought to bear.