Open for all UnRAVeL Members: Weekly Seminar by RTG Eddy: On Neural Differential Equations
Thursday, October 14, 2021, 10:30am
Location: Because of the changeable situation during the COVID-19 pandemic, please consult the webpage for up to date location or, in the event that an online seminar is necessary, the access data.
Speaker: Patrick Kidger (Oxford University (U.K.))
Neural Differential Equations (NDEs) demonstrate that neural networks and differential equations are two sides of the same coin. Traditional parameterised differential equations are a special case. Many popular neural network architectures (e.g. residual networks, recurrent networks, StyleGAN2, coupling layers) are discretisations. By treating differential equations as a learnt component of a differentiable computation graph, then NDEs extend current physical modelling techniques whilst integrating tightly with current deep learning practice.
NDEs offer high-capacity function approximation, strong priors on model space, the ability to handle irregular data, memory efficiency, and a wealth of available theory on both sides. They are particularly suitable for tackling dynamical systems, time series problems, and generative problems.
This talk will offer a dedicated introduction to the topic, with examples including neural ordinary differential equations (e.g. to model unknown physics), neural controlled differential equations (“continuous recurrent networks”; e.g. to model functions of time series), and neural stochastic differential equations (e.g. to model time series themselves). If time allows I will discuss other recent work, such as novel numerical neural differential equation solvers. This talk includes joint work with Ricky T. Q. Chen, Xuechen Li, James Foster, and James Morrill.