Gastvortrag: Sebastian Kassing: On the existence of minimizers in neural network optimisation landscapes
Dienstag, 23. Mai 2023, 10 Uhr
Ort: RWTH Aachen University, Informatikzentrum - Ahornstr. 55, Erweiterungsgebäude E3, Raum 9u10
Vortragender: Sebastian Kassing
We prove existence of global minima in the loss landscape for the approximation of continuous target functions using shallow feedforward artificial neural networks with ReLU activation. This property is one of the fundamental artifacts separating ReLU from other commonly used activation functions and has important consequences for the optimization dynamics. We propose a kind of closure of the search space so that in the extended space minimizers exist. In a second step, we show under mild assumptions that the newly added functions in the extension perform worse than appropriate representable ReLU networks. This then implies that the optimal response in the extended target space is indeed the response of a ReLU network. This is joint work with Steffen Dereich (Uni Münster) and Arnulf Jentzen (Uni Münster, The Chinese University of Hong Kong Shenzhen).