Machine Learning Seminar presentation

Topic: Approximating Hessians for Neural Networks

Speaker: Maciej Skorski, Faculty of Science, Technology and Medicine; University of Luxembourg

Time: Wednesday, 2021.06.30, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Second-order optimization techniques have been proven successful, but cannot be fully leveraged for neural networks due to infeasibility of hessian calculations. I will discuss an approximated way of computing the hessian for neural networks, which emerges from a careful analysis of the hessian chain rule and activation functions.

Additional material: