Machine Learning Seminar presentation
Topic: Approximating Hessians for Neural Networks
Speaker: Maciej Skorski, Faculty of Science, Technology and Medicine; University of Luxembourg
Time: Wednesday, 2021.06.30, 10:00 CET
How to join: Please contact Jakub Lengiewicz
Second-order optimization techniques have been proven successful, but cannot be fully leveraged for neural networks due to infeasibility of hessian calculations. I will discuss an approximated way of computing the hessian for neural networks, which emerges from a careful analysis of the hessian chain rule and activation functions.