Divergences


Feeling closer in different ways
Divergences | romaingd Romain Girard blog personal website github.io github pages data science datascience healthcare medicine cardiology heart

Adam's convergence proof is flawed

Adam, a popular optimization algorithm in deep learning, does not correctly converge in all convex settings. This is linked to the learning rate that is wrongly assumed to be non-increasing.

On the convergence of Adam and beyond, Reddi et al., 2018


10 minute read