Publications

Benchmarking Neural Network Training Algorithms
We motivate, present, and justify our new AlgoPerf Training Algorithms benchmark.
Benchmarking Neural Network Training Algorithms
Late-Phase Second-Order Training
We show that performing a few costly but precise second-order steps can outperform first-order alternatives in wall-clock runtime.
Late-Phase Second-Order Training
Understanding Deep Learning Optimization via Benchmarking and Debugging
Ph.D. Thesis
Understanding Deep Learning Optimization via Benchmarking and Debugging
Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks
We introduce a visual and statistical debugger specifically designed for deep learning helping to understand the dynamics of neural network training.
Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
We empirically compared fifteen popular deep learning optimizers.
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
DeepOBS: A Deep Learning Optimizer Benchmark Suite
We provide a software package that drastically simplifies, automates, and improves the evaluation of deep learning optimizers.
DeepOBS: A Deep Learning Optimizer Benchmark Suite
Inverse generating function approach for the preconditioning of Toeplitz-block systems
We propose a new preconditioner for Toeplitz-block matrices based on the inverse generating function.
Inverse generating function approach for the preconditioning of Toeplitz-block systems
Methods and apparatus for calculating electromagnetic scattering properties of a structure and for reconstruction of approximate structures
We propose two new preconditioners for multi-level Toeplitz matrices.
Methods and apparatus for calculating electromagnetic scattering properties of a structure and for reconstruction of approximate structures
Approximations of Inverses of BTTB Matrices
We suggest several techniques to approximate the inverse of a BTTB matrix with the goal of designing preconditioners for linear systems of this form.
Approximations of Inverses of BTTB Matrices