Brown University
  1. Top
  2. ...
  3. Theses and Dissertations
  4. bdr:dhf4pcyd

The Deep Linear Network – Dynamics, Riemannian Geometry and Overparametrization

Full Metadata

Overview

Year:
2023
Contributor:
Veraszto, Zsolt (creator)
Menon, Govind (Advisor)
Cohen, Nadav (Reader)
Darbon, Jerome (Reader)
Brown University. Department of Applied Mathematics (sponsor)
Genre:
theses
Subject:
generalizability
gradient flows
implicit regularization
rieamannian geometry
matrix completion
deep linear network
Extent:
ix, 76 p.

Files

Description

Abstract:
The deep linear network (DLN) is a model for implicit regularization in gradient based optimization of overparametrized learning architectures. Training the DLN corresponds to a Riemannian gradient flow, where the Riemannian metric is defined by the architecture of the network and the loss function is defined by the learning task. We extend this geometric framework, obtaining explicit expressions for the volume form, including the case when the network has infinite depth. We investigate the link between the Riemannian geometry and the training asymptotics for matrix completion with rigorous analysis and numerics. We develop a stochastic model for training and report our numerical findings. We propose that under small initialization, implicit regularization is a result of bias towards high state space volume.
Notes:
Thesis (Ph. D.)--Brown University, 2023

Content

Citation

Veraszto, Zsolt, "The Deep Linear Network – Dynamics, Riemannian Geometry and Overparametrization" (2023). Applied Mathematics Theses and Dissertations. Brown Digital Repository. Brown University Library. https://repository.library.brown.edu/studio/item/bdr:dhf4pcyd/

Relations

Collection: