Description
- Abstract:
- Deep learning builds functions by composing layers that depend on parameters. In a deep linear network (DLN), this composition is a product of matrices. The same product, and hence the same map, can be realized by many different tuples of matrices. In this thesis, we study the geometry of this non-uniqueness. We focus on balanced factorizations, in which the Gram matrices on the two sides of each intermediate layer agree. On the balanced manifold, the product map is a Riemannian submersion. Therefore, Euclidean gradient flow for the parameters descends to a gradient flow on the end-to-end matrix, with a metric determined by the depth. For a fixed matrix, the set of balanced factorizations realizing that matrix is a compact orbit. We use its volume to define an entropy on the space of end-to-end matrices. In the complex case, the balanced manifold is the zero-level set of a moment map, placing the DLN in a symplectic framework. For real DLNs, we study the effect of entropic regularization. When the loss depends only on the singular values, the loss and entropy combine to form a free energy. We reduce its gradient flow to a closed system for the singular values and use this reduction to identify equilibria for spectral energies. In the final part, we take the limit as the number of layers tends to infinity. The layer index becomes a point in an interval, and the finite-depth symmetry becomes a gauge symmetry. We formulate balancedness as the vanishing of a continuum moment map. After renormalization, we obtain continuum heat-flow equations describing the approach to balancedness. We also recover a metric with a Kähler potential on the space of end-to-end matrices, together with a renormalized entropy. Thus, the geometry created by depth persists in the infinite-depth limit.
- Notes:
- Thesis (Ph. D.)--Brown University, 2026
Citation
Kotwal, Tejas,
"Symmetries and Gradient Flows in the Deep Linear Network"
(2026).
Applied Mathematics Theses and Dissertations.
Brown Digital Repository. Brown University Library.
https://doi.org/10.26300/bd6x-0503
Relations
Has Parts:
-
Supplement for Symmetries and Gradient Flows in the Deep Linear Network
- Type: Archive
- Order: 1
Collection:
-
Applied Mathematics Theses and Dissertations
Theses and Dissertations for the Applied Mathematics department....