Does the Geometry of the Data Control the Geometry of Neural Predictions? (Student Abstract)

No. 11: IAAI-22, EAAI-22, AAAI-22 Special Programs and Special Track, Student Papers and Demonstrations

Volume

Proceedings of the AAAI Conference on Artificial Intelligence, 36

AAAI Student Abstract and Poster Program

This paper studies the over-parameterization of deep neural networks using the Fisher Information Matrix from information geometry. We identify several surprising trends in the structure of its eigenspectrum, and how this structure relates to the eigenspectrum of the data correlation matrix. We identify how the eigenspectrum relates to the topology of the predictions of the model and develop a "model reduction'' method for deep networks. This ongoing investigation hypothesizes certain universal trends in the FIM of deep networks that may shed light on their effectiveness.

10.1609/aaai.v36i11.21602