A new theory of linear deep neural networks allows for the first statistical study of their ``weight space,'' providing insight into the features that allow such networks to generalize so well.