Deep Learning is Not So Mysterious or Different

This paper argues that anomalous generalization behaviors like benign overfitting and double descent are not unique to deep neural networks and can be understood through soft inductive biases within established generalization frameworks. It highlights that deep learning's distinction lies more in...