Residual Network (ResNet)

Original Paper: https://arxiv.org/abs/1512.03385

Residual Network (ResNet) is a Convolutional Neural Network (CNN) architecture that overcame the “vanishing gradient” problem, making it possible to construct networks with up to thousands of convolutional layers, which outperform shallower networks.

https://www.run.ai/guides/deep-learning-for-computer-vision/pytorch-resnet

Deeper models are harder to optimize, because the propagation of the gradient is very hard.

So we can copy the learned layers, to make the propagation of gradients easier. This makes learning faster, and it works super well.