Regularized Methods

Yann LeCun doesn’t believe in contrastive methods, but rather regularized methods.

Regularized = “no negatives, but constrain the system so it can’t collapse”

Examples

• BYOL - Uses an online encoder + momentum target encoder, stop-gradient, and asymmetric predictor.
• SimSiam