Self-Supervised Learning §

This is a subset of unsupervised learning where we try to minimize our loss by trying replicate our output data to our input data.

Lilian Weng https://www.youtube.com/watch?v=7l6fttRJzeU&t=318s&ab_channel=ArtificialIntelligence

Two methods:

1. Self-Prediction: Given an individual data sample, the task is to predict one part of the sample given the other part. Ex
   1. crop part of an image and train a model to predict the pixels cropped, or
   2. decolourize our original image, and predict a colourized image
2. Contrastive Learning: Given multiple data samples, the task is to predict the relationship among them. Ex:
   1. Embeddings?

hg hytfr dgtyfr tgyfrdtgyhgt frgyh