Self-Supervised Learning

Self-supervised learning is a type of machine learning where the model generates its own supervisory signal from the input data. It doesn’t rely on external labels.

Speaking with Hemal Shah

This is one of the most powerful ways to train our problems because we are exploiting the structure of our data to produce labels. We no longer need to rely on human labels.

Learned from Lilian Weng: https://www.youtube.com/watch?v=7l6fttRJzeU&t=318s&ab_channel=ArtificialIntelligence

This is a subset of unsupervised learning where we try to minimize our loss by trying replicate our output data to our input data.

Two methods:

1. Self-Prediction: Given an individual data sample, the task is to predict one part of the sample given the other part. Ex
   1. crop part of an image and train a model to predict the pixels cropped, or
   2. decolourize our original image, and predict a colourized image
2. Contrastive Learning: Given multiple data samples, the task is to predict the relationship among them. Ex:
   1. Embeddings?

Related

• Semi-Supervised Learning