Self-Supervised Learning
This is a subset of unsupervised learning where we try to minimize our loss by trying replicate our output data to our input data.
Lilian Weng https://www.youtube.com/watch?v=7l6fttRJzeU&t=318s&ab_channel=ArtificialIntelligence
Two methods:
- Self-Prediction: Given an individual data sample, the task is to predict one part of the sample given the other part. Ex
- crop part of an image and train a model to predict the pixels cropped, or
- decolourize our original image, and predict a colourized image
- Contrastive Learning: Given multiple data samples, the task is to predict the relationship among them. Ex:
- Embeddings?
hg hytfr dgtyfr tgyfrdtgyhgt frgyh