Sigmoid Loss for Language Image Pre-Training (SigLIP)

SigLIP is an improved version of CLIP which introduces sigmoid-based Contrastive Loss instead of the traditional softmax-based contrastive loss used in CLIP.

• From CLIP

Implementation details:

• Image Encoder is implemented with a Vision Transformer
• Text Encoder is implemented with a Transformer
• Sigmoid-based Contrastive Loss.

It uses separate image and text encoders to generate representations for both modalities.

Resources

• https://huggingface.co/docs/transformers/en/model_doc/siglip

Citation

(Zhai et al., 2023)

Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid Loss for Language Image Pre-Training. https://arxiv.org/pdf/2303.15343