TLDR Tech / AI
Multimodal Embedding & Reranker Models with Sentence Transformers
The v5.4 update of Sentence Transformers introduces multimodal embedding and reranker models that enable encoding and comparing texts, images, audio, and videos within a shared embedding space for tasks like cross-modal search and retrieval-augmented generation. These advanced models map different modalities into a consistent embedding space and assess relevance across mixed-modality pairs, though lower cross-modal similarity scores may occur due to modality clustering. Notably, multimodal rerankers provide high-quality scoring by evaluating individual pairs for more accurate retrieval results.
Read the full story