{"full_name":"huggingface/sentence-transformers","owner":"huggingface","name":"sentence-transformers","shard":"1","url":"https://github.com/huggingface/sentence-transformers","description":"State-of-the-Art Embeddings, Retrieval, and Reranking","homepage":"https://www.sbert.net","topics":[],"tag_pages":[],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/25720743?v=4","created_at":"2019-07-24T10:53:51Z","pushed_at":"2026-11-13T10:51:22Z","metrics":{"stars":19285,"forks":3024,"watchers":147,"issues":2615,"prs":1473,"releases":74,"issues_open":1275,"issues_closed":1340,"prs_open":52,"prs_closed":308,"prs_merged":1113,"discussions":0},"growth":{"day":{"stars":0,"forks":2,"base_day":"2026-11-15"}},"history":{"dates":["2026-11-10","2026-11-11","2026-11-12","2026-11-13","2026-11-14","2026-11-15","2026-11-16"],"stars":[19274,19280,19281,19282,19283,19285,19285],"forks":[3017,3018,3020,3020,3022,3022,3024],"watchers":[null,147,147,147,147,147,147],"issues_open":[null,1272,1271,1272,1273,1273,1275],"prs_open":[null,44,46,46,47,49,52]},"boards":{},"same_owner":[{"full_name":"huggingface/transformers","name":"transformers","shard":"1","boards":{"total":54}},{"full_name":"huggingface/OpenEnv","name":"OpenEnv","shard":"1","boards":{}},{"full_name":"huggingface/accelerate","name":"accelerate","shard":"1","boards":{}},{"full_name":"huggingface/agents-course","name":"agents-course","shard":"1","boards":{}},{"full_name":"huggingface/alignment-handbook","name":"alignment-handbook","shard":"1","boards":{}},{"full_name":"huggingface/diffusers","name":"diffusers","shard":"1","boards":{}},{"full_name":"huggingface/ml-intern","name":"ml-intern","shard":"1","boards":{}},{"full_name":"huggingface/parler-tts","name":"parler-tts","shard":"1","boards":{}},{"full_name":"huggingface/pytorch-image-models","name":"pytorch-image-models","shard":"1","boards":{}},{"full_name":"huggingface/skills","name":"skills","shard":"1","boards":{}},{"full_name":"huggingface/smolagents","name":"smolagents","shard":"1","boards":{}},{"full_name":"huggingface/speech-to-speech","name":"speech-to-speech","shard":"1","boards":{}},{"full_name":"huggingface/transformers.js","name":"transformers.js","shard":"1","boards":{}},{"full_name":"huggingface/trl","name":"trl","shard":"1","boards":{}}],"issues":[{"number":695,"title":"Multilingual Information Retrieval (MS-Marco Bi-Encoders)","comments":62,"created_at":"2021-01-15T20:04:16Z"},{"number":1011,"title":"EN-DE MS-Marco","comments":43,"created_at":"2021-06-17T20:44:45Z"},{"number":350,"title":"Fine Tuning Models","comments":34,"created_at":"2020-08-11T07:46:54Z"},{"number":1795,"title":"Memory leak in SentenceTransformer.encode during the first ~10000 predictions","comments":32,"created_at":"2022-12-23T15:47:35Z"},{"number":2922,"title":"Please future prove `clean_up_tokenization_spaces` ","comments":28,"created_at":"2024-09-03T21:12:18Z"},{"number":617,"title":"Different results each model run","comments":27,"created_at":"2020-12-11T19:07:28Z"},{"number":3072,"title":"In Batch Negatives","comments":26,"created_at":"2024-11-19T13:39:42Z"},{"number":671,"title":"Require some neutral  network construction guide to use Asym","comments":26,"created_at":"2021-01-07T06:13:40Z"}],"discussions":[],"latest_release":{"name":"v6.3.0 - Token-pooling-aware training and `kernels` support","tag":"v6.3.0","published_at":"2026-11-11T14:45:45Z","body":"Train multi-vector models for smaller document representations with token-pooling-aware and Matryoshka training. This release also adds optional kernels for MaxSim, Ward clustering, and transformer backbones, native AWS Neuron inference, faster utilities, and fixes across encoding, training, and evaluation.\r\n\r\n> [!TIP]\r\n> I share much more about training multi-vector models with token-pooling-aware losses in my new [Token Pooling-Aware Training for Multi-Vector Embedding Models](https://huggingface.co/blog/token-pooling) blogpost. Give it a read!\r\n\r\nInstall with:\r\n\r\n```bash\r\n# Inference only, use one of:\r\npip install \"sentence-transformers==6.3.0\"\r\npip install \"sentence-transformers[kernels]==6.3.0\"\r\npip install \"sentence-transformers[openvino]==6.3.0\"\r\n\r\n# Multimodal dependencies:\r\npip install \"sentence-transformers[image]==6.3.0\"\r\npip install \"sentence-transformers[audio]==6.3.0\"\r\npip install \"sentence-transformers[video]==6.3.0\"\r\n\r\n# Training and merging support:\r\npip install \"sentence-transformers[train]==6.3.0\"\r\npip install \"sentence-transformers[merge]==6.3.0\"\r\n\r\n# Or combine as needed:\r\npip install \"sentence-transformers[train,image,kernels]==6.3.0\"\r\n```\r\n\r\n## Token-pooling-aware training (#4286)\r\n\r\n### Train for fewer document embeddings\r\n\r\nToken pooling for multi-vector models merges similar document token embeddings to reduce the embeddings stored and scored during retrieval. With `MultiVectorTokenPoolingLoss`, you can now train the encoder for this compression so its document embeddings retain more useful information after pooling.\r\n\r\nWrap a retrieval loss with the pooling configurations you intend to deploy:\r\n\r\n```python\r\nfrom sentence_transformers import MultiVectorEncoder\r\nfrom sentence_transformers.multi_vector_encoder.losses import (\r\n    CachedMultiVectorMultipleNegativesRankingLoss,\r\n    MultiVectorTokenPoolingLoss,\r\n)\r\nfrom sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling\r\n\r\nmodel = MultiVectorEncoder(\"answerdotai/ModernBERT-base\")\r\npooling = HierarchicalTokenPooling(pool_factor=4)\r\nbase_loss = CachedMultiVectorMultipleNegativesRankingLoss(model, mini_batch_size=64)\r\nloss = MultiVectorTokenPoolingLoss(\r\n    model,\r\n    base_loss,\r\n    token_poolings=[None, pooling],\r\n    weights=[0.5, 0.5],\r\n)\r\n```\r\n\r\nPass `loss` to `MultiVectorEncoderTrainer` to train on the average of the unpooled and factor-4 retrieval losses. Because both variants reuse the same encoder embeddings, training adds clustering and scoring work without another encoder pass.\r\n\r\nA matched experiment compared two ModernBERT-base models with fresh 128-dimensional projections, trained on the same 120,000 MS MARCO triplets with seed 42:\r\n\r\n| Training | Unpooled nDCG@10 | Factor-4 nDCG@10 | Unpooled score retained |\r\n| --- | ---: | ---: | ---: |\r\n| Ordinary | 51.84 | 49.03 | 94.6% |\r\n| Pooling-aware | 52.39 | 51.23 | 97.8% |\r\n\r\nFactor-4 pooling reduces the average document representation from 193.22 to 48.68 embeddings, while the pooling-aware model retains 97.8% of its unpooled score.\r\n\r\nTo compress documents with the trained example model, pass a pooler to `encode_document` or set the runtime default with `MultiVectorEncoder(..., token_pooling=...)`. By default, pooling leaves query embeddings unchanged:\r\n\r\n```python\r\nfrom sentence_transformers import MultiVectorEncoder\r\nfrom sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling\r\n\r\nmodel = MultiVectorEncoder(\"tomaarsen/mve-ModernBERT-base-token-pooling\")\r\nqueries = model.encode_query([\"Which planet is known as the Red Planet?\"])\r\ndocuments = model.encode_document(\r\n    [\"Mars has a reddish surface.\", \"Jupiter is the largest planet.\"],\r\n    token_pooling=HierarchicalTokenPooling(pool_factor=4),\r\n)\r\nscores = model.similarity(queries, documents)\r\n```\r\n\r\nSee the [blogpost](https://huggingface.co/blog/token-pooling) for the method and experiments, and the [training guide](https://sbert.net/examples/multi_vector_encoder/training/token_poolin","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-16T09:23:14Z","history_start":"2026-09-06"}