Hugging Face's logo

LilaBoualili
/
colbert-distilbert-margin_mse-T2-msmarco-encoder-only

Feature Extraction
Transformers
PyTorch
English
distilbert
dense-retrieval
knowledge-distillation
text-embeddings-inference
Model card Files
xet
 Community
1

Margin-MSE Trained ColBERT encoder model only

We provide the encoder model of the complete retrieval trained DistilBert-based ColBERT model. This model is trained with Margin-MSE using a 3 teacher BERT_Cat (concatenated BERT scoring) ensemble on MSMARCO-Passage, for more details check the full model card.

This encoder-only model is used as the oracle for distilling term topic embeddings in our ECIR'23 paper.

Downloads last month
4
Inference Providers NEW
Feature Extraction
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train LilaBoualili/colbert-distilbert-margin_mse-T2-msmarco-encoder-only