emotions-dataset-distilbert-base-uncased

This model is a fine-tuned version of distilbert-base-uncased on an the emotion dataset. It achieves the following results on the evaluation set:

• Loss: 0.1613
• Accuracy: 0.9265
• F1: 0.9261

Model description

DistilBERT base uncased model available at distilbert-base-uncased

Intended uses & limitations

Text classification, sentiment classification

Training and evaluation data

Emotion dataset: Tweets categorized by 6 emotions - sadness, anger, joy, surprise, love, fear.

Dataset available at dair-ai/emotion

Training data: emotions train split, 16000 samples

Evaluation data: emotions test split, 2000 samples

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.7787	1.0	250	0.2564	0.9125	0.9139
0.2047	2.0	500	0.1869	0.9225	0.9234
0.1329	3.0	750	0.1705	0.923	0.9239
0.1045	4.0	1000	0.1618	0.924	0.9239
0.0866	5.0	1250	0.1613	0.9265	0.9261

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0