SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING"

in this code, I've implemented sentiment analysis task with sst-2 dataset.

the below results are for 100 training samples:

cross entropy loss:

cross entropy + contrastive loss:

cross entropy heatmap on test dataset:

Accuracy on test dataset: 90.13

cross entropy + contrastive loss heatmap:

Accuracy on test dataset: 92.20

paper: https://arxiv.org/abs/2011.01403