Team project
Fine-tuning BERT and DistilBERT with LoRA
Four automated training runs on the Stanford Natural Language Inference dataset, to compare fine-tuning methods.
DistilBERT test accuracy
- Period
- Nov 2025 – Jan 2026
- Type
- Team project
- Course
- Advanced Machine Learning for Natural Language Processing
- University
- IT University of Copenhagen
- Models
- Methods
- Low-rank adaptation (LoRA), partial fine-tuning
- Dataset
- Stanford Natural Language Inference
Method
- Automated four training runs on the Stanford Natural Language Inference dataset
- Logged accuracy, training time and peak GPU memory
- Compared low-rank adaptation with partial fine-tuning