Overview
Moderation has to work in more than one language, but labelled toxicity data mostly doesn't. This project fine-tunes distilbert-base-multilingual-cased to separate toxic from non-toxic comments, then stretches that ability beyond the language it was trained on.
How it works
- Preprocessing: comments tokenised with the DistilBERT tokenizer, padded and truncated to a fixed length for efficient batching.
- Two-phase fine-tuning: train on English first, then adapt multilingually, using a weighted loss to counter heavy class imbalance.
- Serving: an end-to-end training and evaluation pipeline, plus a small web demo so anyone can try it live.
Results
- English: Accuracy 0.857 · ROC-AUC 0.942
- Multilingual: Accuracy 0.818 · ROC-AUC 0.875
What I learned
That a model can carry a learned judgement across a language barrier it was never explicitly taught, and that fighting class imbalance honestly (weighted loss, the right metric) matters more than chasing raw accuracy.