Showcasing Agile Safety Classifiers with Gemma | Google Codelabs

llmsafetyfine-tuningloraclassificationgemma

Abstraction: LoRA fine-tuning Gemma as a hate-speech safety classifier

Key points:

Uses LoRA (Low-Rank Adaptation) on Gemma 2B instruct to build a hate-speech classifier trained on the ETHOS dataset (YouTube/Reddit comments).
With only 200 training examples achieves F1=0.80 and ROC-AUC=0.78, matching/exceeding SOTA on the ETHOS leaderboard; full 800-example training yields F1=0.84, AUC=0.88.
Classification is done via next-token probability: the model predicts whether "Positive" or "Negative" is more probable as the next token after a formatted prompt.
Separator tokens between prompt sections improve format reliability over plain newlines.
Method described in "Towards Agile Text Classifiers for Everyone" (arXiv:2302.06541); applicable to diverse safety tasks with minimal data.

Connections: Gemma · Google · Parameter Efficient Tuning · Lora · Text Classification · AI Safety