Roberta

entities · 1 notes linked

Related: Bert · Knowledge Distillation · Transformers · Large Language Models

Notes

• WaLDORf: Wasteless Language-model Distillation On Reading-comprehension — Hybrid convolutional-transformer model via knowledge distillation for fast NLU inference