GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

Paper recorded by Signals 4 on 2026-10-21 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-10-21 on arXiv · recorded by Signals 4 on 2026-10-22

Category: cs.CL · 自然语言处理 · first seen 2026-10-22

Abstract

Vietnamese pre-trained encoders have improved rapidly, but the strongest ones are large, multilingual-vocabulary models or depend on external word segmentation, which makes them costly to deploy. We present GoiBERT, a family of two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. GoiBERT learns from two complementary teachers. A same-tokenizer Vietnamese e

Read on arXiv →