Paper recorded by Signals 4 on 2026-09-30 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-09-30 on arXiv · recorded by Signals 4 on 2026-10-01
Category: cs.LG · 机器学习 · first seen 2026-10-01
Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31.1% by August. Unlike synthetic data or model-collapse setups, this *wild* AI text comes from many models, is written for human readers, and arrives unlabeled in pretra