Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search

Paper recorded by Signals 4 on 2026-09-01 in cs.LG. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-01 on arXiv · recorded by Signals 4 on 2026-09-02

Category: cs.LG · 机器学习 · first seen 2026-09-02

Abstract

Optimal hyperparameter scaling laws describe how the best hyperparameters for large language model (LLM) training change with model and data scale, enabling practitioners to predict optimal configurations at production scales without expensive large-scale tuning. However, estimating these scaling laws conventionally requires exhaustive grid searches over thousands of training runs, consuming enorm

Read on arXiv →