Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

A preprint showing that scaling LLM test-time compute optimally can be more effective than scaling model parameters. Both papers cite its result that, at matched total computation, a smaller model allowed to think longer outperformed one fourteen times its size on problems within its reach. This is the evidence for fungible cognition (P3): a lineage's budget buys its intelligence (Fig. 5b).

Cited in

both papers (Flagship: Who feeds on whom; Companion §3, Fig. 5)

Used for

P3P3; Fig. 5b

Strand

Frontier AI capabilities · Capability trends & benchmarks

Cite

@misc{snell2024scaling,
  title = {Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters},
  author = {Charlie Snell and Jaehoon Lee and Kelvin Xu and Aviral Kumar},
  year = {2024},
  url = {https://arxiv.org/abs/2408.03314},
}