Scaling up compute and data reliably drives down next-token prediction loss. What’s less clear is how that loss maps onto real-world capabilities. Most scaling laws tell you how well a model predicts the next word in a dataset, not whether it can write exploit code, synthesize a novel protein, or pass a bar exam. In this blog, I’ll walk through some naive attempts at chaining scaling laws together to predict real-world performance, and show why those predictions are a lot less reliable than people claim.

Scaling laws aren’t as predictive as you think

Many claim that if you look at these scaling laws and extrapolate, it’s clear that we’ll see massive gains by simply scaling up existing models. Scaling laws are quantitative relationships that relate model inputs (data and compute) to model outputs (how well the model predicts the next word). They’re created by plotting model inputs and outputs at various levels on a graph.

But using scaling laws to make predictions isn’t as easy as people claim. To begin with, most scaling laws (Kaplan et al, Chinchilla, and Llama) output how well models predict the next word in a dataset, not how well models perform at real-world tasks. This 2023 blog post by a popular OpenAI researcher describes how “it’s currently an open question if surrogate metrics [like loss] could predict emergence … but this relationship has not been well-studied enough …”

Chaining two approximations together to make predictions

To fix the above issue, you can fit a second scaling law that quantitatively relates upstream loss to real-world task performance, and then chain the two scaling laws together to make predictions of real-world tasks.

Loss = f(data, compute)
Real world task performance = g(loss)
Real world task performance = g(f(data, compute))

In 2024, Gadre et al and Dubey et al introduced scaling laws of that sort. Dubey uses this method of chained laws for prediction and claims that its predictive ability “extrapolates [well] over four orders of magnitude” for the Llama 3 models.

However, work on these second scaling laws is nascent, and with so few data points, choosing a fitting function becomes a highly subjective judgment call. For example, in the figures below, Gadre assumes an exponential relationship across an average of many tasks (top) across many tasks, and Dubey assumes a sigmoidal relationship for a single task (bottom) for the ARC-AGI task. These scaling laws are also highly task-dependent (see Mosaic).

Without a strong hypothesis for the relationship between loss and real-world task accuracy, we don’t have a strong hypothesis for the capabilities of future models.

A shoddy attempt at using chained scaling laws for predictions

What would happen if we were to use some of these naively chained scaling laws to perform predictions anyways? Note the goal here is to show how one can use some set of scaling laws (Gadre) to obtain a prediction, rather than obtain a detailed prediction itself.¹

To start, we can use publicly available information to estimate the data and compute inputs² for the next few model releases. We can take announcements of the largest data center build-outs, estimate the expected resulting compute based on their GPU capacity, and map them onto each successive model generation.³

Then, we can use scaling laws to estimate the amount of data these clusters would need. The largest cluster estimate from the Institute for Progress would ideally train on 770T tokens to minimize loss according to the scaling laws we’re using, a few times larger than the size of the indexed web.⁴ That seems challenging but feasible, so let’s just use that for now.

See link to Google Sheet of data and compute estimates⁵

Finally, we can plug these inputs into chained scaling laws and extrapolate. The right plot is what we’re interested in. It shows real-world task performance on the vertical axis, plotted against the inputs of data and compute on the horizontal axes. Blue points represent performance on existing models (GPT-2, GPT-3, etc), while red points are projections for the next scale-ups (GPT-5, GPT-6, GPT-7, etc):

See link to Github repository for chained Gadre scaling laws and resulting plots

Using these scaling laws, the predicted improvement in measured real-world tasks from GPT-4 all the way up to a GPT-7 style model (~25000X more compute) is about the same magnitude as the predicted improvement just from GPT-3 to GPT-4 (~100X more compute).

I don’t think that tells us much about scaling. It tells us about these particular scaling laws. They’re fit on open datasets and small models, and they know nothing about algorithmic efficiency gains, synthetic data, or anything labs do after pre-training.

Are we getting close to the irreducible loss?

If you look at the left plot, these scaling laws put us fairly close to the irreducible loss.⁶ The irreducible loss is closely related to the entropy of the dataset, and represents the theoretical best performance a model can reach on that dataset. With Gadre scaling laws on RedPajama, the best possible model reaches an irreducible loss of ~1.84, and GPT-4 is estimated at ~2.05.⁷

But that’s the irreducible loss for one dataset under one set of public scaling laws. Labs train on different data mixtures, generate their own data, and do a lot of work after pre-training that none of this captures. We don’t know whether their loss landscape looks like this one at all.

Subjectivity in fitting functions and the limits of our data

As mentioned previously, the choice of fitting function on the second scaling law is highly subjective. For example, we could refit the loss and performance points from the Gadre paper using a sigmoidal function instead of an exponential one:

See link to Github repository for sigmoidal fit on Gadre data and resulting plots

Yet, the conclusion remains largely unchanged. Comparing the exponential fit (red line) with our custom sigmoidal fit (dotted purple line) in the left graph, the limitation is clear: we simply don’t have enough data points to confidently determine the best fitting function for relating loss to real-world performance.

No one actually knows how strong the next models will be

There are obviously many ways to improve the above “prediction”: using better scaling laws, using better estimates of data and compute, and so on.⁸ The point of the above exercise is more to show the amount of uncertainty baked into these predictions than to perform an accurate prediction.

Ultimately, scaling laws are noisy approximations, and with this chained method of prediction, we’re coupling two noisy approximations together. When you consider that the next models may have entirely different scaling laws, as a result of different architectures or data mixes, no one really knows the capabilities of the next few model scale-ups. That’s not a reason to relax about it.

We need evaluations, not extrapolations

I decided to share my own naive efforts here since I couldn’t easily find any myself. The first scaling law, loss as a function of data and compute, holds up well. The second one, capabilities as a function of loss, is subjective, noisy, and fit on very few points.

Which means you can’t look at a loss curve and say “at this many FLOPs the model will be able to run an autonomous cyber operation” or “at this loss the model will be able to walk someone through synthesizing a novel pathogen.” Those are the questions worth answering ahead of time, and extrapolation isn’t going to answer them.

What would answer them is an empirical science of evaluations: task-specific benchmarks that measure the capability you actually care about. If we knew what the next generation of models could do, we could prepare for it.

Thank you to Celine, Matthew, Jasmine, Coen, Shreyan, Nikhil, Trevor, Namanh, and Gabe for their invaluable feedback; Justin for his help with scaling laws; and Bela, Susan, and Maxwell for ‘encouragement.’

November 30, 2024 update: I changed the data and compute estimates to IFP estimates after Miles’ note instead of using my own compute estimates.

Title update: this was originally titled “Will We Have AGI?”, but reasoning models and RL made that the wrong question, so I renamed it.