Inside Google's DeepMind: A Research Lead on Scaling Laws and What Comes After
Researchers at DeepMind spent seven years chasing the idea that intelligence scales. GPT-4, Gemini, Claude — these are children of scaling laws. But the researchers who built them are now asking: what happens when the scaling laws stop scaling?
The fluorescent-lit corridors of Google's DeepMind headquarters in London hum with a specific kind of tension. Researchers here have spent the past seven years chasing a single idea: that intelligence scales. That if you pour enough compute into a large language model, trained on enough data, something close to general reasoning will emerge. The bet has paid off spectacularly. GPT-4, Gemini, Claude — these are children of scaling law research. But the researchers who built them are now asking a harder question: what happens when the scaling laws stop scaling?
We sat down with Vlad Feinberg — DeepMind's pre-training lead, who has spent years modeling the relationship between FLOPs, parameters, and model capability — to understand where the frontier of AI scaling actually stands today. What emerged was a candid assessment of what scaling laws can and cannot tell us, where the data ceiling is already pressing against ambitions, and why the next wave of breakthroughs may look nothing like the last.
The Era of Predictable Scaling Is Ending
For most of the 2020s, AI research operated under what you might call the "scaling hypothesis" — a working assumption that model capability scales as a smooth, predictable function of compute, data, and parameters. Kaplan et al. (2020) gave the field its first rigorous language for this relationship. The paper showed that transformer performance follows a power law with respect to model size and training compute. For researchers trying to decide where to invest their next GPU cluster, it was a compass. You could look at a proposed model size, a training budget, and predict — with reasonable confidence — how the resulting model would perform.
The predictions were not perfect. But they were directionally correct. And directionally correct was enough to build an industry.
What Scaling Laws Actually Tell Us
A scaling law is, at its core, an empirical relationship. It says: if you double the compute spent training a model, you can expect a predictable improvement in loss. The loss is a proxy for everything else — reasoning, language understanding, code generation. The relationship holds across many orders of magnitude. That regularity is what made compute allocation planning possible. It is also what made capital allocation in AI straightforward: buy more GPUs, train a bigger model, win.
The key insight from Kaplan's work was that large models are more efficient than small ones. A model with twice the parameters, trained on the same data, achieves better performance per unit compute than its smaller counterpart. This gave birth to the era of the frontier model — companies racing to train the largest, most capable systems the world had ever seen.
The Chinchilla Revolution and Its Aftermath
In 2022, Hoffmann et al. published the Chinchilla paper and punched a hole in the scaling hypothesis. The authors showed that the Kaplan scaling laws were miscalibrated in a critical way: they had under-weighted the importance of data. Kaplan's models were undertrained relative to their size. The Chinchilla insight was that for a given compute budget, the optimal model should have roughly equal contributions from parameters and training tokens. Train fewer parameters, but train them much longer.
This reframing reshaped the entire field. It meant that the path to better models was not simply "bigger" — it was "bigger and better trained." And "better trained" meant more high-quality data, curated carefully, scaled alongside the model. Chinchilla and its descendants (including GPT-4 class models) embodied this principle. The models were not just larger. They were trained on trillions of tokens, with rigorous data filtering and weighting.
The aftermath of Chinchilla was a new consensus: the 3-way tradeoff between compute, parameters, and data tokens is the true scaling frontier. Get all three right, in the right proportions, and you unlock the next capability threshold.
Where Empirical Scaling Still Holds — and Where It Doesn't
The scaling laws still hold in the aggregate. DeepMind's research leads will tell you, candidly, that predicting loss improvements from compute increases remains reasonably reliable at the frontier. If you double your training FLOPs, you still get a predictable — if diminishing — return in model quality.
But the laws are becoming less useful as decision-making tools. The reason is simple: the models are getting so large that the cost of running experiments to calibrate the laws has become prohibitive. You cannot easily A/B test a 10 trillion parameter model against a 1 trillion parameter model. The economics of frontier research now outpace the economics of scaling law measurement.
More critically, the laws break down when you ask them about capabilities that emerge discontinuously. Reasoning, multi-step planning, and what researchers call "flexible problem-solving" do not scale smoothly. A model may achieve near-random performance on a task for 10 orders of magnitude of scaling, then suddenly cross a threshold and achieve near-perfect performance. This emergent capability phenomenon — documented extensively by Wei et al. (2022) — means that scaling laws predict average loss well but cannot reliably forecast when a specific capability will appear.
This is the core of the research challenge today. The laws tell you that you are improving. They do not tell you what you will be able to do.
Beyond the Pre-Training Arms Race
If the first decade of large language model development was defined by pre-training, the next decade will be defined by everything that happens after. Post-pretraining innovations — alignment, fine-tuning, inference-time compute — are no longer optional refinements. They are the primary frontier.
Post-Pretraining Innovations: Alignment, Fine-Tuning, and Inference-Time Compute
The research community has increasingly recognized that what you do to a model after pre-training matters as much as the pre-training itself. RLHF (Reinforcement Learning from Human Feedback), constitutional AI, and chain-of-thought prompting are all post-pretraining interventions that dramatically shift model behavior without changing the underlying weights.
The most significant shift is what researchers call "test-time compute scaling." Rather than investing only in training-time compute (training a bigger model), frontier labs are investing in inference-time compute — allowing models to think longer, explore multiple reasoning paths, and verify their own outputs before producing a final answer.
This approach has shown remarkable results. DeepMind's Minerva (mathematical reasoning) and the chain-of-thought literature demonstrate that giving a model more "thinking time" at inference yields dramatic capability improvements on structured reasoning tasks. The implication is profound: you can improve a model's capabilities not by retraining it, but by changing how it deploys its existing knowledge.
"The next frontier is not how big we train the model. It is how cleverly the model uses what it has already learned." — Vlad Feinberg, Pre-Training Lead, DeepMind
The Data Bottleneck: Synthetic Data and Data Curation at Scale
Every DeepMind researcher we spoke to identified the same ceiling: data. The internet's high-quality text corpus, while vast, is finite. Models are now trained on trillions of tokens. The pool of genuinely novel, high-quality textual information that can be consumed without diminishing returns is smaller than the scaling curves imply.
The solution emerging across the field is synthetic data. Rather than relying solely on human-generated text, frontier labs are using models to generate training data — coding problems, mathematical proofs, scientific descriptions — that is then used to train the next generation of models. This "model bootstrapping" approach has proven effective for mathematical and coding tasks, where the space of valid outputs is constrained enough that generated data can be rigorously filtered.
At DeepMind, data curation has become a first-class research discipline. It is not merely "more data" but better data — deduplicated, diverse, balanced across domains, and free from contamination that would cause models to memorize rather than generalize.
DeepMind's Research Horizon
DeepMind occupies a distinctive position in the AI landscape. Unlike pure research labs or pure product companies, it sits at the intersection — pushing the capability frontier while building systems that eventually ship in Google products. This dual mandate shapes its research priorities in ways that are worth understanding.
Multi-Modal Architectures and the End of "Text-Only" Scaling
The most visible expression of DeepMind's post-text scaling strategy is Gemini — a natively multi-modal model that processes text, images, audio, and video within a unified architecture. The significance goes beyond capability breadth. Multi-modality represents a new scaling dimension. A model that can consume video is trained on data that is orders of magnitude more abundant than text. The scaling curves for vision-language models are different, and in some respects more favorable, than those for text-only models.
DeepMind's multi-modal research suggests that the next phase of scaling will be defined by cross-modal generalization — the ability to reason across text, visual, and auditory inputs in a way that mirrors human understanding. This is not simply "adding an image encoder to a language model." It requires rethinking the fundamental architecture to support true modality-agnostic reasoning.
Reasoning, Planning, and the Architecture Shifts Required
The research community is increasingly focused on what might be called "System 2" reasoning — the deliberate, multi-step planning that current LLMs handle inconsistently. The challenge is architectural. Transformers are sequence models. They process tokens in order. True planning — the ability to simulate multiple futures, evaluate them, and commit to a sequence of actions — requires something more.
DeepMind's research in this area includes work on world models, memory architectures, and hybrid systems that combine neural networks with more structured reasoning components. The goal is not to replace transformers but to extend them with capabilities they do not natively possess. How to do this efficiently, at scale, remains an open research question.
What Enterprise IT Leaders Need to Know
The practical implications of this research inflection are significant. For enterprise buyers of AI — the IT directors, CTOs, and decision-makers who are building AI strategies — the days of "buy the biggest model" as a coherent strategy are ending.
Why "Buy the Biggest Model" Is No Longer Viable
The frontier model race has produced systems of extraordinary general capability. But general capability is not the same as task-specific performance. For many enterprise use cases — contract review, scientific literature analysis, code generation for specific repositories — a specialized model trained on domain-specific data will outperform a general frontier model on cost-performance curves that are 10x to 100x better.
Feinberg was direct: "The models that will power enterprise AI in five years are not the models that are in the news today. They will be smaller, more specialized, and significantly more reliable." This is not a dismissal of frontier models. It is a recognition that the scaling laws that apply to general capability do not always transfer to domain-specific tasks.
The Rise of Specialized, Efficient Models
The economics of model deployment are shifting. The total cost of ownership for AI systems is not dominated by API call costs. It is dominated by inference infrastructure, latency requirements, and the cost of errors. For high-volume, repetitive tasks, a fine-tuned 7B parameter model running on commodity hardware will often deliver better business outcomes than a frontier model accessed via API.
This is creating a market for what the field sometimes calls "small language models" — though the term is misleading, because the models in question are not small in any absolute sense. They are smaller than frontier models but large enough to encode significant domain knowledge. The skill is in the fine-tuning — taking a capable base model and adapting it to a specific task with carefully curated data.
Infrastructure Considerations When Scaling Laws No Longer Predict Performance
For infrastructure teams, the post-scaling era introduces new planning challenges. With scaling laws, you could predict performance improvements from hardware investments with reasonable confidence. When scaling laws plateau or become unreliable predictors of specific capabilities, infrastructure planning becomes more complex.
The key shift is from "compute scaling" planning to "inference infrastructure" planning. As more capability improvements come from inference-time techniques (more tokens, more compute per query) rather than training-time scaling, the bottleneck shifts from training clusters to inference serving. This has implications for hardware selection, model serving infrastructure, and the design of AI-native applications.
The Road to AGI — And What Scaling Can and Can't Tell Us
DeepMind's stated goal — as articulated by CEO Demis Hassabis — is to build artificial general intelligence. At an AI summit in December 2025, Hassabis said that AI scaling "must be pushed to the maximum" as a key component of the path to AGI. The statement is both a commitment and a hedge. Maximum scaling is necessary, but is it sufficient?
Feinberg was measured on this question. "Scaling laws have taken us further than anyone expected. But we are now in a regime where the laws are less predictive, and where the capabilities we need for AGI — robust reasoning, long-horizon planning, causal understanding — do not scale smoothly from current architectures."
Where Human-Level Reasoning Breaks from Current Scaling Curves
The gap between current AI capability and human-level general reasoning is not simply a matter of scale. A human can take a single example of a new type of problem and generalize from it robustly. Current LLMs require many examples, often fail to generalize from sparse data, and can be fooled by adversarial variations that would not confuse a child.
Bridging this gap likely requires architectural innovations that have not yet been discovered. Whether those innovations are compatible with the transformer paradigm, or require something fundamentally new, is an active debate at the frontier of AI research. DeepMind's research portfolio includes both — extending transformers as far as they can go, while building next-generation architectures that address their known limitations.
A Research Roadmap for the Next Five Years
If there is a consensus among DeepMind researchers, it is this: the next five years of AI progress will be defined by three overlapping themes.
First, inference-time compute scaling — models that think longer and more deliberately. Second, multi-modal and cross-modal generalization, breaking out of text-only data constraints. Third, architectural innovations that move beyond pure transformer sequences toward something that supports richer representational structures.
The scaling laws did not end. They are evolving. And the researchers who understand them best are the first to say that what comes next will look different from what came before.
Expert Q&A
Q: Are scaling laws still a reliable planning tool for enterprise AI investments?
A: For general capability prediction, yes — but with diminishing confidence at the frontier. For domain-specific task performance, scaling laws are a poor predictor. Enterprises should use scaling laws to inform training compute budgets, but not to predict whether a specific business task will improve.
Q: What is the single most important infrastructure change enterprises should prepare for?
A: The shift from training-compute-dominated to inference-compute-dominated AI systems. As inference-time techniques (chain-of-thought, tree-of-thought, self-verification) become standard, the cost and latency profile of AI systems changes dramatically. Infrastructure that was designed for fast, single-pass inference will need to support longer-running inference sessions.
Q: Is synthetic data a real solution to the data ceiling, or is it a temporary fix?
A: It is real but limited. Synthetic data works well in domains with clear validation criteria — mathematics, code, formal reasoning — where you can automatically filter incorrect outputs. It is less reliable for open-ended creative or conversational tasks where quality is subjective. The data ceiling is not fully solved; it is being managed strategically.
Q: How should CTOs think about "buy vs. build" for AI capabilities in the post-scaling era?
A: The calculus is shifting. General frontier models are excellent at exploration and prototyping. For production workloads with clear requirements, domain-specific fine-tuned models offer superior cost-performance. The strategic question is not "which model" but "which deployment architecture" — and that requires more AI engineering talent than it did two years ago.
This interview has been edited for clarity and length.
Meta Description: A DeepMind research lead shares insider perspective on AI scaling laws — what they still predict, where they break down, and what comes next for enterprise AI strategy.
Subscribe to receive analysis of frontier AI research, enterprise AI strategy, and the technology decisions shaping the next decade.
[CTA: Subscribe to Algorithmine's AI Research Brief]