2026-10-11 16:37 UTC

Jon Saad-Falcon and coauthors claim their Intelligence per Watt study finds local models can successfully answer 88.7% of one million sampled chat and reasoning queries, supporting substantial cloud-demand offloading despite lower measured power efficiency on local accelerators.

state: seedheat: lowuncertainty: mediumknownscott: mediumlocal-inference inference-economics evaluationJon Saad-FalconJohn HennessyAzalia MirhoseiniChristopher Ré

What is this?

Intelligence per Watt is a study by Jon Saad-Falcon and coauthors, including John Hennessy, Azalia Mirhoseini and Christopher Ré, presented on Stanford research sites and arXiv. Across 20+ local language models, eight accelerators and one million real-world single-turn chat and reasoning queries, the authors report 88.7% successful query coverage and a 5.3× improvement in intelligence per watt from 2023–2025. They argue this supports shifting substantial inference demand to local hardware, while reporting at least 1.4× lower intelligence per watt for local accelerators than cloud accelerators running identical models. The snippets do not establish a deployable router achieving that coverage or generalization to multi-turn and agent workloads; they also do not establish which findings are new to the revised paper.

Why it matters to Scott

The radar already tracks this study in radar:intelligence-per-watt-local-ai-metric; the supplied material does not establish which findings are new to the revision. Its coverage and power-efficiency results bear directly on Scott’s gamepc/Ollama bulk inference and hardware-aware runtime policy, motivating workload-specific local-versus-cloud testing rather than assuming local is more efficient, but the 88.7% figure establishes neither deployable routing coverage nor performance on his agent workloads.
dev:project.gamepcdev:technology.ollamadev:concept.hardware-aware-local-inferenceradar:intelligence-per-watt-local-ai-metricradar:concept.local-inferenceradar:concept.inference-economics
queries asked of Scott's wikis
  • local versus cloud inference economics power efficiency
  • task-based model routing small models frontier fallback
  • local-first AI on-device inference projects
  • evaluation query coverage routing accuracy agent workloads
  • distributed inference centralized infrastructure demand

Measured heat

now 0 pts/hpeak 0 pts/hcomments 0/hpeers p14momentum: steady3 platformsage 866h
points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

09-05 14:00⭐ origin echo-reconstructedThe revised paper evaluates 20+ local models and eight accelerators on one million queries, reporting 88.7% successful query coverage, a 5.3
Jon Saad-Falcon and coauthors on paper (echo) · attributed from hn.story.49694035
—
09-14 09:16first on hacker news · published · +211.3hIntelligence per Watt: Measuring Intelligence Efficiency of Local AI
pythonic_hell
—
09-16 15:45first on r/singularity · published · +265.8hIntelligence per Watt: Measuring Intelligence Efficiency of Local AI
yogthos
—
09-14 09:16amplified on hacker news 👑hn.story.49694035
pythonic_hell
peak 169 · 65 comments · 89% of case engagement
09-16 15:45amplified on r/singularityreddit.post.1wi0w66
yogthos
peak 44 · 6 comments · 11% of case engagement
09-16 08:20our radar first saw it · +258.4hdiscovery anchor: hn.story.49694035—
pace: p79 vs 519 stories at the 720h mark (now 866h old) — ahead of linux-distribution-trusting-trust-attack (1.0x), behind forgejo-1604-critical-rce (0.9x)

Evidence (3) — ⭐ canonical anchor

sourceobjectauthorscorecomments
🟧 hnIntelligence per Watt: Measuring Intelligence Efficiency of Local AI
Retrieved article excerpt

Open article · Retrieved 2026-09-16T08:21:41.917298+00:00

# Computer Science > Distributed, Parallel, and Cluster Computing

**arXiv:2511.07885** (cs)

[Submitted on 11 Nov 2025 ([v1](https://arxiv.org/abs/2511.07885v1)), last revised 6 Sep 2026 (this version, v6)]

# Title:Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

Authors:[Jon Saad-Falcon](https://arxiv.org/search/cs?searchtype=author&query=Saad-Falcon,+J), [Avanika Narayan](https://arxiv.org/search/cs?searchtype=author&query=Narayan,+A), [Hakki Orhun Akengin](https://arxiv.org/search/cs?searchtype=author&query=Akengin,+H+O), [J. Wes Griffin](https://arxiv.org/search/cs?searchtype=author&query=Griffin,+J+W), [Herumb Shandilya](https://arxiv.org/search/cs?searchtype=author&query=Shandilya,+H), [Adrian Gamarra Lafuente](https://arxiv.org/search/cs?searchtype=author&query=Lafuente,+A+G), [Medhya Goel](https://arxiv.org/search/cs?searchtype=author&query=Goel,+M), [Rebecca Joseph](https://arxiv.org/search/cs?searchtype=author&query=Joseph,+R), [Shlok Natarajan](https://arxiv.org/search/cs?searchtype=author&query=Natarajan,+S), [Etash Kumar Guha](https://arxiv.org/search/cs?searchtype=author&query=Guha,+E+K), [Shang Zhu](https://arxiv.org/search/cs?searchtype=author&query=Zhu,+S), [Ben Athiwaratkun](https://arxiv.org/search/cs?searchtype=author&query=Athiwaratkun,+B), [John Hennessy](https://arxiv.org/search/cs?searchtype=author&query=Hennessy,+J), [Azalia Mirhoseini](https://arxiv.org/search/cs?searchtype=author&query=Mirhoseini,+A), [Christopher Ré](https://arxiv.org/search/cs?searchtype=author&query=R%C3%A9,+C)

View a PDF of the paper titled Intelligence per Watt: Measuring Intelligence Efficiency of Local AI, by Jon Saad-Falcon and 14 other authors

[View PDF](https://arxiv.org/pdf/2511.07885)
[HTML (experimental)](https://arxiv.org/html/2511.07885v6)
> Abstract:Large language model (LLM) queries are predominantly processed by frontier models in centralized cloud infrastructure. Demand growth strains this paradigm faster than providers can scale. Two advances create an opportunity to rethink it: small, local LMs (<=20B active parameters) now achieve competitive performance to frontier models on many tasks, and local accelerators (e.g., Apple M4 Max) can host these models at interactive latencies. This raises the question: can local inference viably redistribute demand from centralized infrastructure? This requires measuring both whether local LMs can accurately answer real-world queries and whether they can do so efficiently on power-constrained devices (e.g., laptops). We propose intelligence per watt (IPW), task accuracy per unit of power, as a unified metric for the capability and efficiency of local inference across model-accelerator configurations. We evaluate 20+ state-of-the-art local LMs, 8 hardware accelerators (local and cloud), and 1M real-world single-turn chat and reasoning queries. For each query, we measure accuracy (local LM win rate against frontier models), energy, latency, and power. We find three key results. First, local LMs successfully answer 88.7% of these queries, with accuracy varying by domain. Second, longitudinal analysis from 2023-2025 shows IPW improved 5.3x, driven by both algorithmic and accelerator advances, with locally-serviceable query coverage rising from 23.2% to 71.3%. Third, local accelerators achieve at least 1.4x lower IPW than cloud accelerators running identical models, revealing significant headroom for local accelerator optimization. These findings demonstrate that local inference can meaningfully redistribute demand from centralized infrastructure for a substantial subset of queries, with IPW serving as the critical metric for tracking this transition.

|  |  |
| --- | --- |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | [arXiv:2511.07885](https://arxiv.org/abs/2511.07885) [cs.DC] |
|  | (or  [arXiv:2511.07885v6](https://arxiv.org/abs/2511.07885v6) [cs.DC] for this version) |
|  | <https://doi.org/10.48550/arXiv.2511.07885> Focus to learn more  arXiv-issued DOI via DataCite |

## Submission history

From: Jon Saad-Falcon [[view email](https://arxiv.org/show-email/c69de5f7/2511.07885)]   
 **[[v1]](https://arxiv.org/abs/2511.07885v1)**
Tue, 11 Nov 2025 06:33:30 UTC (5,373 KB)  
**[[v2]](https://arxiv.org/abs/2511.07885v2)**
Fri, 14 Nov 2025 00:53:12 UTC (5,538 KB)  
**[[v3]](https://arxiv.org/abs/2511.07885v3)**
Thu, 26 Feb 2026 17:09:14 UTC (5,538 KB)  
**[[v4]](https://arxiv.org/abs/2511.07885v4)**
Thu, 21 May 2026 03:40:21 UTC (5,134 KB)  
**[[v5]](https://arxiv.org/abs/2511.07885v5)**
Fri, 7 Aug 2026 02:40:27 UTC (5,621 KB)  
**[v6]**
Sun, 6 Sep 2026 05:29:37 UTC (5,608 KB)

Full-text links:

## Access Paper:

View a PDF of the paper titled Intelligence per Watt: Measuring Intelligence Efficiency of Local AI, by Jon Saad-Falcon and 14 other authors

- [View PDF](https://arxiv.org/pdf/2511.07885)
- [HTML (experimental)](https://arxiv.org/html/2511.07885v6)
- [TeX Source](https://arxiv.org/src/2511.07885)

[license icon](http://creativecommons.org/licenses/by/4.0/ "Rights to this article")

### Current browse context:

cs.DC

[< prev](https://arxiv.org/prevnext?id=2511.07885&function=prev&context=cs.DC "previous in cs.DC (accesskey p)")
  |   
[next >](https://arxiv.org/prevnext?id=2511.07885&function=next&context=cs.DC "next in cs.DC (accesskey n)")

[new](https://arxiv.org/list/cs.DC/new)
 | 
[recent](https://arxiv.org/list/cs.DC/recent)
 | [2025-11](https://arxiv.org/list/cs.DC/2025-11)

Change to browse by:

[cs](https://arxiv.org/abs/2511.07885?context=cs)  
[cs.AI](https://arxiv.org/abs/2511.07885?context=cs.AI)  
[cs.CL](https://arxiv.org/abs/2511.07885?context=cs.CL)  
[cs.LG](https://arxiv.org/abs/2511.07885?context=cs.LG)

### References & Citations

- [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2511.07885)
- [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2511.07885)
- [Semantic Scholar](https://api.semanticscholar.org/arXiv:2511.07885)

export BibTeX citation
Loading...

## BibTeX formatted citation

×

loading...

Data provided by:

### Bookmark

[BibSonomy](http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/2511.07885&description=Intelligence per Watt: Measuring Intelligence Efficiency of Local AI "Bookmark on BibSonomy")
[Reddit](https://reddit.com/submit?url=https://arxiv.org/abs/2511.07885&title=Intelligence per Watt: Measuring Intelligence Efficiency of Local AI "Bookmark on Reddit")



Bibliographic Tools

# Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer *([What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))*

Connected Papers Toggle

Connected Papers *([What is Connected Papers?](https://www.connectedpapers.com/about))*

Litmaps Toggle

Litmaps *([What is Litmaps?](https://www.litmaps.co/))*

scite.ai Toggle

scite Smart Citations *([What are Smart Citations?](https://www.scite.ai/))*

Code, Data, Media

# Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv *([What is alphaXiv?](https://alphaxiv.org/))*

Links to Code Toggle

CatalyzeX Code Finder for Papers *([What is CatalyzeX?](https://www.catalyzex.com))*

DagsHub Toggle

DagsHub *([What is DagsHub?](https://dagshub.com/))*

GotitPub Toggle

Gotit.pub *([What is GotitPub?](http://gotit.pub/faq))*

Huggingface Toggle

Hugging Face *([What is Huggingface?](https://huggingface.co/huggingface))*

ScienceCast Toggle

ScienceCast *([What is ScienceCast?](https://sciencecast.org/welcome))*

Demos

# Demos

Replicate Toggle

Replicate *([What is Replicate?](https://replicate.com/docs/arxiv/about))*

Spaces Toggle

Hugging Face Spaces *([What is Spaces?](https://huggingface.co/docs/hub/spaces))*

Spaces Toggle

TXYZ.AI *([What is TXYZ.AI?](https://txyz.ai))*

Related Papers

# Recommenders and Search Tools

Link to Influence Flower

Influence Flower *([What are Influence Flowers?](https://influencemap.cmlab.dev/))*

Core recommender toggle

CORE Recommender *([What is CORE?](https://core.ac.uk/services/recommender))*

- Author
- Venue
- Institution
- Topic


About arXivLabs

# arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2511.07885) |
Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))
pythonic_hell16965
🟧 echo.paper ⭐The revised paper evaluates 20+ local models and eight accelerators on one million queries, reporting 88.7% successful query coverage, a 5.3Jon Saad-Falcon and coauthors——
🟠 redditIntelligence per Watt: Measuring Intelligence Efficiency of Local AI
singularity
yogthos416

Interpretation history

Decision trace