Leilei Chen and coauthors claim their reference-free black-box audit detects provider-side output-token inflation and flags consistent behavior in 7 of 15 API services, potentially giving customers a way to identify manipulated inference costs without establishing provider intent.

state: seedheat: lowuncertainty: highconvergesscott: mediuminference-economics token-cost provider-integrity black-box-auditingLeilei ChenLan Zhang

What is this?

The case describes a paper, “The More It Says, the More You Pay,” attributed to Leilei Chen and coauthors, proposing a reference-free black-box audit of provider-induced output verbosity and reporting suspicious behavior in 7 of 15 API services. The supplied search snippets do not locate that paper or verify its authorship, method, or numerical findings. They establish related research on opaque API auditing, including a differently titled token-inflation paper about manipulating billed token counts and a model-substitution test requiring an authentic local reference; neither substantiates this case’s specific claims. The reported flags should therefore remain attributed claims, not established evidence of intentional overcharging.

Why it matters to Scott

The claimed audit converges with Scott’s Capability Audit requirement for vendor-neutral testing and could extend his cost-tiered provider routing with a check for provider-induced verbosity, rather than merely illustrating the Mature Token Law’s warning that usage is not value. The supplied grounding does not verify the paper, method, or 7-of-15 finding, so this is a candidate evaluation technique—not evidence against any provider Scott uses; related radar cases cover API fingerprinting and cost-accounting failures, not this specific development.

ip:concept.capability-auditip:framework.the-mature-token-lawdev:concept.cost-tiered-llm-routingradar:one-token-api-model-fingerprintingradar:agentmeasure-token-accounting-auditradar:hidden-reasoning-real-task-costs

queries asked of Scott's wikis

LLM API cost accounting output-token budgets
provider trust independent verification auditability
agent harness verbosity controls inference costs
black-box evaluation reference-free behavioral testing
local inference versus hosted API economics

Measured heat

now 0 pts/hpeak 0 pts/hcomments 0/hpeers p14momentum: steady2 platformsage 579h

points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

09-17 13:28	⭐ origin echo-reconstructed	The authors demonstrate five provider-side attacks that each increase mean output length to more than 10.2 times the clean baseline while la Leilei Chen and coauthors on paper (echo) · attributed from hn.story.49751033	—
09-18 07:07	first on hacker news · published · +17.6h	The More It Says, the More You Pay: A Black-Box Audit of Token Inflation in LLM sbulaev	—
09-18 07:07	amplified on hacker news 👑	hn.story.49751033 sbulaev	peak 2 · 0 comments · 49% of case engagement
09-19 04:27	amplified on hacker news	hn.story.49763307 donk8r	peak 2 · 0 comments · 49% of case engagement
09-18 07:21	our radar first saw it · +17.9h	discovery anchor: hn.story.49751033	—

pace: p23 vs 1032 stories at the 336h mark (now 579h old) — ahead of aafp-commons-signed-agent-notebook (2.0x), behind agentgate-signed-agent-receipts (0.7x)

Evidence (3) — ⭐ canonical anchor

source	object	author	score	comments
🟧 hn	The More It Says, the More You Pay: A Black-Box Audit of Token Inflation in LLM Retrieved article excerpt Open article · Retrieved 2026-09-18T07:22:21.036483+00:00 # Computer Science > Cryptography and Security arXiv:2609.20370 (cs) [Submitted on 17 Sep 2026] # Title:The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services Authors:[Leilei Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen,+L), [Lan Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+L), [Chen Tang](https://arxiv.org/search/cs?searchtype=author&query=Tang,+C), [Pengcheng Sun](https://arxiv.org/search/cs?searchtype=author&query=Sun,+P), [Jiewei Lai](https://arxiv.org/search/cs?searchtype=author&query=Lai,+J), [Yixiao Huang](https://arxiv.org/search/cs?searchtype=author&query=Huang,+Y), [Zhaopeng Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+Z), [Xinpeng Shen](https://arxiv.org/search/cs?searchtype=author&query=Shen,+X) View a PDF of the paper titled The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services, by Leilei Chen and 7 other authors [View PDF](https://arxiv.org/pdf/2609.20370) [HTML (experimental)](https://arxiv.org/html/2609.20370v1) > Abstract:In pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline. Our experiments show that each attack increases mean output length to more than 10.2x the clean baseline, demonstrating PTIA's financial appeal and feasibility at multiple stages of generation. Yet auditing PTIA from black-box responses is difficult for users. Our key observation is PTIA saturation: an initial attack sharply lengthens output, but further strengthening or composition has much less effect. We trace this saturation to stopping behavior: an initial PTIA sharply lowers the end-of-sequence token probability, whereas further intervention lowers it only marginally. Building on this insight, we design a lightweight single-probe audit that applies a controlled lengthening intervention. Under PTIA, the probe induces far fewer additional tokens than under normal service. The audit requires neither a trusted local reference model nor historical clean responses, and its separately issued original and probed requests resemble ordinary traffic, making evasion difficult. Across four open-weight models, it achieves an average detection rate of 85.1% with false-positive rates below 2%. Across 15 real LLM API services, the audit flags 7 for PTIA-consistent behavior. \| \| \| --- \| \| Comments: \| \| Subjects: \| Cryptography and Security (cs.CR) \| \| Cite as: \| [arXiv:2609.20370](https://arxiv.org/abs/2609.20370) [cs.CR] \| \| \| (or [arXiv:2609.20370v1](https://arxiv.org/abs/2609.20370v1) [cs.CR] for this version) \| \| \| <https://doi.org/10.48550/arXiv.2609.20370> Focus to learn more arXiv-issued DOI via DataCite (pending registration) \| ## Submission history From: Leilei Chen [[view email](https://arxiv.org/show-email/b783e1ba/2609.20370)] [v1] Thu, 17 Sep 2026 13:28:06 UTC (291 KB) Full-text links: ## Access Paper: View a PDF of the paper titled The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services, by Leilei Chen and 7 other authors - [View PDF](https://arxiv.org/pdf/2609.20370) - [HTML (experimental)](https://arxiv.org/html/2609.20370v1) - [TeX Source](https://arxiv.org/src/2609.20370) [view license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/ "Rights to this article") ### Current browse context: cs.CR [< prev](https://arxiv.org/prevnext?id=2609.20370&function=prev&context=cs.CR "previous in cs.CR (accesskey p)") \| [next >](https://arxiv.org/prevnext?id=2609.20370&function=next&context=cs.CR "next in cs.CR (accesskey n)") [new](https://arxiv.org/list/cs.CR/new) \| [recent](https://arxiv.org/list/cs.CR/recent) \| [2026-09](https://arxiv.org/list/cs.CR/2026-09) Change to browse by: [cs](https://arxiv.org/abs/2609.20370?context=cs) ### References & Citations - [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.20370) - [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.20370) - [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.20370) export BibTeX citation Loading... ## BibTeX formatted citation × loading... Data provided by: ### Bookmark [BibSonomy](http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/2609.20370&description=The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services "Bookmark on BibSonomy") [Reddit](https://reddit.com/submit?url=https://arxiv.org/abs/2609.20370&title=The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services "Bookmark on Reddit") Bibliographic Tools # Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ([What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer)) Connected Papers Toggle Connected Papers ([What is Connected Papers?](https://www.connectedpapers.com/about)) Litmaps Toggle Litmaps ([What is Litmaps?](https://www.litmaps.co/)) scite.ai Toggle scite Smart Citations ([What are Smart Citations?](https://www.scite.ai/)) Code, Data, Media # Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ([What is alphaXiv?](https://alphaxiv.org/)) Links to Code Toggle CatalyzeX Code Finder for Papers ([What is CatalyzeX?](https://www.catalyzex.com)) DagsHub Toggle DagsHub ([What is DagsHub?](https://dagshub.com/)) GotitPub Toggle Gotit.pub ([What is GotitPub?](http://gotit.pub/faq)) Huggingface Toggle Hugging Face ([What is Huggingface?](https://huggingface.co/huggingface)) ScienceCast Toggle ScienceCast ([What is ScienceCast?](https://sciencecast.org/welcome)) Demos # Demos Replicate Toggle Replicate ([What is Replicate?](https://replicate.com/docs/arxiv/about)) Spaces Toggle Hugging Face Spaces ([What is Spaces?](https://huggingface.co/docs/hub/spaces)) Spaces Toggle TXYZ.AI ([What is TXYZ.AI?](https://txyz.ai)) Related Papers # Recommenders and Search Tools Link to Influence Flower Influence Flower ([What are Influence Flowers?](https://influencemap.cmlab.dev/)) Core recommender toggle CORE Recommender ([What is CORE?](https://core.ac.uk/services/recommender)) - Author - Venue - Institution - Topic About arXivLabs # arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? [Learn more about arXivLabs](https://info.arxiv.org/labs/index.html). [Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2609.20370) \| Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))	sbulaev	2	0
🟧 echo.paper ⭐	The authors demonstrate five provider-side attacks that each increase mean output length to more than 10.2 times the clean baseline while la	Leilei Chen and coauthors	—	—
🟧 hn	A Black-Box Audit of Provider-Side Token Inflation in LLM Services	donk8r	2	0

source

object

author

score

comments

🟧 hn

The More It Says, the More You Pay: A Black-Box Audit of Token Inflation in LLM

Retrieved article excerpt

Open article · Retrieved 2026-09-18T07:22:21.036483+00:00

# Computer Science > Cryptography and Security

**arXiv:2609.20370** (cs)

[Submitted on 17 Sep 2026]

# Title:The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services

Authors:[Leilei Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen,+L), [Lan Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+L), [Chen Tang](https://arxiv.org/search/cs?searchtype=author&query=Tang,+C), [Pengcheng Sun](https://arxiv.org/search/cs?searchtype=author&query=Sun,+P), [Jiewei Lai](https://arxiv.org/search/cs?searchtype=author&query=Lai,+J), [Yixiao Huang](https://arxiv.org/search/cs?searchtype=author&query=Huang,+Y), [Zhaopeng Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+Z), [Xinpeng Shen](https://arxiv.org/search/cs?searchtype=author&query=Shen,+X)

View a PDF of the paper titled The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services, by Leilei Chen and 7 other authors

[View PDF](https://arxiv.org/pdf/2609.20370)
[HTML (experimental)](https://arxiv.org/html/2609.20370v1)
> Abstract:In pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline. Our experiments show that each attack increases mean output length to more than 10.2x the clean baseline, demonstrating PTIA's financial appeal and feasibility at multiple stages of generation. Yet auditing PTIA from black-box responses is difficult for users. Our key observation is PTIA saturation: an initial attack sharply lengthens output, but further strengthening or composition has much less effect. We trace this saturation to stopping behavior: an initial PTIA sharply lowers the end-of-sequence token probability, whereas further intervention lowers it only marginally. Building on this insight, we design a lightweight single-probe audit that applies a controlled lengthening intervention. Under PTIA, the probe induces far fewer additional tokens than under normal service. The audit requires neither a trusted local reference model nor historical clean responses, and its separately issued original and probed requests resemble ordinary traffic, making evasion difficult. Across four open-weight models, it achieves an average detection rate of 85.1% with false-positive rates below 2%. Across 15 real LLM API services, the audit flags 7 for PTIA-consistent behavior.

|  |
| --- |
| Comments: |
| Subjects: | Cryptography and Security (cs.CR) |
| Cite as: | [arXiv:2609.20370](https://arxiv.org/abs/2609.20370) [cs.CR] |
|  | (or  [arXiv:2609.20370v1](https://arxiv.org/abs/2609.20370v1) [cs.CR] for this version) |
|  | <https://doi.org/10.48550/arXiv.2609.20370> Focus to learn more  arXiv-issued DOI via DataCite (pending registration) |

## Submission history

From: Leilei Chen [[view email](https://arxiv.org/show-email/b783e1ba/2609.20370)]   
 **[v1]**
Thu, 17 Sep 2026 13:28:06 UTC (291 KB)

Full-text links:

## Access Paper:

View a PDF of the paper titled The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services, by Leilei Chen and 7 other authors

- [View PDF](https://arxiv.org/pdf/2609.20370)
- [HTML (experimental)](https://arxiv.org/html/2609.20370v1)
- [TeX Source](https://arxiv.org/src/2609.20370)

[view license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/ "Rights to this article")

### Current browse context:

cs.CR

[< prev](https://arxiv.org/prevnext?id=2609.20370&function=prev&context=cs.CR "previous in cs.CR (accesskey p)")
  |   
[next >](https://arxiv.org/prevnext?id=2609.20370&function=next&context=cs.CR "next in cs.CR (accesskey n)")

[new](https://arxiv.org/list/cs.CR/new)
 | 
[recent](https://arxiv.org/list/cs.CR/recent)
 | [2026-09](https://arxiv.org/list/cs.CR/2026-09)

Change to browse by:

[cs](https://arxiv.org/abs/2609.20370?context=cs)

### References & Citations

- [NASA ADS](https://ui.adsabs.harvard.edu/abs/arXiv:2609.20370)
- [Google Scholar](https://scholar.google.com/scholar_lookup?arxiv_id=2609.20370)
- [Semantic Scholar](https://api.semanticscholar.org/arXiv:2609.20370)

export BibTeX citation
Loading...

## BibTeX formatted citation

×

loading...

Data provided by:

### Bookmark

[BibSonomy](http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/2609.20370&description=The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services "Bookmark on BibSonomy")
[Reddit](https://reddit.com/submit?url=https://arxiv.org/abs/2609.20370&title=The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services "Bookmark on Reddit")



Bibliographic Tools

# Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer *([What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))*

Connected Papers Toggle

Connected Papers *([What is Connected Papers?](https://www.connectedpapers.com/about))*

Litmaps Toggle

Litmaps *([What is Litmaps?](https://www.litmaps.co/))*

scite.ai Toggle

scite Smart Citations *([What are Smart Citations?](https://www.scite.ai/))*

Code, Data, Media

# Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv *([What is alphaXiv?](https://alphaxiv.org/))*

Links to Code Toggle

CatalyzeX Code Finder for Papers *([What is CatalyzeX?](https://www.catalyzex.com))*

DagsHub Toggle

DagsHub *([What is DagsHub?](https://dagshub.com/))*

GotitPub Toggle

Gotit.pub *([What is GotitPub?](http://gotit.pub/faq))*

Huggingface Toggle

Hugging Face *([What is Huggingface?](https://huggingface.co/huggingface))*

ScienceCast Toggle

ScienceCast *([What is ScienceCast?](https://sciencecast.org/welcome))*

Demos

# Demos

Replicate Toggle

Replicate *([What is Replicate?](https://replicate.com/docs/arxiv/about))*

Spaces Toggle

Hugging Face Spaces *([What is Spaces?](https://huggingface.co/docs/hub/spaces))*

Spaces Toggle

TXYZ.AI *([What is TXYZ.AI?](https://txyz.ai))*

Related Papers

# Recommenders and Search Tools

Link to Influence Flower

Influence Flower *([What are Influence Flowers?](https://influencemap.cmlab.dev/))*

Core recommender toggle

CORE Recommender *([What is CORE?](https://core.ac.uk/services/recommender))*

- Author
- Venue
- Institution
- Topic


About arXivLabs

# arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2609.20370) |
Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))

sbulaev

🟧 echo.paper ⭐

The authors demonstrate five provider-side attacks that each increase mean output length to more than 10.2 times the clean baseline while la

Leilei Chen and coauthors

—

🟧 hn

A Black-Box Audit of Provider-Side Token Inflation in LLM Services

donk8r

Interpretation history

2026-09-19T05:27:40Z
The new attachment is another submission of the same paper, not independent validation. This remains a potentially useful provider-audit technique, but there is no new basis for treating its service flags as evidence of manipulation or changing provider choices.
2026-09-19T05:21:32Z
evidence attached: hn.story.49763307 — shared external link with case evidence
2026-09-18T07:25:42Z
grounded: converges/medium — The claimed audit converges with Scott’s Capability Audit requirement for vendor-neutral testing and could extend his cost-tiered provider routing with a check
2026-09-18T07:22:34Z
case created — This first-party paper introduces a bounded, quantitatively testable cost-integrity claim distinct from existing token-accounting and malicious-router cases.

Decision trace

10-01 16:29review_dormantscheduled targets exhausted or 28 quiet days
10-01 16:29drop_targetsquiet through full ladder or over cap 8
09-19 15:27repriceThe new attachment is another submission of the same paper, not independent validation. This remains a potentially useful provider-audit technique, but there is no new basis for treating its service f
09-19 15:21attachshared external link with case evidence
09-19 15:20propose_attachshared external link with case evidence
09-18 17:25groundThe claimed audit converges with Scott’s Capability Audit requirement for vendor-neutral testing and could extend his cost-tiered provider routing with a check for provider-induced verbosity, rather t
09-18 17:22createThis first-party paper introduces a bounded, quantitatively testable cost-integrity claim distinct from existing token-accounting and malicious-router cases.