Skip to content

Back to the catalog

The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services
arxiv.orgpaper

The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services

This arXiv preprint defines and studies "Provider-Side Token Inflation Attacks" (PTIA), where LLM API providers covertly manipulate generation to produce longer outputs (and thus higher bills) while preserving task utility, testing five attack variants at the query, prompt, representation, and model levels that each inflate output length over 10x baseline. The authors identify a "PTIA saturation" effect tied to end-of-sequence token probability suppression, and build a lightweight single-probe black-box audit (no trusted reference model needed) that detects PTIA with 85.1% average detection rate and under 2% false-positive rate across four open-weight models, flagging 7 of 15 real-world LLM API services as PTIA-consistent.

TL;DR A black-box audit method exposes LLM API providers secretly inflating output token counts to overcharge users, flagging 7 of 15 real services as suspect.
Open original ↗
Also seen from another angle🛡️ Security →
Document

What it claims

Pay-per-token LLM providers can covertly lengthen outputs to raise bills while keeping task utility, which the paper names a Provider-Side Token Inflation Attack (PTIA). Five attacks span the query, prompt, representation, and model levels of the provider-controlled pipeline; each pushes mean output length past 10.2x the clean baseline.

How it works

The audit turns the paper's central observation into a test. PTIA saturates: the first intervention sharply lengthens output, but stacking or strengthening attacks adds little. The authors trace this to stopping behavior — an initial PTIA drives the end-of-sequence token probability down hard, and further intervention barely moves it.

The audit exploits that ceiling with a single probe. It issues the original request, then a second request carrying a controlled lengthening intervention. On honest service the probe induces many extra tokens; under PTIA the EOS probability is already suppressed, so the same probe adds far fewer. The gap is the signal.

flowchart TD
  A[Issue original request] --> P[Issue probed request<br/>controlled lengthening]
  P --> D{Extra tokens<br/>from probe?}
  D -->|many| H[Honest service]
  D -->|few| T[PTIA-consistent]

Two properties matter for adoption: it needs no trusted local reference model and no historical clean responses, and the original and probed requests are issued separately and resemble ordinary traffic, which the authors argue makes evasion hard.

Evidence

Across four open-weight models the audit reaches 85.1% average detection rate with false-positive rates below 2%. Applied to 15 real LLM API services, it flags 7 as PTIA-consistent. The 10.2x length figure is the mean across the five attacks over a clean baseline.

Limits

This is an abstract-only arXiv page (submitted 17 Sep 2026), so the specifics behind the headline numbers are not visible here: which four open-weight models, which 15 services, the definition of "utility preserved," the probe's intervention design, and how the 85.1% breaks down per model or per attack level are all unstated. The 7-of-15 flag is a detector verdict, not a confirmed accusation against named providers — the abstract does not claim ground truth for the real-service test. Read the PDF before treating any flagged service as confirmed.

Techniques named here

  • Provider-Side Token Inflation Attack
  • PTIA saturation
  • Single-probe audit

Related in ai-infra