Character.AI reports 2x AMD inference throughput
DigitalOcean and AMD say Character.AI doubled inference throughput on Instinct GPUs and cut token cost 50% versus a generic setup. No independent audit.
DigitalOcean and AMD are still citing a January case: half the cost per token, against a generic setup. No outside audit.
On 13 January 2026, DigitalOcean, AMD, and Character.AI put out the same case. Character.AI inference jobs on Instinct MI300X and MI325X GPUs, running on DigitalOcean Kubernetes (DOKS), had reached up to twice the inference throughput (requests per second) and half the cost per token, they said. The baseline, according to all three, is “standard” GPU infrastructure without the optimizations in the post, on other providers. On 1 October 2026, @AMD repeated the figures on X. That was not a new measurement. For a large chat app’s GPU bill, the gain may be the chip, the software, or the comparison.
In short
- Up to 2x QPS and 50% lower cost per token, according to the companies, in January 2026.
- The baseline is a generic setup on “other providers”, without the post’s optimizations, not a named Nvidia chip.
- In the same text, DP2/TP4/EP4 is about 91% above the generic setup; that is not the 2x figure rounded down.
- AMD’s GD-181 note says the company did not independently verify the data.
First: what inference throughput is
Inference is the moment the model answers, not the moment it is trained. Throughput, measured in QPS (requests per second), is how many of those answers a server finishes. A token is a chunk of text the model reads or writes. For a large chat app, that becomes a GPU bill. AMD sells the chip, DigitalOcean sells the cloud, and Character.AI is both customer and the source of the cost numbers.
What the companies announced
In AMD’s case study, David Brinker, senior vice president of partnerships at Character.AI, credits Instinct GPUs with doubling production inference throughput and cutting cost per token by 50 percent. DigitalOcean’s IR release, from the same day, quotes Vamsi Boppana, AMD’s SVP of artificial intelligence, on Instinct silicon, the open ROCm software stack, and the work on DigitalOcean’s platform.
DigitalOcean’s engineering post, republished on Character.AI’s blog, describes a multi-year, eight-figure annual agreement in dollars for GPU infrastructure. That range is $10 million to $99 million a year. The exact figure was not published. The line does not appear in the IR release.
Character.AI, according to DigitalOcean IR and the AMD case study, handles more than 1 billion queries a day. The deep-dive says about 20 million users worldwide. The AMD case study says GPUs are the company’s single largest operating cost. There is no date or method for those volume figures. The write-ups talk about migrated workloads and one production deployment, not about what share of the fleet has left other providers.
How the test was set up
The deep-dive describes a server with 8 MI325X GPUs running the Mixture-of-Experts model Qwen3-235B Instruct in FP8 (in vLLM, Qwen/Qwen3-235B-A22B-Instruct-2507-FP8). An MoE model, on each stretch of text, turns on only some of its internal specialists; the post cites 128 experts. AMD’s MI325X spec sheet lists 256 GB of HBM3E per card; the engineering post says 256 GB of HBM. MI300X accelerators also appear in the press release and the case study.
The “up to 2x” QPS figure, in the footnote, applies to that model, with a 5600/140 load (input and output length) and a p90 latency cap, against the generic setup. p90 means 90% of responses stayed under that cap. In the same post, the DP2/TP4/EP4 split (data, tensor, and expert parallelism) is about 91% better than the generic setup and about 45% better than DP1/TP8/EP8 on the same Instinct hardware, at about 64 concurrent requests.
The software is vLLM, with ROCm (AMD’s stack in place of Nvidia’s CUDA) and AITER, AMD’s kernels for MoE. Weights and the KV-cache, the memory of the conversation so far, ran in FP8. DigitalOcean says that cut about 50% of the cache’s VRAM versus the default BF16 cache, not the whole model. The weights, about 240 GB, were cached over NFS, which cut 10% to 15% off checkpoint load time.
What the 2x figure does not prove
RUMOR: reading the case as a bake-off against Nvidia (H100, H200, or Blackwell), or as proof that Character.AI left CUDA. The primary sources talk about “other providers” and generic infrastructure without the optimizations in the post. They do not name the previous cloud or the chip that was replaced.
The results are internal tests by DigitalOcean and AMD, with cost figures supplied by Character.AI. All three sell or use the stack they are describing. The October post says: “When inference is the operating budget, efficiency matters. With AMD Instinct GPUs on @DigitalOcean, @Character_ai doubled production inference throughput and cut cost per token by 50%.” It only recaps the January case.
What we still don’t know
We do not know what share of the fleet already runs on this stack, the exact contract value, or the name of the previous provider. With no independent test, the 2x and the 50% cut remain company figures, measured in January 2026 against a generic setup without the optimizations in the post. The October tweet does not add a new measurement.
Sources
Official