Proving LLMs at Scale

AI workloads are easy to claim and hard to prove.

A model provider can say that a particular model processed a particular input. A cloud can say it served the requested weights and architechture. An AI lab can say its system followed an approved policy. But unless the relying party can inspect and rerun the entire computation, those claims ultimately depend on trust.

Zero-knowledge proofs can replace that trust with mathematics. A proof can establish that a committed model produced a specific output from a specific input, without revealing the model’s weights or other private information.

The problem has always been performance.

Proof generation that works for a small classifier is not automatically useful for a production LLM. Frontier-scale inference involves billions of parameters, long contexts, floating-point arithmetic, and enormous matrix multiplications. A theoretically sound proof system is irrelevant if producing the proof takes hours, days, or more compute than the world can afford.

Today, we are publishing the first results from Attestable’s prover for production-scale transformer inference.

On a single NVIDIA H100, our current system reaches:

Across all successful benchmark configurations, proof sizes range from 4.35 to 7.92 MiB, and verification takes 157 to 648 milliseconds on our reference CPU verifier.

These are alpha results. There is still substantial optimization ahead of us. But they put zero-knowledge proofs for LLM inference into a fundamentally different performance regime.

This is no longer ZKML for toy models. For comparison, the natural open-source alternative, ezkl, claimed in the end of 2023 a proving time of 16 minutes for a 1M parameter model on a CPU (x70,000 smaller model, and about x50,000 slower!).

What exactly are we proving?

For a model (F) with committed weights (W), input (x), random seed (r), and output (y), Attestable generates a proof of a statement of the form:

y = F(W, x,r)

The proof binds the execution to a cryptographic commitment to the model weights. It can also bind the input, output, model configuration, decoding parameters, and other policy-relevant values.

The proofs are short, and verification is fast and easy, and does not depend on how large the model or how long the context is.

A verifier can then check that:

• the committed model was used
• the claimed input was processed
• the claimed output resulted from that execution
• the required execution policy and random seed were followed

The verifier does not need the model weights and does not rerun the model.

You can think of this as a digital signature created by the computation itself, rather than by an organization holding a signing key. Change the model, input, output, or proven policy, and verification fails.

Our prover operates alongside the inference process. Once a sequence has been decoded, the system produces one proof covering the complete benchmarked execution. The proof can then be checked independently by a standalone CPU verifier.

Our security and limitations

All of our results are based only on the security of cryptographic hash functions, and thus we are post-quantum secure. We are not relying on public-key cryptography at all, and uphold 100 bits of security - meaning, a malicious attacker would need to compute 2¹⁰⁰ values before they can bypass our solution.

In our current version, we have two temporary limitations:

• We support up to 16K tokens in the context window.
• Though we prove all non-linear operations in floting points, we quantize all matrix multiplications to integers.

We believe both limitations will be lifted in the not-too-far future.

Gemma 4 31B as a case study

Gemma 31B is a useful test of whether a proof system is ready for real LLM workloads, as a dense medium-sized model.

Performance across different configurations

Proving throuhgput depends on the batch size and context length. Our current Gemma prover reaches 53 tokens per second for a batch-one 16K sequence. At batch four, it reaches 77 tokens per second across four 4K sequences.

What our quantization does to Gemma?

Performance is only useful if the proven model remains useful. Compared to other ZKML systems that only use integers, we use floating points for almost all of the operations during inference. Our only current limitation is dynamically quantizing the linear operations to 8-bit integers.

We evaluated the original Gemma 31B model, a public INT8 variant, and Attestable’s current quantization on GPQA Diamond and IFEval.

The GPQA-D result demonstrates that our approach can preserve difficult reasoning performance better than a standard INT8 baseline while enabling efficient proof generation. The IFEval result shows where the current quantization still needs improvement.

We are committed to improving those results, both by improving the quantization schemes we use, and by supporting floating points in the linear operations as well.

‍

‍