Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

What it claims

Each of the 207 WebGPU kernels is a separately versioned Hub repository behind a stable JavaScript contract. The loader picks a shape- and device-specific variant behind that contract, so callers never choose one themselves. The headline 2.57x speedup over ONNX Runtime Web counts GPU execution time only, on one device (Apple M4), for single operations. It does not cover full models.

How it works

Each kernel repo (for example webgpu-kernels/ai.onnx.Add) contains:

• manifest.json: the operation contract. It defines inputs, outputs, attributes, type constraints and shape-derivation rules.
• metadata.json: the kernel identifier, digests and provenance.
• test.json: correctness cases.
• bench.json: benchmark and tuning cases.
• *.wgsl.jinja: parameterized WGSL templates. These are rendered into a shader for a given request and device.

Even Add ships four variants: equal shapes, vectorized broadcasting, scalar processing, and general broadcasting.

flowchart LR
  A[getKernel repo + version] --> B[manifest.json contract]
  B --> C[derive output shape/dtype, allocate]
  C --> D{select variant for shape + device}
  D --> E[equal-shape vectorized]
  D --> F[broadcast paths]
  E --> G[render WGSL template, dispatch]
  F --> G

The version option refers to the kernel contract version. It is distinct from the ONNX opset, an operator's since_version, and any model revision.

Evidence

The comparison was run against ONNX Runtime Web 1.30.0-dev.20260826-b1f76d586a (WebGPU backend) on an Apple M4 GPU.

• Of 1,756 test cases across all 207 ops, 809 were kept. A case was kept only when both sides produced matching outputs and reliable timings.
• Across those 809 cases: 2.57x geometric-mean speedup, 1.90x median, with 629 wins, 176 losses and 4 ties.
• Outliers:
• Bilinear Einsum i,ij,j at size 4096: 0.136 ms vs 1,396 ms.
• Row-wise CumSum over [256, 4096]: 0.016 ms vs 4.784 ms, about 301x.

The authors call these slow-path cases, not typical results.

The page does not say why the other 947 cases were excluded, for example mismatched outputs versus unreliable timing.

How to use it

npm install @huggingface/kernels@preview

import { getKernel } from "@huggingface/kernels";
const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });
const { c } = await add({
  a: { data: new Float32Array([1, 2, 3, 4, 5, 6]), shape: [2, 3] },
  b: { data: new Float32Array([10, 20, 30]), shape: [3] },
});

• Calling convention: inputs are named { data: TypedArray, shape } objects, and outputs come back as named fields that are allocated for you.
• Other operations: the call is the same for everything, such as ai.onnx.MatMul; only the repo ID and inputs change.
• WebGPU check: test for support with "gpu" in navigator.
• Fleet: an in-browser suite that runs correctness and performance checks on your hardware. With your consent, it contributes the results privately to Hugging Face.

Limits

• Preview release: the npm package is published under the @preview tag.
• What the timings exclude: they leave out kernel loading, session creation, input upload, shader compilation and readback. Small cases may benefit from GPU cache. Treat the numbers as a relative comparison, not an expected application speedup.
• Coverage of the benchmark:
• It measures single operations, not full models.
• It used one GPU and browser combination.
• Results will vary with GPU, browser, driver and available WebGPU features.
• Small inputs: for tiny inputs, the GPU round trip costs more than the math.
• No higher-level tooling yet: the kernels are not connected to model tooling. That integration is described as a next step, and upstreaming to ONNX Runtime is in progress.
• Platform support: WebGPU availability depends on browser, OS, GPU and driver.