How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

What it claims

Papers with Code serves paper search over 110,000+ arXiv and Daily Papers entries by fusing PostgreSQL full-text search with pgvector dense retrieval. Only query embedding sits on the request path, and lexical search alone is always a valid answer. In a 5,000-paper pilot, a 256-dim Qwen3-Embedding-0.6B HNSW index reached 0.9955 Recall@20 against exact search, with 1.31 ms p50 and 2.21 ms p95 lookup latency, using about 27% of the storage of the 1024-dim version.

How it works

Embedding contract. Each paper is encoded as normalized title + "\n\n" + normalized abstract. Every vector generation records:

• model repo and exact revision
• output dimension
• input-format version
• query vs. document prompt
• normalization method
• content hash of the title and abstract

Documents use Qwen's document prompt (encode_document). Live queries use its query prompt.

Batch build.

• The exporter reads a repeatable-read Postgres snapshot, streaming rows into bounded JSONL shards plus a manifest with SHA-256 checksums.
• Output goes to a private Bucket under immutable runs/<run-id>/ prefixes. Buckets are mutable, so "never overwrite a run ID" is enforced by the application, not the storage.
• The Job worker verifies checksums, sorts texts by length to cut padding, and shrinks the batch size automatically on GPU out-of-memory errors.
• It truncates vectors to 256 dims, L2-normalizes them, and writes float16 Parquet atomically.
• Each finished shard gets a completion marker, so a restarted Job resumes instead of overwriting.

Activation. The importer rechecks schema, checksums, dimension, normalization, unique IDs and current content hashes. It then builds a separate HNSW index per generation. The new generation is marked active atomically only when it covers every eligible current paper. Rollback is a config change.

Incremental updates. An hourly process sends missing or changed papers to the same TEI endpoint with the document prompt, at most 500 papers per run in batches of 16. Before writing, it locks the row and rechecks the content hash. If the paper changed during inference, the vector is discarded and the next run picks it up.

Query path:

flowchart TD
  Q[User query] --> L[Postgres weighted full-text, top 50]
  Q --> C{Endpoint healthy within 1s?}
  C -- yes --> V[pgvector HNSW, top 50]
  C -- "no: cold, timeout, bad vector, breaker open" --> S[Skip semantic branch]
  L --> R[Weighted RRF, k=60]
  V --> R
  S --> R
  R --> I[Identity rules: exact title/arXiv ID first, trigram fuzzy, abstain if ambiguous]

The two branches are fused with weighted RRF, equal branch weights and k=60. Deterministic rules then sit on top:

• exact titles and arXiv IDs pin to the top
• a method taxonomy handles navigational queries such as "the original BERT paper"
• conservative trigram matching handles partial titles and typos
• ambiguous fuzzy matches abstain rather than return a bad result

The query client enforces:

• a 1 s timeout
• a non-blocking concurrency limit
• validation of dimension, finiteness and norm
• a short cache keyed by query and generation
• a circuit breaker after repeated failures
• logs that record only a normalized query fingerprint, never raw text

Related-paper recommendations reuse the stored vectors as a single nearest-neighbor query with no model call. When a vector is missing, they fall back to a previous arXiv version or to task- and citation-based results (citations come from Semantic Scholar).

Evidence

All figures come from the 5,000-paper pilot on an L4 GPU:

• Throughput: about 75 papers/s at 1024 dims.
• Truncation: the same pass was materialized deterministically at 512 and 256 dims with no extra inference.
• Recall and latency: see the 256-dim versus exact-search numbers above.
• Hybrid quality: no retrieval-quality comparison of hybrid against single-branch search is reported.

How to use it

The full-corpus Job, run with the Bucket mounted via hf-mount:

hf jobs uv run \
  --flavor l4x1 \
  --timeout 6h \
  --volume hf://buckets/OWNER/pwc-paper-embeddings:/bucket \
  embed_papers_job.py \
  --input /bucket/runs/RUN_ID/input \
  --output /bucket/runs/RUN_ID/output \
  --model Qwen/Qwen3-Embedding-0.6B \
  --revision MODEL_REVISION \
  --dimensions 256 \
  --allow-matryoshka

The vector lookup query:

SELECT paper_id,
       embedding <=> CAST(:query_vector AS halfvec(256)) AS distance
FROM paper_embeddings
WHERE generation_id = :active_generation
ORDER BY embedding <=> CAST(:query_vector AS halfvec(256))
LIMIT 50;

The Inference Endpoint runs TEI with at most one replica and scale-to-zero; the page notes vLLM or SGLang would also work. embed_papers_job.py itself is not published on the page. The search is also exposed through a pwc search CLI and an agent Skill.

Limits

• All latency, recall and storage numbers come from the 5,000-paper pilot, not the 110k production corpus.
• Recall is measured as ANN against exact search, not as relevance.
• With scale-to-zero, cold starts are routine. Queries arriving during a cold start get lexical-only results.
• No reranker is used. The authors suggest Qwen3-Reranker as a possible addition at extra latency.
• The authors advise starting with keyword search and adding hybrid only if it measurably improves quality.