A Large-Scale Measurement of AI Bill of Materials Completeness in Hugging Face Models

What it claims

The paper measures field-level completeness of AIBOMs generated from Hugging Face model repositories, and the operational detail the summary omits is which tool and what threshold: the authors mined a snapshot of 2,942,466 public HF model records, kept the 97,940 with more than 100 downloads, and ran each through the OWASP GenAI Security Project AIBOM Generator, which emits CycloneDX JSON and a completeness score out of 100. Finding: required structural fields are fully populated, but AI-specific transparency fields (documented limitations, safety-risk assessment, intended use, ethical considerations, environmental/carbon info, meaningful descriptions) are weakly represented or missing.

How it works

The OWASP generator takes a HF model ID, scrapes repository metadata, model-card content, license, config, and external references, then maps them into CycloneDX and scores across five weighted categories:

• Required fields (minimal CycloneDX structure for model identification)
• Metadata (AIBOM and model-purpose info)
• Component-basic information (model identity, licensing)
• Model-card documentation (description, parameters, evaluation, limitations, ethics)
• External references (source code, datasets, docs, model files)

The authors then parse the artifacts to measure individual field presence and cross-tabulate coverage against task, license availability, dataset declaration, paper reference, model family, architecture, parameter size, and format (RQ3). Section III-D holds the weights and scoring procedure, not reproduced in the fetched text.

Evidence

The scale claim is the contribution against prior work: the closest tool study, ALOHA, evaluated AIBOMs on a 312-model sample and flagged metadata/model-card standardization gaps; this study runs the same motivation across 97,940 artifacts. Per-field and per-category completeness numbers, and the RQ3 cross-tabulations, are in Section IV — not present in the fetched abstract-and-intro text, so the actual percentages are not available here.

How to use it

Experimental data and analysis are posted: https://figshare.com/s/185c21d7bf411fa4b952. If you generate AIBOMs for HF models, the takeaway is that the OWASP generator reliably fills CycloneDX-required structure but cannot manufacture AI-specific documentation the source repo never wrote down — so a passing structural validation does not mean the AIBOM carries provenance, safety, or environmental content.

Limits

The fetched text is abstract, introduction, and related work only; the concrete completeness scores, category weights, and RQ3 results tables are not included, so verify the actual numbers in Section IV before citing them. Scope is HF models with >100 downloads only (long-tail low-download models excluded), and completeness is measured on what the OWASP generator extracts — a specific tool, not AIBOM completeness in the abstract. The study measures documentation presence, not correctness or quality of the fields.