Skip to content

Back to the catalog

Solidity Meets LLMs: A Transformer-Based Approach to Smart Contract Vulnerability Detection
arxiv.orgpaper

Solidity Meets LLMs: A Transformer-Based Approach to Smart Contract Vulnerability Detection

This arXiv paper fine-tunes a BERT-based transformer model on a purpose-built labeled dataset of Solidity code fragments to classify them as vulnerable or safe, reporting a 92% F1 score. It demonstrates applying pre-trained LLM representations to capture Solidity-specific syntactic and semantic patterns for smart contract vulnerability detection, as an alternative or complement to traditional static analysis tools.

TL;DR A fine-tuned BERT transformer classifies Solidity code as vulnerable/safe with 92% F1, showing LLMs can be adapted for smart contract vulnerability detection.
Open original ↗
Document

What it is

A fine-tuned BERT model for binary (vulnerable/safe) classification of Solidity code fragments, introduced in the paper by Ghorab, Mokhati, and Ghorab. The paper's substantive contribution the summary omits is its labeling methodology and its taxonomy of five Solidity vulnerability classes.

How it works

The ground-truth dataset is built by scanning raw Solidity contracts (sourced from prior datasets and Etherscan-verified contracts) with two existing tools that use different detection strategies: Slither (symbolic execution) and SolScan (pattern recognition, modified by the authors to tag vulnerability type and line numbers automatically). Labeling rule: a contract is marked safe only if both tools agree it's safe; a detection by either tool is treated as vulnerable ("better safe than sorry"). For vulnerable contracts, Slither is run twice — a general scan, then a second scan with specific modules to classify the vulnerability type against the paper's taxonomy. Results are merged into records of the form:

contract_name<SEP>Safety<SEP>Type<SEP>Lines
flowchart TD
    A[Raw Solidity contract] --> B[Scan with Slither]
    A --> C[Scan with SolScan]
    B --> D{Both tools say safe?}
    C --> D
    D -->|Yes| E[Label: Safe]
    D -->|No, either flags it| F[Label: Vulnerable + type + line numbers]

Preconditions / Affected

The taxonomy used to classify vulnerability type covers five categories: cross-call (e.g., reentrancy), denial of service, arithmetic (overflow/underflow, division by zero, rounding), access control (e.g., tx.origin misuse), and bad randomness.

What to do

The paper positions this as a complement to, not full replacement for, existing scanners — ground truth itself depends on Slither and SolScan being correct. For evaluation/reuse, note the model works on source code fragments, not bytecode, and (per the authors' comparison table) is intended to handle partial/incomplete code better than AST- or bytecode-dependent tools like SmartCheck or Osiris.

Caveats

The labeling methodology is a double-edged sword: treating any single-tool detection as "vulnerable" inflates recall of the ground truth but inherits both tools' false positives, and the 92% F1 is measured against this tool-derived label, not an independently verified ground truth. The page text is truncated before the results/evaluation section, so the actual train/test split, dataset size, and per-category performance breakdown are not available from this fetch.

Technologies named here

  • Ethereum smart contracts
  • BERT-based transformer model
  • Solidity
  • Ethereum
  • LLM
  • BERT

Techniques named here

  • LLM-based static vulnerability detection

Related in AI Security