Skip to content

Back to the catalog

openai.comarticle

Advancing Coding in Low-Resource Programming Languages

OpenAI describes two evaluations for coding in low-resource languages such as COBOL, Fortran and Visual Basic. MultiLangBench has a model build a SQLite-compatible database engine in 20 languages. The COBOL migration eval requires byte-for-byte output parity. GPT-6 Astra scores 87.4% on MultiLangBench (GPT-5.6 Sol: 26.8%), and GPT-6.1 Sol reaches 86.6% on COBOL migration at low effort. The setup is on GitHub.

TL;DR OpenAI releases MultiLangBench and a COBOL migration eval, showing large gains for GPT-6 models in low-resource languages.
Open original ↗

Organisations named here

  • OpenAI
  • IBM
  • National Oceanic and Atmospheric Administration
  • U.S. Nuclear Regulatory Commission

Technologies named here

  • GPT-6 Astra
  • MultiLangBench
  • SQLite
  • GPT-6 Luna
  • GPT-5.6 Sol
  • GPT-6.1 Sol
  • Rust
  • GitHub
  • OpenAI API
  • C++
  • COBOL
  • Fortran
  • Java
  • Kotlin
  • OCaml
  • Python
  • Visual Basic

Techniques named here

  • MultiLangBench database-file interface
  • Behavior-preserving COBOL migration test

Related in evals