OpenAI Models Make Big Gains in COBOL, Fortran and Visual Basic Coding

Why low-resource languages matter

Low-resource languages have far less code and documentation available for models to learn from than Python or JavaScript. They still support critical infrastructure. OpenAI cites figures that COBOL supports more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is used in weather forecasting and nuclear record safety analysis.

The company says its work is guided by feedback from developers and enterprise customers, who told it that coding models often struggle with these languages. Maintaining such software is hard because documentation may be incomplete, original developers may have left, and important business rules may exist only in code. Teams fixing bugs or migrating to another language must preserve the behavior their systems depend on.

MultiLangBench: building SQLite from scratch

OpenAI developed two evaluations. The first, MultiLangBench, asks a model to build a SQLite-compatible database engine from scratch in a specified language, across 20 languages. The model must organize a codebase, manage state, handle errors and add features while preserving existing functionality. Database files serve as a shared interface: the candidate program receives an initial database and a SQL query and produces a modified database, which is checked against the reference SQLite implementation. Tests were adapted from SQLite's existing test suite.

At maximum reasoning effort, GPT-6 Astra reaches an average test pass rate of 87.4% across the 20 languages, up from 26.8% for GPT-5.6 Sol. OpenAI reports gains in every language tested:

• COBOL: 10.9% to 72.1%
• OCaml: 25.6% to 85.8%
• Fortran: 21.4% to 83.5%
• Visual Basic: 27.5% to 89.4%

Astra scores above 83% in 19 of the 20 languages, with COBOL the hardest at 72%.

GPT-6 Luna is positioned as the lower-cost option. It reaches an average of 40.3% and also improves on GPT-5.6 Sol in every language, including COBOL (10.9% to 17.1%), OCaml (25.6% to 34.1%), Fortran (21.4% to 31.4%) and Visual Basic (27.5% to 41.6%). Its estimated average API cost is $0.77 per sample, compared with $3.48 for GPT-5.6 Sol, approximately 78% lower. OpenAI is releasing the MultiLangBench setup, test suite and per-language results on GitHub.

Migrating COBOL while preserving behavior

The second evaluation asks models to migrate COBOL programs to C++, Java, Kotlin, Python or Rust while preserving what each program does. OpenAI notes that details such as rounding rules and missing-record handling often reflect business requirements. COBOL's fixed-point arithmetic, for example, discards excess decimal digits unless rounding is explicitly requested, so a rewrite that rounds can run without errors yet produce different financial results.

Hidden tests require byte-for-byte parity on observable outputs, and a migration counts as successful only if all tests pass. GPT-6.1 Sol achieves 86.6% accuracy at low reasoning effort, versus 5.4% for GPT-5.6 Sol at the same setting, at 61% of the API cost. At higher reasoning effort, accuracy reaches 97.9%, which OpenAI says brings the evaluation close to saturation.

What it means for teams

For teams maintaining older systems, the results point to more accurate and affordable code migration. OpenAI adds a caveat: applying this to production software still requires expert review and testing against the system's business rules and dependencies.

The company invites developers to try the models through the API on coding tasks in the languages their teams use, and to report where the models struggle and which languages or workflows should come next. Because the MultiLangBench setup is public, developers can also check performance in the languages they rely on and spot gaps that matter for their own work.