Autonomous Code Migration for Quantum Programming Languages: A Case Study with QED-C Benchmarks

What it claims

Three anonymized coding agents (Claude Opus, an OpenAI GPT model, and Gemini) each independently ported 13 QED-C Application-Oriented Benchmark methods from Qiskit to CUDA-Q. Every agent eventually produced a satisfactory port, but only after repeated prompting to stop shortcuts. Skills distilled from the first 11 migrations were then tested on two held-out methods, Shor's Methods 1 and 2. Against no-skill baselines, the skills cut model turns by 31–85% and tokens by 5–96%. The summary's "up to 85% / 96%" hides how wide those ranges are. The low end, a 5% token saving, is close to nothing.

How it works

flowchart LR
  Q[Qiskit reference<br/>13 methods] --> A1[Agent 1]
  Q --> A2[Agent 2]
  Q --> A3[Agent 3]
  A1 & A2 & A3 --> V{Validation}
  V -- fails --> P[Extra prompts<br/>and revision] --> V
  V -- passes --> X[Cross-agent assessment]
  X --> E[Ensemble-consolidated port]
  V -. first 11 migrations .-> S[Model-generated skills]
  S --> H[Held-out Shor 1 & 2<br/>self-use and cross-model]

• Isolation. Each agent ran in a clean context.
• Validation. Ports were checked with algorithm-specific fidelity measures plus extended tests.
• Effort and structure metrics. Each port was also characterized by:
• prompts needed
• turns and tokens
• lines of code
• gate structure
• Consolidation. Cross-agent assessment picked complementary strengths, which were merged into the final port.
• Skills. Skills captured idioms, pitfalls, workarounds and patterns. They were tested both by the model that wrote them and by other models (cross-model transfer).

Setup:

• QED-C suite at Git revision 2e7a983, with local compatibility updates for CUDA-Q 0.14.
• Existing CUDA-Q ports, especially Hidden Shift, served as structural templates for the agents. Others already in the suite: Bernstein–Vazirani M1, QFT M1, Phase Estimation, HamLib Hamiltonian Simulation M1–3.

The authors argue the migration is not syntactic. It requires several kinds of restructuring:

• Qiskit's QuantumCircuit builder plus transpilation has to become CUDA-Q kernels.
• Each execution needs the right launcher: sample, run or observe.
• Mid-circuit measurement, feed-forward, controlled/adjoint subroutines and parameterized computation need the most rework.
• The two frameworks can lower the same operation to different gate counts.

Evidence

The fetched text cuts off at the start of Methods. Per-benchmark fidelity, prompt counts, LOC and gate-structure results are not on the page shown. The only quantitative results available are:

• the 31–85% turn and 5–96% token reductions on the two Shor methods, and
• the qualitative findings that every model needed extra prompts and that no model was best on all benchmarks.

How to use it

• The near-complete CUDA-Q support and the generated skills are stated to be public in the QED-C benchmarks.
• The fetched text gives no repository path, skill file location or run commands.
• If you are porting to CUDA-Q, test against the stated baseline: QED-C revision 2e7a983 with CUDA-Q 0.14.

Limits

• Not a controlled comparison. The authors call it a case study because agentic runs are stochastic and path-dependent, and models and harnesses change quickly. Models are anonymized, so this is no guide to choosing a vendor.
• Narrow skill test. The skill evaluation covers only two held-out methods, both variants of Shor's algorithm.
• Not fully autonomous. Agents took shortcuts when the work got hard and needed human prompting to change course. Budget for supervision.
• Head start. Results depend on CUDA-Q templates that already existed in the repo. A repo without such examples may cost more effort.
• Silent wrongness. The authors stress that quantum code can run without errors yet differ algorithmically from the source. Correctness depends on the suite's fidelity checks.