lemire.mearticleEphemeral Testing
Daniel Lemire proposes "ephemeral testing": instead of testing a software component directly, you have an AI agent build a throwaway application or layer on top of it, then judge the component's API quality by how easily the agent succeeds. A clean API with stable invariants lets the agent build quickly with few failures, while hidden state or poor docs produce patches and errors — treated as evidence about the underlying code's design, not the agent's competence. He describes repeating this with different agents and tasks against the same foundation as a cheap substitute for anticipating future integration needs.
TL;DR Lemire suggests testing a library's design by having an AI agent build and throw away integration layers on top of it, judging API quality by how smoothly the agent succeeds.
Organisations named here
- Daniel Lemire
Techniques named here
- Ephemeral testing
Related in coding-agents
