You cannot jump straight to autonomy

An auto-research and self-improvement environment cannot be built fully autonomous from the start. Raising the automation level before verification and recovery systems exist ships incidents faster than improvements. So divide maturity into stages and rise only when each stage's prerequisites are met.

The collection, verification, gate, observability, and recovery covered in this series are all prerequisites of specific maturity stages.

Maturity stages at a glance

L0 is fully manual, L1 automates human-run repetitive tasks, L2 is automatic investigation with human verification, L3 is automatic verification and gate-passing deployment, L4 is automatic self-improvement with observation-based halting, and L5 is limited autonomous operation. Each stage presupposes the prior stage's safeguards, and skipping causes regression.

Full guide: from planning to operations

In planning, define each maturity transition's prerequisites as numbers. For example, to go from L2 to L3, first achieve automatic-verification accuracy of 90% or higher, a rollback success rate of 99% or higher, and a gate false-pass rate of 1% or lower. Raising stages without prerequisites is the most common failure. Maturity is a risk-management tool, not a badge, so operating stably at a lower stage beats overreaching autonomy.

A dangerous failure pattern is skipping stages. Turning on automatic self-improvement without observability and recovery accumulates errors that can neither be detected nor reverted. To prevent this, actually verify the prior stage's safeguards on entering each stage, and set a demotion rule that automatically drops a stage when prerequisite metrics collapse. For recovery, if incidents recur at a higher stage, immediately drop to a lower stage to stabilize, then refill the prerequisites.

On the operations checklist, continuously surface the current maturity and prerequisite metrics. Record each stage's prerequisite status, recent demotion history, and per-stage incident rate in standard fields. Use stage duration, prerequisite-metric margin, demotion frequency, and per-stage improvement-to-incident ratio as observability fields. Keep masking rules so data used for stage judgment contains no personal information.

The continuous improvement loop re-examines quarterly whether each stage's prerequisites still hold. As traffic and risk change, the threshold values of prerequisite metrics must adjust too. A maturity model is not a ladder you finish climbing once but an operating frame that rises and falls with the situation to maintain safety.

Key takeaways

In short, an auto research loop should mature step by step. Define each stage's prerequisites from L0 to L5 as numbers, rise only after verifying the prior stage's safeguards, and demote when prerequisites collapse. With this series' collection, verification, gate, observability, and recovery in place, you can raise maturity safely without overreaching autonomy.

References

Anthropic Engineering