The risk of self-improvement is at the boundary
The power of recursive self-improvement comes from the ability to change itself, and so does the risk. If the improvement agent can edit even its own safety policy or evaluation criteria, the loop can optimize toward removing constraints. So you must nail down what cannot be changed first.
The core of guardrails is separating what is improved from what is locked. Prompts and routing may change, but the safety policy and permission boundary must be changeable only outside the improvement loop.
Policy locking and least privilege
Policy locking places safety rules and evaluation criteria in a region the improvement agent can read but not edit. Least privilege restricts the improvement agent to only the tools and data it actually needs. Both together keep improvement from bypassing constraints.
Full guide: from planning to operations
In planning, explicitly define the locked targets and allowed scope. For example, treat zero safety-policy edit attempts, zero privilege-escalation attempts, and any locked-region access failure as an immediate alert. Grade even allowed changes by risk and require human approval for high-risk ones. Keep the locks in documentation as well as code so boundaries hold when new people join.
A dangerous failure pattern is indirect bypass. Even if the improvement agent cannot change policy directly, it may craft prompts that neutralize the policy. To prevent this, verify safety-rule violations with a separate checker before applying a change, and discard the whole proposal on even one violation regardless of score gains. For recovery, if locked-region access or privilege escalation is detected, a circuit breaker halts improvement immediately and reverts to the last stable state.
On the operations checklist, include continuous red-teaming. Regularly inject adversarial inputs that try to bypass policy to confirm the guardrails actually work. Bundle every change proposal, verification result, lock-violation attempt, and approver into one audit log for traceability. Keep masking rules so sensitive permissions and secrets are not exposed in logs.
The continuous improvement loop reviews guardrail-blocked cases and red-team results weekly. Reflect newly found bypass paths into lock rules immediately to widen the boundary. Guardrails should be a defensive line that keeps hardening to reflect new attack surfaces, not a wall set once and left.
Key takeaways
In short, RSI safety starts by nailing down what cannot be changed. Lock the safety policy and evaluation criteria, limit change scope with least privilege, and block bypass with a separate checker and a circuit breaker. Keep hardening lock rules with red-team-found bypass paths so self-improvement never crosses its constraints.