docs(ops): add fail-closed EVM migration runbooks

[mateeullahmalik]

Summary

Add fail-closed operator artifacts, scripts, and runbooks for manual EVM account/validator migration, based on the chain history-preservation PR.

This PR does not change consensus state transitions. It hardens release compatibility approval, mnemonic handling, relayer address verification, governance helper invocation, and repeatable rehearsal evidence.

Behavior change

• Add a JSON Schema-backed release compatibility manifest and blocked template.
• Require the destination pre-stage implementation to bind:
  • exact full argv;
  • a sole absolute executable path in argv[0];
  • tag, commit, source, SHA-256, and no-echo contract.
• Verify executable SHA immediately before array execution; no eval or mnemonic in argv.
• Fail closed unless Hermes key listing succeeds and exactly one named key derives the expected migrated destination address.
• Add manual operator runbooks for account, validator, SuperNode, relayer, evidence, rollback, and compatibility gates.
• Resolve governance helper compose paths relative to script location instead of caller cwd.
• Restrict plaintext devnet mnemonic registry lifecycle to mode 0600.
• Make recursive Makefile dry-run tests deterministic from a clean checkout.

Protocol/state-machine impact

• State keys: none changed by this PR.
• CheckTx / DeliverTx / BeginBlock / EndBlock: no chain behavior changes.
• Determinism / replay: unaffected; changes are off-chain scripts, schema, tests, and documentation.
• Migration: none.
• Upgrade handler: none beyond the prerequisite chain PR.

Operator scripts submit existing migration/governance messages; the chain remains the source of truth for accepted state transitions.

Operational invariants

• No migration proceeds from an unresolved or placeholder compatibility manifest.
• The executable used for destination pre-stage is represented once as absolute argv[0] and hash-verified immediately before execution.
• Mnemonics are never accepted in argv, echoed, or stored in world-readable registry files.
• Hermes process failure, missing key, duplicate key, or derived-address mismatch aborts before migration/restart.
• Validator migration requires explicit stopped-workload acknowledgement and post-restart verification.
• Logs/evidence are sanitized; no secret material belongs in artifacts or commits.

Failure modes and recovery

• Manifest/schema/compatibility failure: stop before key staging or broadcast.
• Hermes key-list failure or address mismatch: keep the relayer stopped; do not migrate or activate the key.
• Broadcast ambiguity: verify tx/migration record before retrying; do not blindly rebroadcast.
• Validator migration failure: keep the workload stopped, verify committed state, and follow the documented recovery matrix.
• Old SuperNode compatibility failure: treat as a release dependency, not an on-chain repair opportunity.

Known compatibility boundary

Exact v2.5.3-testnet SuperNode setup rejects destination-key recovery after EVM migration with a duplicate-address error and exits before daemon launch. On-chain registrations, histories, migrations, and delegations remain correct, but upload/download/action verification cannot be claimed with that old daemon after migration.

The compatibility manifest therefore remains fail closed until release owners bind and verify a compatible no-echo pre-stage implementation and supported daemon artifacts. This PR does not silently substitute a newer daemon or modify chain logic.

Risks

• Runbooks are intentionally conservative and may stop operators earlier than prior narrative guidance.
• Approved manifests are release-specific and must be regenerated/reverified for changed artifacts.
• Operational correctness still depends on service identity, custody controls, maintenance-window discipline, and exact network endpoints.

Rollback

• Revert this PR to restore the previous scripts/docs; no chain state rollback is required.
• Do not roll back or reseed production state merely because an operator prerequisite fails.
• If a migration transaction committed, use chain queries and the documented recovery matrix; do not rerun blindly.

Observability

Evidence surfaces include:

• compatibility-manifest validation;
• artifact checksums and build identities;
• transaction hashes and migration records;
• validator bonded/jailed/consensus status;
• exact delegation snapshots;
• SuperNode histories and relationships;
• sanitized service logs;
• block-height progression before/after restart.

Verification

• 210 script tests pass.
• Compatibility schema tests pass for blocked/approved, placeholder, relative-path, argv, and artifact cases.
• Hermes parser tests pass for exact match, mismatch, missing key, and duplicate key.
• Devnet common tests pass, including compose cwd resolution and account-registry permission lifecycle.
• ShellCheck and repository lint pass with 0 issues.
• Full unit/integration suite passes on the exact stacked source tree.
• Pre-restack exact candidate with identical source behavior:
  • commit 54ab2f8f81ed7b9a593fb585fa53987b7e90741d;
  • version 1.20.2;
  • binary SHA-256 e4de48318a0e23ac36698fc3c53863dabe5d010b2bd9bc83ea6ffec79de086fa.
• Final preserved-state census after rollout:
  • 5/5 validators synchronized and bonded;
  • 10/10 migration records;
  • 10/10 delegations;
  • continued block production;
  • no legacy control accounts remaining.

The current head differs by ancestry-only stack correction and is being revalidated by CI.

Stack order

1. fix(evmigration): preserve SuperNode history during migration #196 — protocol/history preservation
2. app: register v1.20.2 migration-only upgrade handler #198 — v1.20.2 migration-only upgrade handler
3. this PR — operator tooling and runbooks