docs(devops): record team access, RBAC enablement and training infra

[Enskc05]

docs(devops): record team access, RBAC enablement and training infra

The setup log stopped at §11 (DVC, 16 July). This brings it current with what
was done on 7 August, in the same format as the rest of the document.

New sections

§12 — Team access and RBAC. The SSH model (shared root plus a per-person
key), the sshd drop-in, and the three things about it that are easy to get
wrong: the filename must start with 99- because cloud-init writes
PasswordAuthentication yes into 50-cloud-init.conf; PermitRootLogin must
be prohibit-password rather than no or the whole team is locked out; and
LogLevel VERBOSE is the only attribution a shared account has, which is why
the name comments in authorized_keys must not be disturbed. Also covers the
separate unprivileged account created for the ML team, and why a scoped
kubeconfig would have been pointless without it.

The setup log stopped at §11 (DVC, 16 July). This brings it current
was done on 7 August, in the same format as the rest of the document.

New sections

§12 — Team access and RBAC. The SSH model (shared root plus a
key), the sshd drop-in, and the three things about it that are easy to get
wrong: the filename must start with 99- because cloud-init writes
PasswordAuthentication yes into 50-cloud-init.conf; PermitRootLogin must
be prohibit-password rather than no or the whole team is locked
LogLevel VERBOSE is the only attribution a shared account has, whi
the name comments in authorized_keys must not be disturbed. Also covers the
separate unprivileged account created for the ML team, and why a scoped
kubeconfig would have been pointless without it.

§13 — Training infrastructure. The base-image decision, the registry
tagging rule, the dedicated ml-trainer MinIO user (rather than reusing
ml-team, whose password people also depend on), the /dev/shm default that
kills DataLoader workers, and the end-to-end verification result.

New error entries

#10 — RBAC was never enabled. The cluster ran with
--authorization-mode=AlwaysAllow for 24 days, so every ServiceAccount in it
was effectively cluster-admin and the RBAC manifests that had been written
were never enforced. The entry records the diagnosis path — a namespace-scoped
Role cannot grant cluster-wide access, so the permissions had to be
from somewhere else — along with the three pre-flight checks that made the fix
safe, and the lesson: on a new cluster, check the authorization mode itself
before trusting auth can-i.

#11 — MLflow OOMKilled during artifact upload. Why proxied uploads cost
memory, why the crash loop sustained itself (the client retried and
server again on each restart, so the Job has to be deleted first), and how the
limit was sized from a Prometheus measurement — kubectl top does not work on
this cluster.

Status and open work

The closing status now reflects RBAC being enforced, key-only SSH, and
training running as a Job. The open-work table is re-prioritised, wi
MLflow Postgres backup called out as the highest priority — it is the only
item on the list whose failure mode is unrecoverable. Resources that
applied by hand and never made it into GitOps (ml-trainer RBAC, monitoring)
now each have their own row, since both disappear if the cluster is