openova/platform/self-sovereign-cutover/chart/templates/rbac.yaml
hatiyildiz 9774d7ba53 feat(self-sovereign-cutover): add step 10 — pivot vCluster HelmReleases to Sovereign Harbor (Refs #2034)
The chart's own comment at platform/bp-mgmt-vcluster/chart/values.yaml:77-79
promised "post-handover, the per-Sovereign overlay rewrites to
`harbor.<sovereign-fqdn>/proxy-ghcr/...`" — but the rewrite step never
existed anywhere in the cutover sequence. As a result, every Sovereign
post-handover keeps pulling vCluster control-plane images from
`harbor.openova.io` indefinitely, a direct violation of Principle #11
(no tether to harbor.openova.io after handover). Caught by the TBD-V24
tether audit on 2026-05-20.

Why step 04 (containerd registries.yaml pivot) doesn't catch it:
registries.yaml.v2 only mirrors the 7 canonical UPSTREAMS (ghcr.io,
docker.io, registry.k8s.io, gcr.io, quay.io, xpkg.upbound.io,
public.ecr.aws). The host `harbor.openova.io` is treated as a literal
endpoint, not an upstream, so containerd routes those image pulls
direct to mothership Harbor regardless of mirror config.

This step adds:
- Phase 1: live `kubectl patch helmrelease` against each of
  {bp-mgmt-vcluster, bp-rtz-vcluster, bp-dmz-vcluster} in flux-system,
  patching BOTH `spec.values.<role>Vcluster.image.repository`
  (umbrella) AND `spec.values.vcluster.controlPlane.statefulSet.image.
  {registry,repository}` (loft-sh subchart). Topology-aware: secondaries
  skip MGMT (not present), primary skips RTZ (not present). Idempotent:
  re-runs no-op when already pivoted.
- Phase 2: git push to local Gitea injecting the same override blocks
  into clusters/_template/bootstrap-kit/{54,58,59}-bp-*-vcluster.yaml
  so the bootstrap-kit Kustomization doesn't revert the live patch on
  next reconcile (same pattern as step 06 Phase 2 + Phase 2.5).

Coordination with chart 0.1.34 (TBD-V25, PR #2036, already merged):
totalSteps bumped from "9" → "10" in 09-cutover-status-configmap.yaml.
Contract test (tests/cutover-contract.sh) asserts shift from 9 → 10
step ConfigMaps and from 8 → 9 job-mode ConfigMaps. New Case 21
verifies Step 10's wrapper + subchart patches are wired correctly.

RBAC: ClusterRole gains helm.toolkit.fluxcd.io.helmreleases
{update,patch}. Step-06 Phase-1.6 (the openova-catalog HR patch shipped
in chart 0.1.31) was silently relying on this verb already — chart
0.1.31's RBAC change was missed, so this bump ALSO closes a latent
permission gap that would have surfaced on any cluster where the prior
patch attempt happened to require it.

Operator note: existing actively-running vCluster Pods do NOT churn on
this step — they're already running with images pulled at startup. The
patch ensures the NEXT image-pull (chart bump, Pod restart, region
add) routes through the Sovereign-local Harbor.

Refs #2034 (NOT Closes — operator-walk on fresh prov + screenshot
required per CLAUDE.md §4 anti-theater discipline).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 04:18:04 +02:00

131 lines
6.4 KiB
YAML

{{- /*
RBAC for the cutover runner ServiceAccount.
CRITICAL — feedback_rbac_create_no_resourcenames.md (auto-memory anchor):
Kubernetes RBAC forbids combining `create` verbs with `resourceNames`.
A POST request has no resource name yet; the apiserver MUST evaluate the
rule against the request without a name match, and a `resourceNames` set
with `create` produces a 403 every time. This caused the bp-openbao
6+ provisioning loop. We split `create` into its own Rule with NO
`resourceNames` and keep `update/patch/get/delete` in a separate Rule
that may be name-scoped.
The cutover runner needs cluster-scope reach because:
- Step 02 reads bp-harbor admin Secret in `harbor` namespace.
- Step 02 reads ghcr-pull Secret in `flux-system` namespace.
- Step 03 only writes to local Harbor over HTTP (no K8s API).
- Step 05 patches GitRepository in `flux-system` namespace.
- Step 06 patches HelmRepositories in `flux-system` namespace, AND
(phase-0, #1184) merges harbor.<sov-fqdn> auth into the ghcr-pull
Secret in `flux-system` so source-controller can pull from the
per-Sovereign Harbor mirror without manual kubectl patch.
- Step 07 patches Deployment in `catalyst-platform` namespace.
- Step 08 creates+deletes a CiliumNetworkPolicy cluster-wide and
queries HelmRelease/Kustomization status across all namespaces.
- Registry-pivot DaemonSet writes to /etc/rancher/k3s/registries.yaml
on the host filesystem (hostPath, no K8s API needed beyond status
update on the cutover-status ConfigMap).
We use ClusterRole + ClusterRoleBinding because the namespaces touched
are heterogeneous and cluster-scoped resources (CiliumNetworkPolicy)
must be addressable.
*/ -}}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
labels:
{{- include "bp-self-sovereign-cutover.labels" . | nindent 4 }}
rules:
# ───────────────────────────────────────────────────────────────
# CREATE verbs — MUST be in their own Rule WITHOUT resourceNames.
# ───────────────────────────────────────────────────────────────
- apiGroups: [""]
resources: ["configmaps"] # cutover-status ConfigMap is created+updated by every step
verbs: ["create"]
- apiGroups: ["cilium.io"]
resources: ["ciliumnetworkpolicies"] # step 08 creates the egress-block NP
verbs: ["create"]
- apiGroups: [""]
resources: ["events"] # informational events posted as steps complete
verbs: ["create"]
- apiGroups: ["batch"]
resources: ["jobs"] # catalyst-api stamps Jobs in this namespace from step ConfigMaps
verbs: ["create"]
# ───────────────────────────────────────────────────────────────
# READ verbs — namespace-spanning per the step inventory above.
# ───────────────────────────────────────────────────────────────
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods", "pods/log"] # step 08 inspects pod status during the egress block window
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources: ["deployments", "daemonsets", "statefulsets"]
verbs: ["get", "list", "watch"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["get", "list", "watch"]
- apiGroups: ["source.toolkit.fluxcd.io"]
resources: ["gitrepositories", "helmrepositories", "ocirepositories"]
verbs: ["get", "list", "watch"]
- apiGroups: ["helm.toolkit.fluxcd.io"]
resources: ["helmreleases"]
verbs: ["get", "list", "watch"]
- apiGroups: ["kustomize.toolkit.fluxcd.io"]
resources: ["kustomizations"]
verbs: ["get", "list", "watch"]
- apiGroups: ["cilium.io"]
resources: ["ciliumnetworkpolicies"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["nodes"] # registry-pivot DaemonSet reports node status into the status ConfigMap
verbs: ["get", "list", "watch"]
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways"] # step 06 Phase -1 (#1871, chart 0.1.32): wait for cilium-gateway Programmed=True before URL rewrite
verbs: ["get", "list", "watch"]
# ───────────────────────────────────────────────────────────────
# UPDATE / PATCH / DELETE — separate from create per RBAC rule.
# ───────────────────────────────────────────────────────────────
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["update", "patch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["update", "patch"] # step 06 phase-0 merges harbor.<sov-fqdn> auth into ghcr-pull (#1184)
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["update", "patch"] # step 07 sets env on catalyst-api deployment
- apiGroups: ["source.toolkit.fluxcd.io"]
resources: ["gitrepositories", "helmrepositories"]
verbs: ["update", "patch"] # steps 05 + 06
- apiGroups: ["helm.toolkit.fluxcd.io"]
resources: ["helmreleases"]
verbs: ["update", "patch"] # step 06 phase-1.6 (catalog override) + step 10 vcluster-registry-pivot (TBD-V24 MISS-1)
- apiGroups: ["cilium.io"]
resources: ["ciliumnetworkpolicies"]
verbs: ["delete", "patch", "update"] # step 08 removes the policy at end-of-test
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
labels:
{{- include "bp-self-sovereign-cutover.labels" . | nindent 4 }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
subjects:
- kind: ServiceAccount
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}