The chart's own comment at platform/bp-mgmt-vcluster/chart/values.yaml:77-79 promised "post-handover, the per-Sovereign overlay rewrites to `harbor.<sovereign-fqdn>/proxy-ghcr/...`" — but the rewrite step never existed anywhere in the cutover sequence. As a result, every Sovereign post-handover keeps pulling vCluster control-plane images from `harbor.openova.io` indefinitely, a direct violation of Principle #11 (no tether to harbor.openova.io after handover). Caught by the TBD-V24 tether audit on 2026-05-20. Why step 04 (containerd registries.yaml pivot) doesn't catch it: registries.yaml.v2 only mirrors the 7 canonical UPSTREAMS (ghcr.io, docker.io, registry.k8s.io, gcr.io, quay.io, xpkg.upbound.io, public.ecr.aws). The host `harbor.openova.io` is treated as a literal endpoint, not an upstream, so containerd routes those image pulls direct to mothership Harbor regardless of mirror config. This step adds: - Phase 1: live `kubectl patch helmrelease` against each of {bp-mgmt-vcluster, bp-rtz-vcluster, bp-dmz-vcluster} in flux-system, patching BOTH `spec.values.<role>Vcluster.image.repository` (umbrella) AND `spec.values.vcluster.controlPlane.statefulSet.image. {registry,repository}` (loft-sh subchart). Topology-aware: secondaries skip MGMT (not present), primary skips RTZ (not present). Idempotent: re-runs no-op when already pivoted. - Phase 2: git push to local Gitea injecting the same override blocks into clusters/_template/bootstrap-kit/{54,58,59}-bp-*-vcluster.yaml so the bootstrap-kit Kustomization doesn't revert the live patch on next reconcile (same pattern as step 06 Phase 2 + Phase 2.5). Coordination with chart 0.1.34 (TBD-V25, PR #2036, already merged): totalSteps bumped from "9" → "10" in 09-cutover-status-configmap.yaml. Contract test (tests/cutover-contract.sh) asserts shift from 9 → 10 step ConfigMaps and from 8 → 9 job-mode ConfigMaps. New Case 21 verifies Step 10's wrapper + subchart patches are wired correctly. RBAC: ClusterRole gains helm.toolkit.fluxcd.io.helmreleases {update,patch}. Step-06 Phase-1.6 (the openova-catalog HR patch shipped in chart 0.1.31) was silently relying on this verb already — chart 0.1.31's RBAC change was missed, so this bump ALSO closes a latent permission gap that would have surfaced on any cluster where the prior patch attempt happened to require it. Operator note: existing actively-running vCluster Pods do NOT churn on this step — they're already running with images pulled at startup. The patch ensures the NEXT image-pull (chart bump, Pod restart, region add) routes through the Sovereign-local Harbor. Refs #2034 (NOT Closes — operator-walk on fresh prov + screenshot required per CLAUDE.md §4 anti-theater discipline). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
131 lines
6.4 KiB
YAML
131 lines
6.4 KiB
YAML
{{- /*
|
|
RBAC for the cutover runner ServiceAccount.
|
|
|
|
CRITICAL — feedback_rbac_create_no_resourcenames.md (auto-memory anchor):
|
|
Kubernetes RBAC forbids combining `create` verbs with `resourceNames`.
|
|
A POST request has no resource name yet; the apiserver MUST evaluate the
|
|
rule against the request without a name match, and a `resourceNames` set
|
|
with `create` produces a 403 every time. This caused the bp-openbao
|
|
6+ provisioning loop. We split `create` into its own Rule with NO
|
|
`resourceNames` and keep `update/patch/get/delete` in a separate Rule
|
|
that may be name-scoped.
|
|
|
|
The cutover runner needs cluster-scope reach because:
|
|
- Step 02 reads bp-harbor admin Secret in `harbor` namespace.
|
|
- Step 02 reads ghcr-pull Secret in `flux-system` namespace.
|
|
- Step 03 only writes to local Harbor over HTTP (no K8s API).
|
|
- Step 05 patches GitRepository in `flux-system` namespace.
|
|
- Step 06 patches HelmRepositories in `flux-system` namespace, AND
|
|
(phase-0, #1184) merges harbor.<sov-fqdn> auth into the ghcr-pull
|
|
Secret in `flux-system` so source-controller can pull from the
|
|
per-Sovereign Harbor mirror without manual kubectl patch.
|
|
- Step 07 patches Deployment in `catalyst-platform` namespace.
|
|
- Step 08 creates+deletes a CiliumNetworkPolicy cluster-wide and
|
|
queries HelmRelease/Kustomization status across all namespaces.
|
|
- Registry-pivot DaemonSet writes to /etc/rancher/k3s/registries.yaml
|
|
on the host filesystem (hostPath, no K8s API needed beyond status
|
|
update on the cutover-status ConfigMap).
|
|
|
|
We use ClusterRole + ClusterRoleBinding because the namespaces touched
|
|
are heterogeneous and cluster-scoped resources (CiliumNetworkPolicy)
|
|
must be addressable.
|
|
*/ -}}
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRole
|
|
metadata:
|
|
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
|
|
labels:
|
|
{{- include "bp-self-sovereign-cutover.labels" . | nindent 4 }}
|
|
rules:
|
|
# ───────────────────────────────────────────────────────────────
|
|
# CREATE verbs — MUST be in their own Rule WITHOUT resourceNames.
|
|
# ───────────────────────────────────────────────────────────────
|
|
- apiGroups: [""]
|
|
resources: ["configmaps"] # cutover-status ConfigMap is created+updated by every step
|
|
verbs: ["create"]
|
|
- apiGroups: ["cilium.io"]
|
|
resources: ["ciliumnetworkpolicies"] # step 08 creates the egress-block NP
|
|
verbs: ["create"]
|
|
- apiGroups: [""]
|
|
resources: ["events"] # informational events posted as steps complete
|
|
verbs: ["create"]
|
|
- apiGroups: ["batch"]
|
|
resources: ["jobs"] # catalyst-api stamps Jobs in this namespace from step ConfigMaps
|
|
verbs: ["create"]
|
|
|
|
# ───────────────────────────────────────────────────────────────
|
|
# READ verbs — namespace-spanning per the step inventory above.
|
|
# ───────────────────────────────────────────────────────────────
|
|
- apiGroups: [""]
|
|
resources: ["secrets"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: [""]
|
|
resources: ["configmaps"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: [""]
|
|
resources: ["pods", "pods/log"] # step 08 inspects pod status during the egress block window
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments", "daemonsets", "statefulsets"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["batch"]
|
|
resources: ["jobs"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["source.toolkit.fluxcd.io"]
|
|
resources: ["gitrepositories", "helmrepositories", "ocirepositories"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["helm.toolkit.fluxcd.io"]
|
|
resources: ["helmreleases"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["kustomize.toolkit.fluxcd.io"]
|
|
resources: ["kustomizations"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["cilium.io"]
|
|
resources: ["ciliumnetworkpolicies"]
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: [""]
|
|
resources: ["nodes"] # registry-pivot DaemonSet reports node status into the status ConfigMap
|
|
verbs: ["get", "list", "watch"]
|
|
- apiGroups: ["gateway.networking.k8s.io"]
|
|
resources: ["gateways"] # step 06 Phase -1 (#1871, chart 0.1.32): wait for cilium-gateway Programmed=True before URL rewrite
|
|
verbs: ["get", "list", "watch"]
|
|
|
|
# ───────────────────────────────────────────────────────────────
|
|
# UPDATE / PATCH / DELETE — separate from create per RBAC rule.
|
|
# ───────────────────────────────────────────────────────────────
|
|
- apiGroups: [""]
|
|
resources: ["configmaps"]
|
|
verbs: ["update", "patch"]
|
|
- apiGroups: [""]
|
|
resources: ["secrets"]
|
|
verbs: ["update", "patch"] # step 06 phase-0 merges harbor.<sov-fqdn> auth into ghcr-pull (#1184)
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments"]
|
|
verbs: ["update", "patch"] # step 07 sets env on catalyst-api deployment
|
|
- apiGroups: ["source.toolkit.fluxcd.io"]
|
|
resources: ["gitrepositories", "helmrepositories"]
|
|
verbs: ["update", "patch"] # steps 05 + 06
|
|
- apiGroups: ["helm.toolkit.fluxcd.io"]
|
|
resources: ["helmreleases"]
|
|
verbs: ["update", "patch"] # step 06 phase-1.6 (catalog override) + step 10 vcluster-registry-pivot (TBD-V24 MISS-1)
|
|
- apiGroups: ["cilium.io"]
|
|
resources: ["ciliumnetworkpolicies"]
|
|
verbs: ["delete", "patch", "update"] # step 08 removes the policy at end-of-test
|
|
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: ClusterRoleBinding
|
|
metadata:
|
|
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
|
|
labels:
|
|
{{- include "bp-self-sovereign-cutover.labels" . | nindent 4 }}
|
|
roleRef:
|
|
apiGroup: rbac.authorization.k8s.io
|
|
kind: ClusterRole
|
|
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: {{ include "bp-self-sovereign-cutover.serviceAccountName" . }}
|
|
namespace: {{ .Release.Namespace }}
|