2648 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
hatiyildiz	c9b558900d	fix(blueprints): blueprint.yaml lockstep for kyverno 1.2.1 + add kyverno-policies 1.0.0 blueprint.yaml	2026-05-20 02:40:45 +02:00
hatiyildiz	b71a37d520	fix(ci): register bp-kyverno-policies in expected-bootstrap-deps.yaml	2026-05-20 02:36:22 +02:00
hatiyildiz	fb952fadf9	feat(security/kyverno): split policies into bp-kyverno-policies@1.0.0 Blueprint ... Splits the 20 EPIC-1 (#1096) compliance ClusterPolicy templates out of bp-kyverno (engine umbrella chart) into a dedicated Blueprint bp-kyverno-policies@1.0.0 with its own HelmRelease, ordered via HR-to-HR dependsOn on bp-kyverno in the bootstrap-kit Kustomization. WHY (the bug we're killing): PR #1138 (2026-05-08) shipped 20 ClusterPolicy templates with `enabled: false` defaults → dead-on-arrival for 11 days. PR #1933 (2026-05-19) flipped 18 defaults to `enabled: true` + bumped chart 1.1.0 → 1.2.0 + bumped the bootstrap-kit pin — but hit a CRD install- ordering race on fresh prov t33: ClusterPolicy CRs (in templates/policies/baseline/*.yaml) and Kyverno CRDs (in upstream charts/crds/templates/) render in the SAME Helm pass, and the apiserver's RESTMapper has not yet learned kyverno.io/v1.ClusterPolicy when Helm applies the ClusterPolicy CRs. PR #1935 reverted ONLY the bootstrap-kit pin (1.2.0 → 1.1.0) — chart source kept claiming policies were on by default while the deployed pin pulled an engine-only artifact with zero policies. "Theater on theater" — founder walk on t34 confirmed GET /api/v1/sovereigns/<id>/compliance/policies returns `policyCount=0`, only `useraccess-boundary` (from bp-crossplane-claims) was installed. The structural fix is splitting the chart so the engine + CRDs reconcile + register first, THEN the policy chart applies its CRs cleanly. Audit mode default = non-blocking (admission still passes, PolicyReport rows populate). Operators flip individual policies to Enforce per-Sovereign overlay or via EnvironmentPolicy.spec.compliance.modes (slice C2 controller path — separate work item). CHANGES: 1. NEW chart `platform/kyverno-policies/chart/`: - Chart.yaml: name=bp-kyverno-policies, version=1.0.0, no subchart deps - values.yaml: `compliancePolicies:` block moved verbatim from bp-kyverno (defaults: 18 enabled+Audit, 2 intentionally OFF — `hubbleFlowsSeen` stub for W2 evaluator, `cosignVerified` until operator supplies PEM) - templates/baseline/01-..20-*.yaml: 20 ClusterPolicy templates moved via `git mv` (preserves blame; preserves PR #1933's 3 operator fixes — regex_match JMESPath + operator: Equals for 11/12/19) - tests/fixtures/: moved with the policies (fixtures reference policy output, not engine output) 2. ENGINE chart `platform/kyverno/chart/`: - Chart.yaml: 1.2.0 → 1.2.1 (policies removed, source no longer drift-claims compliance content) - values.yaml: `compliancePolicies:` block deleted (now lives in bp-kyverno-policies) - templates/clusterpolicy-mutate-add-openova-labels.yaml + ...require- openova-labels.yaml KEPT (engine-coupled mutating policies, EPIC-0 label-vocab E1/E2, defaults OFF — separate concern from EPIC-1 compliance library) - Empty `templates/policies/` directory removed 3. NEW bootstrap-kit slot `clusters/_template/bootstrap-kit/27a-kyverno- policies.yaml`: - HelmRelease bp-kyverno-policies pinned at chart `1.0.0` - HR-level `dependsOn: [bp-kyverno]` — same-kind, honored by Flux (per docs/INVIOLABLE-PRINCIPLES.md #14 cross-kind HR→Kustomization dependsOn is silently ignored, so we keep ordering at HR→HR within the single bootstrap-kit Kustomization) - targetNamespace: kyverno (same as engine — ClusterPolicy is cluster- scoped but the umbrella overlay namespacing matches the engine) - disableWait: true — Kyverno reports ClusterPolicy Ready asynchronously so we don't want downstream HRs stalling on policy-level health 4. UPDATED `clusters/_template/bootstrap-kit/kustomization.yaml`: - Added `27a-kyverno-policies.yaml` immediately after `27-kyverno.yaml` 5. BUMPED `clusters/_template/bootstrap-kit/27-kyverno.yaml`: - Engine pin 1.1.0 → 1.2.1 (engine-only; install behavior identical to 1.1.0 since policies + their values are no longer in this chart) VALIDATION (Principle #15 — validate against fresh state, not stable state): $ helm template bp-kyverno-policies platform/kyverno-policies/chart \ | grep -c '^kind: ClusterPolicy' 18 $ helm lint platform/kyverno-policies/chart && helm lint platform/kyverno/chart ==> 1 chart(s) linted, 0 chart(s) failed (both) $ helm template bp-kyverno platform/kyverno/chart \ | grep -c '^kind: ClusterPolicy' 0 # engine no longer renders any ClusterPolicy CRs $ helm package platform/kyverno-policies/chart Successfully packaged → bp-kyverno-policies-1.0.0.tgz (20 templates) CRD-race REPRODUCED locally without container runtime: applying the rendered policy YAML to a cluster WITHOUT Kyverno CRDs returns "no matches for kind \"ClusterPolicy\" in version \"kyverno.io/v1\" ensure CRDs are installed first" for every policy — proving the install-order fix is necessary. Full `helm install` from-scratch on Kind blocked locally (no container runtime on bastion); the Blueprint-Release CI workflow runs the full `helm dependency build` + package + GHCR push pipeline AND a `helm template` smoke render at publish time — that is the fresh-state Helm install gate before any pin lands. CI / GHCR (Principle #13): Blueprint-Release workflow auto-detects `platform/kyverno-policies/chart/**` and publishes `oci://ghcr.io/openova-io/bp-kyverno-policies:1.0.0` on push to main. The slot pin in 27a-kyverno-policies.yaml is set to `1.0.0` to match (auto-bump-pin step is a no-op when source version already matches the slot pin). DELIBERATELY OUT OF SCOPE: - W2 Go evaluator for `hubble-flows-seen` (stub stays a no-op) - Cosign publicKey supply path for `cosign-verified` - Per-Environment EnvironmentPolicy.spec.compliance.modes enforcement flip controller - Score-aggregator weight defaults configuration UI - `useraccess-boundary` (lives in bp-crossplane-claims, unchanged) This does NOT close #1096. The EPIC remains open until a fresh-prov walk shows `kubectl get clusterpolicies -A` returning the 18 baseline policies + useraccess-boundary, plus the AppDetail Compliance tab rendering non- zero policyCount. Founder closes #1096 after that walk. Refs #1096, Refs #2019, Refs #1929, Refs #1936	2026-05-20 02:32:54 +02:00
github-actions[bot]	5c2b934295	deploy: bump continuum-controller image to e72efb8	2026-05-20 00:14:25 +00:00
github-actions[bot]	b03deddda0	deploy: bump environment-controller image to e72efb8	2026-05-20 00:14:18 +00:00
github-actions[bot]	470c3950a4	deploy: bump useraccess-controller image to e72efb8	2026-05-20 00:14:12 +00:00
github-actions[bot]	440eb47233	deploy: bump application-controller image to e72efb8	2026-05-20 00:12:52 +00:00
e3mrah	e72efb87cd	chore(ci): add auto-bump-images + pkg/** path filter to all build-*-controller workflows (Closes #2006) (#2012) ... TBD-A69. PR #2005 fixed build-organization-controller.yaml only. The other six controller workflows (application, blueprint, continuum, environment, sandbox, useraccess) had the same gaps that caused the #1997 18h deploy gap: - application-controller: missing pkg/** in path filter (auto-bump already present from earlier work). - blueprint, continuum, environment, useraccess: missing BOTH pkg/** path filter AND auto-bump pipeline (permissions promotion + values.yaml bump + commit/push + blueprint-release dispatch). - sandbox: already complete (pkg/** + auto-bump to platform/sandbox chart) — left untouched. Each updated workflow inherits the canonical shape from build-organization-controller.yaml (PR #2005): 1. `core/controllers/pkg/**` added to BOTH push.paths and pull_request.paths. Without this, a fix that only touches the shared HTTP-client tree (gitea/keycloak/kc-mappers) silently fails to rebuild the controller image. 2. `permissions.contents: write` + `actions: write` so the build job can push the values.yaml bump and dispatch the downstream chart re-publish. 3. An awk-scoped `Bump controllers.<who>.image.tag in values.yaml` step that updates ONLY the targeted controller's tag (verified locally — sibling tags remain untouched). 4. A commit/push step that bumps products/catalyst/chart/values.yaml (or products/continuum/chart/values.yaml for continuum, which has its own chart). 5. A `gh workflow run blueprint-release.yaml` dispatch so the bot-pushed commit fires the downstream chart re-publish (GitHub Actions silently filters bot pushes from path-trigger workflows otherwise). Adds two new files to lock the shape in: - `scripts/check-controller-workflow-uniformity.sh` — a CI regression test that grep-asserts every controller workflow has the canonical pkg/** filter + auto-bump pipeline. Fails loudly if any new controller workflow ships without the canonical shape, or if an existing one regresses. - `.github/workflows/check-controller-workflow-uniformity.yaml` — push-on-touch + pull_request-on-touch event-driven wrapper that runs the script. Mirrors the shape of check-vendor-coupling.yaml. Verified locally: - YAML syntax valid for all 7 controller workflows + the new check workflow. - Regression script passes on all 7 controller workflows. - Simulated awk bumps against products/catalyst/chart/values.yaml and products/continuum/chart/values.yaml — each script bumps ONLY the targeted controller's tag, sibling tags untouched. No chart bumps. No Go/chart changes. CI-workflow-only. Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 04:11:04 +04:00
github-actions[bot]	c194767187	deploy: update catalyst images to e2ba34a	2026-05-20 00:10:59 +00:00
e3mrah	e2ba34a70f	fix(self-sovereign-cutover): make state-resume idempotent across orchestrator restart (#2018) ... The bp-self-sovereign-cutover orchestrator stuck at step 5/9 on t38 2026-05-19 when catalyst-api restarted mid-cutover. The in-process runCutover goroutine died; the durable status ConfigMap captured the in-flight state but NOTHING auto-fired the engine on the fresh Pod. The chart's auto-trigger Helm Job only runs on post-install / post-upgrade hooks; a catalyst-api Pod restart AFTER the chart is already installed leaves the cutover stranded. Step 09 (gitea-token-mint) was never created → PR #2008's provisioning init-container blocked forever waiting for the cutover-step-09 token annotation → tenant onboarding flow stuck (Pillar 1 + 4 + 5 blocked). Root cause (cutover.go, lines 770-790): the engine reads `priorStatus` on a fresh /start call and skips steps where result==success, but only HandleCutoverStart / HandleCutoverInternalTrigger can trigger that code path. No startup hook → no auto-resume. Additionally, in-flight step rows whose result==running stay "running" forever in the durable record. Fix (single PR, no chart changes — purely catalyst-api Go code): 1. Handler.ResumeInterruptedCutover(ctx) — new exported method that reads the cutover status ConfigMap, detects in-flight cutovers (cutoverComplete=false AND cutoverStartedAt!=""), resets every step row whose .result=="running" back to "" (so the engine treats it as not-yet-attempted), and spawns runCutover with a background context. 2. cmd/api/main.go — call h.ResumeInterruptedCutover(ctx) just before ListenAndServe so a startup-resume race against a stale auto- trigger Job retry is serialised through the in-process running flag (tryStartRun). 3. createCutoverJob — Create-or-Get on AlreadyExists (concurrent trigger fires from operator CTA + auto-trigger Job hitting catalyst-api simultaneously is now benign). Tests (cutover_test.go): - TestResumeInterruptedCutover_ResumesAndCompletes — seeds 3-step status with step-1 success, step-2 running, step-3 untouched. Asserts after resume: step-1 NOT re-run, step-2 re-run, step-3 run, cutoverComplete=true. - TestResumeInterruptedCutover_NoOpWhenComplete — already-done status produces zero Job creates. - TestResumeInterruptedCutover_NoOpWhenNeverStarted — empty cutoverStartedAt MUST not pre-empt the chart's auto-trigger Job. Chart bump: bp-catalyst-platform 1.4.219 → 1.4.220 + bootstrap-kit pin in lockstep (clusters/_template/bootstrap-kit/ 13-bp-catalyst-platform.yaml). No bp-self-sovereign-cutover chart changes — every step PodSpec is already idempotent by design. Refs #2016 Co-authored-by: hatiyildiz <hatiyildiz@openova.io>	2026-05-20 04:08:00 +04:00
github-actions[bot]	4568298c9b	deploy: bump sandbox-mcp-server image to 0ee94cb	2026-05-20 00:06:21 +00:00
e3mrah	0ee94cb7bc	fix(continuum-witness/cfkv): stabilise RenewExtendsTTLAndBumpsGeneration (unblocks pre-existing CI red) (#2014) ... Root cause: CFKVClient.Renew compared the server-stamped ExpiresAt against the client's wall-clock (time.Now()). The Cloudflare Worker is the timestamping authority — ExpiresAt is in the Worker's clock frame. Whenever the Worker's clock and the client's wall-clock diverged (NTP skew, fake-clock tests, or simply the test fixture clock pinned to 2026-05-09 while CI runs on a later date), the client's check declared the lease expired and Renew returned ErrLeaseLost — even though the Worker still considered the lease healthy. This caused the Build continuum-controller workflow to red on every push since 2026-05-09 with: --- FAIL: TestCFKV_ContractSuite/RenewExtendsTTLAndBumpsGeneration contract.go:214: Renew: witness: lease lost --- FAIL: TestCFKV_ContractSuite/GenerationMonotonicityAcrossOps contract.go:298: Renew: witness: lease lost Fix: remove the client-side wall-clock expiry check. Expiry is enforced server-side — an expired renew returns 412, which write() already maps to ErrLeaseHeldByAnother, which the Renew wrapper then re-maps to ErrLeaseLost. This keeps a single source of truth for "is the lease alive" (the Worker), avoiding the dual-clock disagreement. The non-holder early return (cur.Holder != holder -> ErrLeaseLost) is preserved because it never depended on time. Validation: - TestCFKV_ContractSuite/RenewExtendsTTLAndBumpsGeneration GREEN - All 14 contract suite sub-tests GREEN - ./continuum/internal/witness/cloudflarekv/... -count=10 GREEN - All ./continuum/... packages GREEN Refs #2012 Co-authored-by: Emrah Baysal <emrah.baysal@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 04:04:44 +04:00
e3mrah	271855d419	fix(bp-sandbox): correct default NEWAPI_BASE_URL to actual bp-newapi service name (#2017) ... The bp-sandbox chart defaulted `env.newapiBaseURL` to `http://newapi.newapi.svc.cluster.local:3000`. That assumes the bp-newapi ClusterIP Service is named bare `newapi`. In practice the canonical service name rendered by `helm template newapi platform/newapi/chart/ -s templates/service.yaml` is `newapi-bp-newapi`, because `bp-newapi.fullname` in `platform/newapi/chart/templates/_helpers.tpl` emits `{Release.Name}-{Chart.Name}` and `clusters/_template/bootstrap-kit/ 80-newapi.yaml` sets `releaseName: newapi` against chart `bp-newapi`. The bootstrap-kit overlay at `clusters/_template/bootstrap-kit/19a-bp-sandbox.yaml` does NOT override `env.newapiBaseURL`, so every Sovereign's sandbox-controller resolved a DNS name no Service ever publishes: POST /admin/tokens/sandbox → lookup newapi.newapi.svc.cluster.local on 10.43.0.10:53: no such host Walker on t38 (chart 1.4.216, substrate be4f78bc872e2c56, 2026-05-19) caught the live regression. Every qwen-code Sandbox session failed at TokenMint, blocking the canonical Pillar-4 customer journey (console.<orgslug>.omani.homes → Sandbox → qwen-code provisions additional app). Fix scope: - platform/sandbox/chart/values.yaml: default flipped to `http://newapi-bp-newapi.newapi.svc.cluster.local:3000`. - platform/sandbox/chart/templates/deployment.yaml: inline `default` in the env block matched. - platform/sandbox/chart/Chart.yaml: bp-sandbox 0.3.0 -> 0.3.1. - clusters/_template/bootstrap-kit/19a-bp-sandbox.yaml: pin 0.3.0 -> 0.3.1 in lockstep (Inviolable Principle #14). Verification: - `helm template bp-sandbox platform/sandbox/chart/ -s templates/deployment.yaml` with required values set renders the env literal `value: "http://newapi-bp-newapi.newapi.svc.cluster.local:3000"`. - `helm template newapi platform/newapi/chart/ -s templates/service.yaml` renders `metadata.name: newapi-bp-newapi`. DoD per anti-theater discipline (CLAUDE.md §0): issue stays open until a fresh-prov Sandbox session successfully mints a NewAPI token and reaches qwen-code. This PR ships the source-of-truth env-var fix only; it does NOT defensively retry alternate names in the dial path. Refs #2015 Co-authored-by: hatiyildiz <hatice.yildiz@openova.io>	2026-05-20 03:52:53 +04:00
github-actions[bot]	a7a2a9f5fe	deploy: bump sandbox-mcp-server image to 5b44a66	2026-05-19 23:44:37 +00:00
e3mrah	5b44a66991	fix(blueprint-controller): align mode enum with bp-*-vcluster blueprint files (unblocks pre-existing CI red) (#2013) ... Two tiers of placement modes coexist in the Blueprint corpus but only one was registered in the validator + CRD enum, causing TestValidate_ExistingBlueprintCorpus to fail on the 4 bp-*-vcluster blueprints since 2026-05-09: - Application-tier (marketplace 99%): single-region / active-active / active-hotstandby - Bootstrap-topology tier (docs/SOVEREIGN-MULTI-REGION-DOD.md A4): primary-only / secondary-only / every-region The 4 affected blueprints (bp-mgmt-vcluster / bp-dmz-vcluster / bp-rtz-vcluster / bp-vcluster-helmrepo) correctly use the bootstrap- topology tier — these are NOT operator-selectable; they document which regions the bootstrap layer auto-installs the chart into. Extends: - validate.go canonicalPlacementModes with the three bootstrap modes + inline documentation of the two-tier taxonomy - blueprint.yaml CRD enum (placementSchema.modes.items + .default) kept in sync per the validator's "must mirror" comment - 4 new unit-test cases for the bootstrap-topology modes Result: TestValidate_ExistingBlueprintCorpus 71/71 GREEN (previously 67/71, 4 FAIL). Unblocks #2012 and every other PR touching blueprint-controller. Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 03:42:54 +04:00
hatiyildiz	3de581c906	deploy(bp-catalyst-platform): bump bootstrap-kit pin 1.4.218 -> 1.4.219 (auto, Refs TBD-A6)	2026-05-19 23:23:16 +00:00
github-actions[bot]	497f782d17	deploy: update sme service images to 59367b2 + bump chart to 1.4.219	2026-05-19 23:22:40 +00:00
e3mrah	59367b2fe5	fix(billing): transactional voucher redemption — only decrement on order.placed success (Closes #2000) (#2011) ... t38 walk caught the canonical TBD-V9 bug: customer redeems voucher WALK-T38-2138 on a 50 OMR order, voucher credit is only 10 OMR, Stripe is unconfigured in the Sovereign, Checkout returns 503 "payment processor not configured" — but promo_codes.times_redeemed had already advanced 0→1, promo_redemptions row was inserted, and a credit_ledger grant was written. Voucher shows "Exhausted 1/1" with no order to show for it; the customer's one-per-customer promo is silently burned. Root cause: store.RedeemPromoCode runs its own transaction (necessary for the FOR UPDATE concurrency cap) and commits the three side effects up front. The rest of the Checkout pipeline (GetCreditBalance, GetSettings, CreditOnlyCheckout, Stripe customer + session creation) can fail without undoing the redemption. Fix (saga / compensating action): - store.RollbackPromoCodeRedemption(customerID, code) — single tx that DELETEs promo_redemptions, decrements times_redeemed (GREATEST(..,0) underflow guarded), and DELETEs the credit_ledger redeem grant (filter reason='promo:<code>' AND order_id IS NULL so order spend ledger rows are not touched). Idempotent: 0-row DELETE on promo_redemptions short-circuits the rest, so re-running a failed checkout never double-decrements. - handlers.Checkout tracks voucherRedeemed and calls RollbackPromoCodeRedemption on every downstream failure: settings load, Stripe-unconfigured 503 (the t38 walk path), CreateOrder failure, Stripe customer rejection, Stripe session rejection, plan-price unresolvable. - Voucher only stays committed once (a) CreditOnlyCheckout commits the order+spend+sub transactionally and order.placed fires, or (b) the Stripe Checkout Session URL is handed back to the customer (canonical abandoned-cart: credit persists on ledger for the next attempt). Tests: - store_test.go: three new tests cover the rollback contract — happy path (all three side effects undone in one tx), idempotent no-op when no redemption row exists, empty-args no-op (no DB hit). - checkout_test.go: TestCheckout_VoucherPartialCover_StripeUnconfigured_RollsBackRedemption is the t38 regression — full sqlmock walk asserting the rollback tx fires before the 503 response. bp-catalyst-platform Chart.yaml + bootstrap-kit pin bumped 1.4.214 → 1.4.215. Co-authored-by: Claude Code (hatiyildiz) <claude@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 03:21:34 +04:00
github-actions[bot]	3b2b9c2ffd	deploy: update Catalyst marketplace image to 17ea3f3	2026-05-19 23:12:49 +00:00
e3mrah	17ea3f3873	fix(marketplace): post-checkout redirects to console.<slug>.<pool-tld> not operator console (Closes #2001) (#2010) ... TBD-V10 — t38 walk: after successful /redeem + /checkout the customer was redirected to the operator console URL (`console.<sov-fqdn>`) instead of the per-tenant console (`console.<slug>.<sov-fqdn>`). Root cause: `core/marketplace/src/lib/config.ts::deriveConsoleURL` mapped `marketplace.<sov-fqdn> → console.<sov-fqdn>`, never prepending the tenant slug. PR #1993 (TBD-A67) restored the `console.` prefix in the chart-side HTTPRoute (tenant-public-routes.yaml) AND the runtime organization-controller's tenant_route.go (both emit `console.<slug>.<parentDomain>` byte-identically), but the marketplace JS that does the post-checkout redirect never picked up the slug- prefixed shape. Fix --- - `src/lib/config.ts`: `deriveConsoleURL(slug?)` now splices the slug as the left-most label when the marketplace host is `marketplace.<sov-fqdn>`. Slug source: explicit arg → localStorage (`sme-active-org-slug`) → fallback to slug-less operator host. Exported pure helper `composeTenantConsoleURL(host, slug)` for testability. Mothership (`marketplace.openova.io`) and partner vanity hosts unchanged. - `src/lib/api.ts`: new `setActiveOrgSlug()`. `logout()` clears both `sme-active-org-slug` and `sme-checkout-tenant-slug`. - `src/components/CheckoutStep.svelte`: persist `tenant.slug` to `sme-checkout-tenant-slug` BEFORE the Stripe hop so the cross- origin return can re-stamp it; call `setActiveOrgSlug(tenant.slug)` on credit-covered path; pass the slug through `consoleHref(..., { slug })` for the redirect navigation. - `src/layouts/Layout.astro`: inline returning-user redirect now pulls the slug from the live-orgs response (preferring the org matching `sme-active-org`) and stamps `sme-active-org-slug` before redirecting to `console.<slug>.<sov-fqdn>`. Validation ---------- - `playwright/customer-journey.spec.ts` step 16 extended with the brief's exact assertion: `marketplace.omani.homes` + slug `demo` → `https://console.demo.omani.homes`. Plus regression guards for multi-label sov-fqdn (`marketplace.t38.omani.works` + `acme` → `console.acme.t38.omani.works`), mixed-case slug lowercasing, empty/ null slug falling back to operator host, and mothership ignoring the slug. - `git grep '\.openova\.io"' core/marketplace/src/` returns ZERO new hits introduced by this PR (existing references are the tenant table for `omantel.openova.io` and the canonical mothership host guard — both intentional). - `npm run build` clean on the affected files (Astro static export including CheckoutStep.svelte rebuild). Chart bump ---------- - products/catalyst/chart/Chart.yaml: 1.4.213 → 1.4.214 - clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml pin: 1.4.213 → 1.4.214 Refs: PR #1993 (TBD-A67 console-prefix chart fix), #1949 (/redeem) Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 03:11:51 +04:00
hatiyildiz	d4b995c551	deploy(bp-catalyst-platform): bump bootstrap-kit pin 1.4.215 -> 1.4.216 (auto, Refs TBD-A6)	2026-05-19 23:03:59 +00:00
github-actions[bot]	0785e3470e	deploy: update sme service images to b190566 + bump chart to 1.4.216	2026-05-19 23:03:15 +00:00
e3mrah	b190566c40	fix(sme-notification): align JWT signing secret with catalyst-api bridge (Closes #1999) (#2009) ... TBD-V8: voucher email never delivered. On t38 canonical walk (agent a550281a, 2026-05-19 21:37:33Z) operator issued voucher, row persisted, HTTP 200 returned, but recipient IMAP stayed empty. catalyst-api logs showed sme/notification returning 401 to the downstream dispatch. Trace (end-to-end, per docs/INVIOLABLE-PRINCIPLES.md #1): FE → catalyst-api → SME gateway → billing → notification catalyst-api → gateway → billing wire is correct: catalyst-api mints an HS256 bridge token from the operator's RS256 Keycloak session via sharedauth.MintSMEAccessToken (signed with the reflector mirror of sme-secrets/JWT_SECRET into catalyst-system), gateway and billing both verify HS256 with the same bytes. billing → notification wire was broken: billing's sendVoucherIssuedEmail (core/services/billing/handlers/vouchers.go) POSTed with only Content-Type — NO Authorization header. notification's HTTP surface is gated by the shared HS256 JWTAuth middleware (core/services/shared/middleware/jwt.go); a missing header returns 401 silently. The voucher upsert already persisted so the operator saw 200, but no email ever landed. TBD's hypothesis ("JWT signing-secret mismatch between catalyst-api and sme/notification") was incorrect — both Pods already read from the SAME sme-secrets/JWT_SECRET (chart templates/sme-services/billing.yaml line 67-71 and notification.yaml line 47-51, both pointing at the same secretKeyRef). The real gap was that billing never USED those bytes to mint an outbound service token. Fix (Go-side only, no chart-template change): 1. Add JWTSecret []byte to billing's Handler struct (core/services/billing/handlers/handlers.go). 2. Wire it in core/services/billing/main.go from the same JWT_SECRET env the inbound JWTAuth middleware already consumes. 3. In sendVoucherIssuedEmail, mint a 5-minute HS256 service token via sharedauth.MintSMEAccessToken (the SAME helper catalyst-api's RS256→HS256 bridge uses, so the wire contract is symmetric) and forward it as Authorization: Bearer <token>. Claims: sub="sme-billing", role="superadmin", typ="session". 4. Empty JWTSecret falls back to the legacy no-header path so a stale chart that doesn't wire JWT_SECRET into billing doesn't crash the voucher upsert (mirrors optional:true on catalyst-api's CATALYST_SME_JWT_SECRET secretKeyRef). Tests: - TestIssueVoucher_SendsAuthorizationHeader: exercises the full round- trip. Billing mints with the test bytes; we re-parse the captured token with the SAME bytes (the exact path notification's JWTAuth middleware takes on receive) and assert claim shape — sub, role, typ, exp. Pre-fix the captured request had no Authorization header so this would have failed at the first check. - TestIssueVoucher_NoAuthHeader_WhenJWTSecretUnset: back-compat guard for the legacy no-secret path. - All pre-existing TestIssueVoucher_* tests still pass. Chart bumped 1.4.213 → 1.4.214 and bootstrap-kit pin in clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml updated to match. Validation: - go test ./core/services/billing/... → PASS (3 packages) - helm template products/catalyst/chart --set ingress.marketplace.enabled=true → both sme/billing and sme/notification Deployments read JWT_SECRET from secretKeyRef.name=sme-secrets, key=JWT_SECRET. Refs #1842 (D28 voucher email arrival), #1829 (D29 customer journey). Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 03:02:02 +04:00
github-actions[bot]	886c48d4e2	deploy: update catalyst images to cdd7eac	2026-05-19 22:45:02 +00:00
e3mrah	cdd7eac20a	fix(bp-sme): wait for gitea user-bootstrap before provisioning starts (Closes #2002) (#2008) ... TBD-V11 / Issue #2002. On t38 fresh prov, sme/provisioning Pod logged `HTTP 401 user does not exist [uid: 0, name: ""]` on the first tenant Org CR creation. Root cause: provisioning Pod started with the chart's first-install placeholder GITHUB_TOKEN (the Gitea admin password mirrored verbatim by provisioning-github-token.yaml — enough to clear Container- ConfigError but NOT a valid Gitea API token). Step 09 of bp-self- sovereign-cutover later mints a real API token + patches the Secret + rollout-restarts the Pod, but the FIRST tenant journey always 401'd because the Pod was already serving with the bad placeholder. Approach (B): add an init container `wait-for-cutover-token` to the SME provisioning Deployment that polls the Secret for the cutover annotation `catalyst.openova.io/token-source: self-sovereign-cutover- step-09` (stamped by Step 09 alongside the minted token bytes). The Pod stays in Init:0/1 until Step 09 has actually completed, then the main container starts with a guaranteed-valid token. Default poll budget = 10s × 180 = 1800s (covers Hetzner cold-start ~18m + slack). Why NOT HelmRelease.dependsOn: - Per Principle #14, HR.dependsOn → Kustomization is silently ignored. - bp-self-sovereign-cutover HR is dormant + disableWait:true: it goes Ready=True at install BEFORE Step 09's Job actually runs. Adding it to bp-catalyst-platform.dependsOn would buy nothing. - Pod-level init gating waits on the actual condition (Secret annotation set by Step 09), not on a proxy. Why NOT change bp-self-sovereign-cutover trigger order: - Step 09 must run AFTER bp-catalyst-platform creates the Secret (otherwise the patch has no target). Reordering would break the inverse dependency. Why NOT a Job that bootstraps the user upfront: - Step 09 already mints the token; we don't need a second bootstrap. - The bug is timing, not absence of bootstrap. Files changed: - products/catalyst/chart/templates/sme-services/provisioning.yaml: add initContainers block gated on smeServices.provisioning.waitForCutoverToken.enabled (default true). Re-uses existing `provisioning` SA (already has secrets get/list/watch in `sme` ns via sme-provisioning ClusterRole — no new RBAC). - products/catalyst/chart/values.yaml: add smeServices.provisioning.waitForCutoverToken.{enabled,image, intervalSeconds,timeoutSeconds} block. - products/catalyst/chart/Chart.yaml: bump 1.4.213 → 1.4.214 with full TBD-V11 changelog entry. - clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml: bump HelmRelease pin 1.4.213 → 1.4.214 (chart bump only delivers the fix when the pin moves — TBD-A68 / 1.4.213 precedent). Validation: - `helm template` Sovereign-mode render shows the init container in the provisioning Deployment with kubectl-poll loop. - Default-values smoke render unaffected (gate is ingress.marketplace.enabled=true; smoke uses defaults where false). - `helm lint products/catalyst/chart/` passes. - Contabo-Zero render path safe by construction (chart only renders the Deployment when ingress.marketplace.enabled=true; contabo doesn't enable marketplace via this chart). Closes #2002. Refs #1829 (D29 tenant materialisation gate). Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 02:43:01 +04:00
github-actions[bot]	a090477aa1	deploy: bump organization-controller image to 8d8ce40	2026-05-19 22:33:22 +00:00
github-actions[bot]	102521cb71	deploy: bump sandbox-controller image to 8d8ce40	2026-05-19 22:33:13 +00:00
github-actions[bot]	f9ea3c9c9e	deploy: bump sandbox-mcp-server image to 8d8ce40	2026-05-19 22:32:00 +00:00
e3mrah	8d8ce40045	fix(build-organization-controller): add missing auto-bump pipeline + pkg/** path filter + wire-level test (Refs #1997) (#2005) ... Followup hardening for #1997 (PR #2004 catch-up bumped the organization-controller chart pin to c9b58ea). PR #2004 unblocks t38 right now, but the underlying cause — `build-organization-controller.yaml` has no auto-bump step and its path filter misses `core/controllers/pkg/**` — is still live and will re-strand the next gitea-client fix the moment it lands. This PR closes both gaps so the bug cannot recur. Two surgical additions: 1. `.github/workflows/build-organization-controller.yaml` a. Promote `permissions.contents: read` → `write` (+ `actions: write`), mirroring `build-application-controller.yaml`. b. Add `Bump controllers.organization.image.tag in values.yaml` step (awk-scoped to the `organization:` block only — cannot accidentally bump a sibling controller's tag). c. Add `Commit and push values.yaml bump` step (rebase-safe, skip-if-no-change). d. Add `Dispatch blueprint-release for chart re-publish` step — anti-recursion bypass for the GH-Actions rule that bot pushes don't fire downstream workflows. Without this the rebuilt image is NEVER baked into a new chart version. e. Add `core/controllers/pkg/**` to push + pull_request path filters. The shared HTTP-client tree (gitea, keycloak, kc-mappers, …) is COPYed into every Group C controller's image via the Containerfile, so a change to it MUST rebuild. PR #1910 only triggered a rebuild because it happened to also touch `organization_controller_test.go`; a pure pkg/ fix would silently skip the workflow. 2. `core/controllers/pkg/gitea/client_test.go` New `TestCreateOrg_HitsOrgsEndpointWithAuth` — wire-level regression guard that: - Fails hard if the client EVER hits `/api/v1/admin/orgs` (would catch a refactor accident that re-introduces the Gitea 1.22+ 405 bug regardless of which chart pin is deployed). - Asserts the request is `POST /api/v1/orgs` exactly once. - Asserts the request carries `Authorization: token <hex>` with the exact expected value (defense-in-depth: even if the URL is right, Gitea 1.22+ still returns 405 without the token). Sibling controllers (environment, blueprint, useraccess, …) likely have the same missing-auto-bump + missing-pkg/** path filter. NOT fixing them in this PR — blast-radius discipline. Follow-up recommended: audit every `build-*-controller.yaml` for both gaps. Validation: • go vet ./pkg/gitea/... — clean • go test -race ./pkg/gitea/... — ok, all pre-existing + new tests pass • go test -run TestCreateOrg_HitsOrgsEndpointWithAuth -v — PASS Refs #1997 (PR #2004 closed the immediate symptom; this PR closes the deploy gap so #1997 cannot recur) Refs #1910 (the original /admin/orgs → /orgs code fix) Refs #1829 (D29 customer journey hardening) Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 02:29:59 +04:00
e3mrah	1ca37ea7f8	fix(bp-valkey): default auth.enabled=false to match bp-newapi passwordless REDIS_CONN_STRING (Closes #2003) (#2007) ... Pre-1.0.2 bp-valkey shipped `valkey.auth.enabled: true` (bitnami default) while bp-newapi's REDIS_CONN_STRING default was the passwordless URL `redis://valkey-primary.valkey.svc.cluster.local:6379`. On every freshly-franchised Sovereign the newapi Pod CrashLoopBackOff'd 45x on the Redis ping probe with `NOAUTH Authentication required` — caught on t38 sandbox walk 2026-05-20. This is the Pillar-4 verifier-killing bug for the Sandbox + qwen-code + MCP end-user DoD (#1986). Approach A (simpler, this PR): flip bp-valkey's default to `auth.enabled: false` so the upstream bitnami chart exports `ALLOW_EMPTY_PASSWORD=yes` to the Valkey container. Verified via `helm template` — the render now contains: - name: ALLOW_EMPTY_PASSWORD value: "yes" Other in-cluster consumers tolerate the change: - products/catalyst sme-services (auth.yaml + gateway.yaml) read VALKEY_PASSWORD via `secretKeyRef ... optional: true` and fall back to the no-auth connect path in core/services/shared/db/valkey.go when the value is empty. - products/catalyst projector wraps the password Secret mount in `{{- with .Values.services.projector.valkey.passwordSecret }}` so an absent Secret simply skips the password env var. Approach B (deferred): make bp-newapi mirror the bp-valkey auto-generated password Secret into the newapi namespace and template it into REDIS_CONN_STRING. Larger scope, tracked under #2003 follow-up. Changes: - platform/valkey/chart/values.yaml — auth.enabled: true → false - platform/valkey/chart/Chart.yaml — version 1.0.1 → 1.0.2 - platform/valkey/blueprint.yaml — spec.version + configSchema default - clusters/_template/bootstrap-kit/17-valkey.yaml — chart pin 1.0.1 → 1.0.2 Verified: - `helm dependency build` succeeds (bitnami/valkey 5.5.1 unchanged) - `helm template` renders `ALLOW_EMPTY_PASSWORD=yes` on the Pod - tests/observability-toggle.sh — all 4 cases PASS Closes #2003 Refs #1986 Co-authored-by: hatiyildiz <catalyst@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 02:26:56 +04:00
github-actions[bot]	c45d10a1d9	deploy: update catalyst images to f90d697	2026-05-19 22:23:46 +00:00
e3mrah	f90d697846	fix(chart): bump organization-controller 72e3f08 -> c9b58ea so PR #1910's gitea-client fix actually ships (Closes #1997) (#2004) ... TBD-A68: t38 walkthrough on 2026-05-19 21:41Z (chart 1.4.211) put two tenant Organization CRs (walkdemo38, walk-t38-2138) into Ready=False/GiteaOrgFailed with `POST .../api/v1/admin/orgs HTTP 405`. Investigation showed the code fix already landed on main as PR #1910 (merged 2026-05-19 03:59Z, commit f442c28): `gitea.EnsureOrg` now hits `POST /api/v1/orgs` (the user-token endpoint) instead of the admin-only `/api/v1/admin/orgs` that returns 405 to the in-cluster service-account token. The build-organization-controller workflow successfully produced fresh images at f442c28 and then again at c9b58ea (most recent main- HEAD push touching the controller, 2026-05-19 20:58Z). The bug on t38 was deployment-time: the chart's image pin at products/catalyst/chart/values.yaml:369 still pointed at `72e3f08` from 2026-05-10 across three subsequent chart bumps (1.4.210 / 1.4.211 / 1.4.212). The CI auto-bump-images job covers SME images only, not controller images, so this class of stale pin slips through. Filing TBD-A69 separately to close that CI gap. Files (pure deployment-pin update, no code change): - products/catalyst/chart/values.yaml:369 tag: "72e3f08" -> tag: "c9b58ea" - products/catalyst/chart/Chart.yaml version + appVersion 1.4.212 -> 1.4.213, changelog entry added. - clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml version: 1.4.212 -> 1.4.213, changelog entry added. Validation: - `helm template products/catalyst/chart | grep organization-controller` -> `image: "ghcr.io/openova-io/openova/organization-controller:c9b58ea"` - `grep -c "72e3f08" <helm template output>` -> 0 - GHCR manifest probe for c9b58ea returns HTTP 200 with application/vnd.docker.distribution.manifest.v2+json (image exists and is pullable by the in-cluster ghcr-pull secret). Post-deploy expectation: - organization-controller Pod rolls to c9b58ea on `helm upgrade`. - Controller logs flip from `POST /api/v1/admin/orgs HTTP 405` (every 30s) to `POST /api/v1/orgs 201` on the existing stuck Organization CRs. - walkdemo38 + walk-t38-2138 auto-recover to Ready=True without operator intervention (gitea EnsureOrg is idempotent; the reconcile loop will re-fire and succeed). - Unblocks D29 tenant-org provisioning chain (Keycloak group + vCluster + tenant URL HTTPRoute + WordPress install all gate on the Organization CR being Ready). Closes #1997 Refs #1829 (D29 tenant onboarding), #1842, #1945, #1910 (the upstream code fix this chart bump finally ships). Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 02:21:46 +04:00
e3mrah	ebfc59c18e	fix(bp-flux-stuck-hr-recovery): grant helmreleases/status patch RBAC + log stderr (Closes #1995) (#1998) ... * fix(bp-flux-stuck-hr-recovery): grant helmreleases/status patch RBAC + log stderr (Closes #1995) Agent ae9d7638 verifying PR #1991 on t38 (2026-05-19 21:18Z) found the bp-flux-stuck-hr-recovery CronJob correctly detected bp-alloy in `Ready=Unknown for 427s, history[0].status=deployed` state, entered the TBD-A66 branch B, and attempted the patch — but the in-Pod `kubectl patch hr --subresource=status` silently failed because its stderr was swallowed by `2>&1` into the same /dev/null pipe as stdout. A manual identical patch from bastion succeeded immediately, so RBAC was not the blocker. Investigation: the 1.2.3 ClusterRole already grants `helmreleases` + `helmreleases/status` patch+update verbs (it was added in PR #1991 to enable the new branch in the first place). The actual root cause of the silent failure was diagnostic-blind: the script could not distinguish a successful patch from a failing one, so the human-readable `RECOVER ... — patching` log line emitted in both cases. Fix (1.2.4): - Capture `kubectl patch --subresource=status` stderr to a tempfile under /tmp (the writable emptyDir mount) so multi-line apiserver errors survive intact. - Emit three structured `[A66]` log lines that operators / agents can grep: detection: `[A66] HR <ns>/<name> Ready=Unknown for <age>s, history[0]=deployed → attempting patch` success: `[A66] HR <ns>/<name> patched to Ready=True` failure: `[A66] HR <ns>/<name> patch FAILED: <stderr>` - Same treatment for the annotation-rollback path so a stuck idempotency annotation can also be diagnosed. - Add Case 8 to leader-election-and-recovery.sh asserting: * detection / success / failure log lines render in the script * the `>/dev/null 2>&1` pattern is no longer on the critical `kubectl patch --subresource=status` line * stderr is captured via `mktemp /tmp/a66-patch-err.XXXXXX` Chart 1.2.3 -> 1.2.4; bootstrap-kit pin 03-flux.yaml bumped in lockstep (bootstrap-kit pin-sync check passes for bp-flux). Refs #1989 (TBD-A66). Closes #1995. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(bp-flux): bump blueprint.yaml spec.version 1.2.3 → 1.2.4 in lockstep with Chart.yaml manifest-validation's TestBootstrapKit_BlueprintCardsHaveRequiredFields + TestBootstrapKit_BlueprintVersionLockstepSweep require blueprint.yaml spec.version to track Chart.yaml version exactly (TBD-A20 / #1856). Forgotten in the previous commit. Refs #1995. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: hatiyildiz <hatiyildiz@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 01:55:14 +04:00
github-actions[bot]	9183bc938f	deploy: update catalyst images to 3d20ee3	2026-05-19 21:40:14 +00:00
e3mrah	3d20ee35bf	fix: purge 5 .openova.io leaks — tenant users now reach their Sovereign not mothership (Closes #1994) (#1996) ... Five surgical fixes for TBD-A68 (#1994) — every tenant-facing URL the catalyst-api / SPA / chart could emit now follows the Sovereign FQDN the deployment is bound to, instead of hardcoding the mothership host. 1. products/catalyst/bootstrap/api/internal/handler/auth.go PIN email plaintext + HTML bodies now read SOVEREIGN_FQDN env via a new pinEmailLoginURL() helper. Chroot mode (SOVEREIGN_FQDN set) emits `https://console.<fqdn>/login`; mothership mode keeps the historical `https://console.openova.io/sovereign/login`. The HTML visible-link text is also derived from the resolved host. 2. core/console/src/lib/config.ts MARKETPLACE_URL / CHECKOUT_URL / MARKETPLACE_HOME_URL now lazy- resolve via resolveMarketplaceOrigin() — Astro public env `PUBLIC_MARKETPLACE_ORIGIN` first, runtime `window.location.host` second (strip `console.<slug>?` + prepend `marketplace.`), legacy `https://marketplace.openova.io` fallback for SSR snapshots. 3. products/catalyst/chart/templates/sme-services/configmap.yaml CORS_ORIGIN_PUBLIC / CORS_ORIGIN_ADMIN / CORS_ORIGIN_GATEWAY / PUBLIC_BASE_URL / PUBLIC_API_BASE_URL / CNAME_TARGET / CHECKOUT_SUCCESS_URL / CHECKOUT_CANCEL_URL now templated against `marketplace.<global.sovereignFQDN>` + sibling platform zone. Catalyst-Zero render (no sovereignFQDN, no host override) keeps the legacy `sme.openova.io` byte-identical so contabo's existing CORS / public URLs don't drift. 4. products/catalyst/chart/templates/sme-services/notification.yaml Notification Deployment's CORS_ORIGIN env now sources from the shared `sme-services-config.CORS_ORIGIN_PUBLIC` key instead of hardcoding `https://sme.openova.io`. Per-Sovereign FQDN substitution flows through automatically. 5. Regression test: TestPinEmail_SovereignFQDNRoutesLoginURL in auth_pin_test.go covers both modes (chroot routes to sovereign console; mothership keeps openova.io target) and asserts the HTML body never routes tenant traffic through openova.io when SOVEREIGN_FQDN is set. Validation: - `helm template products/catalyst/chart --set global.sovereignFQDN=t38.omani.works` renders ZERO openova.io strings in CORS / PUBLIC_BASE_URL / CHECKOUT keys. Catalyst-Zero render preserves the legacy sme.openova.io paths. - `go test ./internal/handler/` passes 101.4s (full suite + new TestPinEmail regression test). Chart bump: bp-catalyst-platform 1.4.211 -> 1.4.212 + bootstrap-kit pin in clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml. Closes #1994 Co-authored-by: hatiyildiz <claude@anthropic.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 01:38:01 +04:00
hatiyildiz	675e863082	deploy(bp-catalyst-platform): bump bootstrap-kit pin 1.4.210 -> 1.4.211 (auto, Refs TBD-A6)	2026-05-19 21:00:59 +00:00
github-actions[bot]	999236d7e3	deploy: update sme service images to c9b58ea + bump chart to 1.4.211	2026-05-19 21:00:03 +00:00
e3mrah	c9b58eacca	fix(tenant-route): restore console.<slug>.<parent> prefix + drop .openova.io hardcode (Closes #1990) (#1993) ... TBD-A67: three surgical fixes for the tenant org URL drift between the founder's spec (`console.<slug>.<parent>` per CLAUDE.md §0) and the runtime emit. Pre-fix the controller emitted `<slug>.<parent>` while the chart-side overlay AND sme_tenant_gitops.go:536 emitted `console.<slug>.<parent>`; tenant onboarding emails on every non- openova.io Sovereign leaked the platform marketing host into the WorkspaceURL. Files (three production paths + symmetric tests): - core/controllers/organization/internal/controller/tenant_route.go:113 -> emits `console.<subdomain>.<parentDomain>` so the runtime reconciler and the chart-side overlay produce byte-identical HTTPRoute shapes. - products/catalyst/chart/templates/sme-services/tenant-public-routes.yaml:82 -> chart-side analogue mirrors the new console-prefixed shape. - core/services/notification/handlers/enrich.go -> WorkspaceURL now `https://console.<sub>.<parentZone>` where parentZone comes from a new TENANT_PARENT_DOMAIN env (same name the provisioning service uses for Handler.TenantParentDomain). Empty parent zone yields empty URL — NEVER falls back to `.openova.io`, restoring compliance with the "never touch openova.io" rule on per-Sovereign deployments. Tests: - new enrich_test.go: 5 truth-table cases on the pure workspaceURL helper + 2 end-to-end Lookup cases. Hard regression guard that the rendered URL contains neither a missing `console.` prefix nor a leaked `openova.io` substring. - organization_controller_test.go: TenantPublic_RendersHTTPRoute assertion bumped from `acme.omani.homes` to `console.acme.omani.homes` + HasPrefix("console.") regression guard. Chart bump: bp-catalyst-platform 1.4.209 -> 1.4.210; bootstrap-kit pin in clusters/_template/bootstrap-kit/13-bp-catalyst-platform.yaml follows. Refs #1990 TBD-A67. Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 00:57:57 +04:00
e3mrah	71e8101363	fix(bp-flux-stuck-hr-recovery): detect+correct deployed-but-unknown-Ready HRs (Refs #1989) (#1991) ... * fix(bp-flux-stuck-hr-recovery): detect+correct deployed-but-unknown-Ready HRs (Refs #1989) t37 canonical walk on nbg1-2 / hel1-1 secondary CPs surfaced a second stuck-HR failure mode: helm-controller completes the install — the HR's own `.status.history[0].status` flips to "deployed" — but apiserver flap on the slow secondary CP loses the write that flips `.status.conditions[type=Ready]` from Unknown to True. The existing suspend-toggle recovery (issue #925) does NOT fix this because helm- controller's "release in storage" short-circuit returns yes on every subsequent reconcile, so it never re-evaluates Ready. This PR extends the stuckHelmReleaseRecovery CronJob with a second detection branch: for hr where .status.conditions[type=Ready].status == "Unknown" AND age(Unknown) > stuckThreshold (default 5m) AND .status.history[0].status == "deployed" AND metadata.annotations["stuck-hr-recovery.openova.io/auto-corrected-at"] == "" → kubectl annotate hr stuck-hr-recovery.openova.io/auto-corrected-at=<RFC3339> → kubectl patch hr --subresource=status --type=merge status.conditions=[{type:Ready, status:True, reason:ReconciliationSucceeded, message:"auto-corrected from deployed-but- unknown-Ready by stuck-hr-recovery (TBD-A66)", lastTransitionTime:<RFC3339>}] Safety / idempotency: - Annotation acts as both audit trail AND idempotency guard. Re-runs on an already-corrected HR skip immediately. - If the status patch fails, the annotation is rolled back so the next CronJob run re-attempts. - Guardrail unchanged: >10 acted-on HRs in a single run → exit 1 + operator alert. - The 10-HR guardrail spans BOTH branches combined. RBAC additions: - helmreleases/status with verbs [patch, update] — status subresource is a separate RBAC target in Kubernetes. Without this rule `kubectl patch --subresource=status` returns 403. Validation: - tests/leader-election-and-recovery.sh: 6 → 7 cases (existing 6 issue #925 cases still PASS; new Case 7 covers TBD-A66 — script contains history[0].status check, status-subresource patch verb, audit annotation key, helmreleases/status ClusterRole verb, and operator-greppable "auto-corrected from deployed-but-unknown-Ready" audit string). - Mock JSONPath replay against 4 synthetic HRs: branch B routes deployed-but-unknown to status patch, branch A still handles pending-install via the secret check, idempotency annotation correctly skips re-run, healthy Ready=True HR is no-op. Chart bump: - platform/flux/chart/Chart.yaml: 1.2.2 → 1.2.3 - clusters/_template/bootstrap-kit/03-flux.yaml: bp-flux HR pin 1.2.2 → 1.2.3 (the existing pin for omantel/otech live clusters sits at 1.1.3 — unchanged, those clusters are pre-#925 baseline). Closure note: - Refs #1989 (not Closes — closure happens when the t37 canonical walk reaches handover successfully on a fresh prov). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(bp-flux): bump blueprint.yaml spec.version 1.2.2 → 1.2.3 (lockstep with Chart.yaml) Companion to TBD-A66 / #1989 bump. CI gate `TestBootstrapKit_BlueprintVersionLockstepSweep` (TBD-A20, #1856) asserts blueprint.yaml spec.version == chart/Chart.yaml version per platform/*. Missed this in the parent commit because the older bp-flux bumps (1.2.1 → 1.2.2 etc.) did not require this companion bump back when the lockstep gate didn't exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: claude-bot <claude-bot@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 00:38:25 +04:00
github-actions[bot]	20f4923201	deploy: bump sandbox-pty-server image to d55ed45	2026-05-19 20:38:16 +00:00
e3mrah	d55ed45a7c	feat(sandbox-pty-server): agent catalogue + lazy-spawn on attach (Refs #1986 B3) (#1992) ... Adds the agent-slug -> binary mapping inside pty-server, closing the B3 wiring hole identified in TBD-P4 #1986. Design source: /tmp/p4-b3-design-spec.md (agent abfeafd7, 2026-05-19). Files touched: - products/sandbox/pty-server/internal/agentcatalog/agentcatalog.go (NEW) Hardcoded 7-row table: 6 real-agent slugs in lock-step with the FE / catalyst-api / chart-CRD enum, plus sovereign-shell as a rescue row that's always present (black-screen prevention). Lookup / AllSlugs / Resolve API + optional JSON override at /etc/openova/sandbox-agents.json (path overridable via OPENOVA_SANDBOX_AGENTS_PATH). - products/sandbox/pty-server/internal/agentcatalog/agentcatalog_test.go (NEW) 7 unit tests: known slugs / unknown slug / override file / override-supersedes-builtin / argv shape / env-merge precedence / AllSlugs sorted+exhaustive + upstream-catalogue drift guard. - products/sandbox/pty-server/internal/agentcatalog/export_test_helpers.go (NEW) ResetCache helper for sibling-package tests. - products/sandbox/pty-server/internal/server/routes.go createRequest gains Agent + ExtraArgs + EnvMap fields. Exactly one of {agent, command} required; unknown slug -> 400 with the canonical list (NOT bash fallback); RequiredEnv presence check surfaces missing wiring at create time. New lazySpawn helper wires WS /sessions/{id}/attach to either ?agent= query or SANDBOX_DEFAULT_AGENT env so the FE stays zero-touch when the controller renders that env from spec.agentCatalogue[0]. - products/sandbox/pty-server/internal/server/routes_test.go 9 HTTP-level tests covering happy path / unknown slug 400 / both set / neither set / missing required env / backward-compat command path + 4 lazy-spawn scenarios (env-set, query-overrides, neither -> 404, unknown slug surfaces invalid-agent). - products/sandbox/pty-server/internal/session/manager.go +CreateWithID for the lazy-spawn path, where the session id is the Sandbox CRD name (carried in the WS URL) rather than a pty-server-minted hex string. Design notes preserved: - No new MCP env-injection code. The controller already renders every relevant env var (NEWAPI_URL, OPENAI_*, LLM_GATEWAY_*, ANTHROPIC_*, SANDBOX_*) on the pty-server StatefulSet at gitops/manifests.go:321-359; session.New passes os.Environ() through to exec.Cmd.Env at session.go:89. - No chart bump. SANDBOX_DEFAULT_AGENT is consumed only if rendered; pty-server falls back to the historic 404 behaviour when the env is empty (forward-compat with current chart). - B3-followup (SANDBOX_* rename on the pty-server StatefulSet to match #1987's MCP Deployment) is deferred per the design spec. Refs #1986 Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 00:35:32 +04:00
github-actions[bot]	ef5377896f	deploy: bump sandbox-pty-server image to 29a7df0	2026-05-19 20:18:18 +00:00
e3mrah	29a7df0975	feat(sandbox-pty-server): bundle qwen-code + claude-code + aider + opencode in agent-runner image (Refs #1986 B1) (#1988) ... The pty-server image used `distroless/static-debian12:nonroot` which shipped only the Go pty-server binary. `exec.Command("qwen-code")` returned ENOENT — Pillar 4 of the end-user DoD (customer picks `qwen-code` in Sandbox → agent launches with MCP) could not work on any prov regardless of the controller/MCP wiring. Swap the final stage to `node:22-bookworm-slim` and install the four publicly fetchable agents architecture.md §1+§7 promises: qwen-code npm @qwen-code/qwen-code (Node) claude-code npm @anthropic-ai/claude-code (Node) opencode npm opencode-ai (Node) aider pip aider-chat (Python venv) Symlink the slug form (`qwen-code`, `claude-code`) over the short binary names the npm packages expose (`qwen`, `claude`) so the existing `exec.Command(<slug>)` shape lights up without waiting on B3 (the slug→binary registry). `cursor-agent` is intentionally not bundled — Cursor's product shape is a cloud-hosted IDE companion, not a self-hosted CLI; the analogous bring-your-own bridge for hosted vendors lives in `claude-code-byos.md`. Non-root posture preserved (runs as `node` uid 1000). `tini` added for clean PID-1 signal propagation on session DELETE. Image grows ~580 MiB (distroless 14 MiB → ~600 MiB) — worth it: the four agents are the Sandbox surface, and Pillar 4 cannot be GREEN without them on PATH. Chart bump: bp-sandbox 0.2.0 → 0.3.0 in both `platform/sandbox/chart/ Chart.yaml` and `clusters/_template/bootstrap-kit/19a-bp-sandbox.yaml` so the next bootstrap-kit reconcile picks up the runtime image bump the build-sandbox-pty-server workflow will commit on push. Refs #1986 (TBD-P4 umbrella — B2 newapi default, B3 slug registry, B4 MCP env-var drift remain). Co-authored-by: Hatice Yildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 00:15:33 +04:00
github-actions[bot]	dfdc5dc3e0	deploy: bump sandbox-controller image to edf8665	2026-05-19 20:08:38 +00:00
github-actions[bot]	26919d184e	deploy: bump sandbox-mcp-server image to edf8665	2026-05-19 20:06:57 +00:00
e3mrah	edf8665a03	fix(sandbox-controller): emit canonical SANDBOX_* env vars for MCP plugin (Refs #1986) (#1987) ... TBD-P4 B4 — env-var name drift between the sandbox-controller and the MCP plugin silently degraded every MCP tool family to "not configured" at runtime. The controller emitted bare `ORG_ID` and `SOVEREIGN_FQDN` on every rendered MCP Deployment while the MCP binary (products/sandbox/mcp-server/internal/tools/env.go) reads the namespaced canonical `SANDBOX_ORG_ID` / `SANDBOX_SOVEREIGN_FQDN`. Per agent a99ea3aa's investigation, six additional env-var families the MCP requires were never wired at all. Surgical alignment across renderer + chart + controller wiring: 1. core/controllers/sandbox/internal/gitops/manifests.go — MCP Deployment template renamed the bare names AND grew env entries for the canonical set the MCP plugin reads: Rename (MCP Deployment only; pty-server StatefulSet keeps the bare names since they are inherited into the user's agent shell — that is a distinct contract): ORG_ID -> SANDBOX_ORG_ID (tool family: all) SOVEREIGN_FQDN -> SANDBOX_SOVEREIGN_FQDN (tool family: all) Added (the MCP plugin was reading them; controller wasn't emitting): SANDBOX_ID -> identifies the Sandbox CR SANDBOX_NAMESPACE -> rendered ns sandbox-<owner-uid> SANDBOX_TENANT_ID -> scopes marketplace/byod handler SANDBOX_GITEA_BASE_URL -> sandbox.deploy / gitea tool family SANDBOX_GITEA_TOKEN (secret) -> ditto, via secretKeyRef optional SANDBOX_DOMAIN_API_URL -> marketplace tool family SANDBOX_MARKETPLACE_API_URL -> marketplace tool family SANDBOX_STORAGE_S3_ENDPOINT -> sandbox.storage tool family SANDBOX_STORAGE_S3_REGION -> ditto SANDBOX_STORAGE_S3_USE_TLS -> ditto SANDBOX_STORAGE_S3_ACCESS_KEY -> ditto, via secretKeyRef optional SANDBOX_STORAGE_S3_SECRET_KEY -> ditto, via secretKeyRef optional KEYCLOAK_ADMIN_URL -> sandbox.auth tool family KEYCLOAK_PARENT_REALM -> ditto KEYCLOAK_ADMIN_TOKEN (secret) -> ditto, via secretKeyRef optional 2. platform/sandbox/chart — bp-sandbox HR surfaces the new wiring as chart-level values (mcp.giteaBaseURL, mcp.domainAPIURL, mcp.storage.*, mcp.keycloak.*) defaulting to the in-cluster Service DNS of a stock Sovereign install. Per-Sovereign overlays may override any value. Secrets are NEVER written from this chart — name+key references only with `optional: true` so a fresh-prov Sovereign with a credential source in flight does NOT crash the per-Sandbox MCP Pod; the affected tool family surfaces a clean "not configured" error at call time (matches the MCP plugin's existing per-tool guard pattern). 3. Chart.yaml + bootstrap-kit pin (19a-bp-sandbox.yaml) bumped to 0.2.0 so the per-Sovereign overlay picks up the new env surface on the next reconcile. 4. sandbox_controller_test.go — extended deployment-mcp.yaml assertion block to assert the canonical SANDBOX_* env-var set + value plumbing AND added a negative assertion that the bare `ORG_ID` / `SOVEREIGN_FQDN` names MUST NOT appear on the MCP Deployment (they remain on the pty-server StatefulSet, distinct contract). Regression test against future re-introduction of the drift. Validation: - go test ./sandbox/... — all green (controller / gitops / idlescaler / newapi / sandboxapi). - helm template platform/sandbox/chart --set enabled=true ... — clean render, 16 SANDBOX_MCP_* env vars emitted on the controller Deployment. Hard rules honoured: - READ-ONLY against existing cluster (no kubectl writes). - No Secret writes — name+key references only, all `optional: true`. - emrah.baysal mailbox + Stalwart admin untouched. - Principle #12 fresh clone validation. Refs #1986 Co-authored-by: hatiyildiz <hatice.yildiz@openova.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 00:05:09 +04:00
github-actions[bot]	6fc50b2719	deploy: update catalyst images to 2050e72	2026-05-19 19:01:20 +00:00
e3mrah	2050e72c69	fix(infra): refactor L3 ExternalIP reconciler to write_files + bump CP guardrail to 32256 (Closes #1981, Refs #1979 #1941) (#1985) ... PR #1979 (TBD-A50 layer 3, merged 18:00Z 2026-05-19) added the idempotent ExternalIP reconciler as inline runcmd heredocs and bumped the rendered cloud-init guardrail from 30720 to 31744. The ~3 KiB of inline bash + systemd unit heredocs overshot the new headroom: t36 fresh-prov tofu plan FAILED with rendered control-plane cloud-init at ~32498 B vs the 31744 B guardrail (754 B over). Issue #1981. This PR repackages PR #1979 using the PR #1978 pattern that fixed the analogous #1977 / TBD-A52 incident: - Adds an `l3` subcommand to /usr/local/bin/openova-externalip-bootstrap.sh (the same write_files script that hosts `l1` + `l2`). Same reconciler logic — read /etc/openova/cp-public-ipv4, compare to Node ExternalIP, restart k3s on mismatch, log to /var/log/openova-externalip.log. - Adds two new write_files entries for the systemd .service + .timer unit files (replaces the 3× cat-heredoc runcmd block). - The runcmd L3 step collapses from 77 lines of inline heredocs to a single token: `systemctl daemon-reload && systemctl enable --now openova-extip-reconcile.timer`. - Bumps the CP cloud-init guardrail from 31744 to 32256 (Hetzner hard cap 32768 minus 512 B safety buffer), applied to both primary + secondary CP preconditions in main.tf. The +512 B headroom buys room for the next legitimate addition without re-tripping the gate. ## Behavior Behavior identical to PR #1979 — same reconciler script, same exit codes (0=ok, 2=no-file, 3=apiserver-unreachable, 4=unrecovered), same systemd .service `SuccessExitStatus=0 2 3 4`, same .timer `OnBootSec=2min / OnUnitActiveSec=5min`. Diagnostic strings trimmed (~150 B saved) but key tokens preserved (`OK`, `MISMATCH`, `RECOVERED`, `FATAL nofile`, `FATAL apiserver`, `FATAL unrec`, `#1941` reference). ## Validation (Principle #15) - `tofu validate infra/hetzner/` → Success - Templatefile() measurement harness (`/tmp/measure-cloudinit/`, same fixture PR #1978 used): - pre-fix rendered: 31865 B (over fixture 30720 by 1145 B) - post-fix rendered: 31130 B (under new 32256 guardrail with 1126 B headroom) - savings: ~735 B vs PR #1979 baseline - Production headroom (after +633 B fixture↔prod variance offset): estimated 31763 B in prod, 493 B headroom under new 32256 guardrail. - `shellcheck` on rendered bootstrap script: clean (only one pre- existing SC2034 for loop counter `i`, present before this PR). - Mock test 3-case battery (matching/missing-file/mismatch-recovers): rc=0/2/0 with expected log tokens. ## Hard rules - `Closes #1981` because acceptance is code-level (size proof + tofu validate). The functional Refs #1941 closure still depends on fresh- prov walk demonstrating timer fires + log accumulates. - READ-ONLY on cluster. No Secrets touched. No emrah.baysal email / Stalwart admin API touched. Refs #1941, #1979, #1978, #1977, #1958, #966. Co-authored-by: hatiyildiz <alierenbaysal@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-19 22:58:57 +04:00
github-actions[bot]	785948ce6d	deploy: update catalyst images to 86e0eb1	2026-05-19 18:06:36 +00:00
e3mrah	86e0eb1349	fix(catalyst-ui): cosmetic regressions — logo alloy + wizard legacy tabs + AppDetail testid alias (PR γ of 3, Refs #1976) (#1980) ... Three surgical fixes for the 11 cosmetic-guard regressions caught on CI run 26112245005 (issue #1976 / TBD-A64). 8 of 11 deferred — see TBD-A65..A71 for the architectural follow-up tickets. 1. wizard/steps/logoTone.ts:126 `alloy` tile background `#FFFFFF` → `#FD6F00` (canonical Grafana Alloy swirl colour per grafana.com/oss/alloy hero). The vendored Badge already paints a white glyph; on a white tile the mark was invisible. Cosmetic-guards `logo tiles use canonical brand surface` test now matches LOGO_SURFACE_CANON[alloy] = '#FD6F00'. 2. wizard/steps/stepComponentsCopy.ts:33-34 + StepComponents.tsx:920-941 Retired the legacy "Choose Your Stack" / "Always Included" labels (renamed to "Components" / "Foundation") and dropped `role="tablist"` + `role="tab"` on the section toggle. Matches the canonical SME marketplace single-grid pattern in core/marketplace/src/components/AppsStep.svelte. The `tab === 'choose' | 'always'` state machine stays — only the operator-visible strings + ARIA semantics changed. `stepDescription` rephrased to drop both legacy phrases. StepComponents.test.tsx updated for the new labels + `aria-pressed`. 3. sovereign/AppDetail.tsx:806-859 `data-testid="sov-app-tab-${id}"` alias exposed on every TabButton via an absolutely-positioned aria-hidden span overlay (a single DOM node can't carry two `data-testid` values, the primary `app-tab-${id}` stays on the <button> for back-compat with the AppDetail.test.tsx matrix). Unblocks the 22+ existing `sov-app-tab-*` Playwright selectors in application-pages-t-o-p, continuum-dr-section, compliance-dashboards, and rbac-membership that have been broken since the rename. Chart bump: bp-catalyst-platform 1.4.208 → 1.4.209. Bootstrap-kit pin: 13-bp-catalyst-platform.yaml 1.4.208 → 1.4.209. Refs #1976 TBD-A64. Co-authored-by: hatiyildiz <hatiyildiz@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-19 22:04:29 +04:00