CodeBeaver hosted + self-host roadmap

Status: planning (2026-09-28). Owner: to be assigned per phase.

CodeBeaver ships as one codebase in two run modes, plus an enterprise tier that reuses both:

• Self-host — the customer is the operator: their Cloudflare account, their LLM keys in their secret manager, dashboard behind their Cloudflare Access. Delivered by scripts/selfhost-init.mjs and docs/self-hosting.md (P0, done).
• Hosted SaaS — we operate one multi-tenant deployment; customers bring their own model keys (BYOK) and pay per seat.
• Enterprise hosted — a dedicated deployment provisioned for the customer by running the same self-host installer in an account we manage. The self-host installer is the enterprise provisioning tool; there is no second codebase.

Architecture decisions

1. Multi-tenancy = GitHub App installations. A hosted tenant is created when a customer installs the GitHub App and completes signup. Most KV keys, queue messages, and cache entries are already repo/installation-scoped (several are not — dashboard-stats:v1, provider-status:v1, and review-log records are global today); the tenant layer adds a tenant_id dimension on top of that keying rather than a parallel data model.
2. Single multi-tenant Worker for the hosted tier. Isolation between tenants comes from tenant-scoped data keys and per-tenant rate/concurrency caps (GitHubRateGovernor DO). When a customer needs hard isolation, we provision a dedicated deployment with the self-host installer. That is the enterprise SKU, not Workers-for-Platforms.
3. BYOK is the default; managed keys are an add-on. Customers store provider credentials (OpenRouter/OpenAI/Anthropic/Z.ai) in a per-tenant vault. Resolution order per review: tenant keys → CodeBeaver-managed pool (metered, higher tier) → fail-open neutral (existing provider-outage behavior).
4. Seats = committers. A seat is a distinct PR author with reviews in the billing month. Dashboard viewers are free. Metering reads the existing usage/stats pipeline with a tenant dimension.
5. Billing via GitHub Marketplace first. Per-unit (per-seat) pricing on the Marketplace listing handles payment, plan changes, and cancellation webhooks. Stripe is added later only for enterprise invoicing and non-GitHub buyers.
6. Auth split. Tenant dashboard: GitHub OAuth (reuse the CLI's OAuth → JWT exchange) with org-admin/member roles. Operator console: current staff dashboard behind Cloudflare Access, which becomes our internal console for the hosted business.

P0 — Self-host installer (DONE 2026-09-28)

• [x] scripts/selfhost-init.mjs: preflight (wrangler whoami), fills account_id, provisions KV (codebeaver-review-cache), Vectorize index (codebeaver-codebase), six queues (three consumers + three DLQs), comments out the [[routes]] and [[services]] blocks when no custom domain is provided, deploys Worker, collects secrets (wrangler secret put with temp-file stdin so PEM newlines survive), health-checks /health, writes apps/dashboard/.env.production, builds and deploys the dashboard, prints next steps.
• [x] docs/self-hosting.md public guide; README + AGENTS.md wiring.
• [x] Real Cloudflare IDs removed from both wrangler.toml files (REPLACE_WITH_*).

Acceptance: a new operator with an empty Cloudflare account reaches a healthy /health and a served dashboard by running one command and creating a GitHub App. Verify by running the installer against a fresh Cloudflare account.

P1 — Tenant layer (core build)

• [x] tenants store: tenant_id, display name, plan, billing state, list of GitHub App installation IDs; installation → tenant resolution at webhook entry. (Landed: src/tenants.ts, installation.created/deleted webhook handling, 22 tests in src/__tests__/tenants.test.ts authored fail-first; self-host falls back to the implicit default tenant via resolveTenantId.)
• [~] Tenant-scoped data (in progress): ReviewJob.tenantId is stamped at enqueue (src/review-dispatch.ts); review-log records carry tenantId and write a review-log-by-tenant: pointer index (same pattern as the by-repo index) with listReviewLogByTenant for tenant-scoped reads; recordReviewLog resolves the tenant best-effort from the repo when the record has none (silent skip when unmapped — self-host stays index-free). 9 tests in src/__tests__/review-log-tenants.test.ts authored fail-first. Remaining: provider-status:v1 partitioning belongs with the BYOK vault (P2) — provider-status is an operator-billing metric until per-tenant keys exist.
• [x] Dashboard-stats tenant query path (P1 slice 3a): handleApiStats(request, env, { tenantId }) reads the tenant's records via listReviewLogByTenant, caches under dashboard-stats:v1:<tenantId>, filters currentInFlight to the tenant's repos, and hides global learning data (learning + "learning" in unavailable metrics). The no-options call is byte-identical to the operator view. 6 tests in src/__tests__/stats-api-tenants.test.ts authored fail-first.
• [ ] Provider resolution reads tenant config instead of only worker env; keep env chain as the operator fallback (self-host unchanged — a single implicit tenant seeded from env).
• [~] Per-tenant review caps (P1 slice 3b): plan caps table (TENANT_PLAN_CAPS — free 100/2, team 1000/6, enterprise ∞/12); enqueueWebhookReview enforces the monthly cap (KV counter tenant-reviews:v1:<tid>:<YYYY-MM>, incremented by recordReviewLog) and the concurrent cap (fixed-cap tenant-review lane in GitHubRateGovernor, DO id tenant:<tenantId>, never ramped/halved by reports); denials log tenant_rate_exceeded with kind: "monthly"|"concurrent" and skip the enqueue. 20 tests in src/__tests__/tenant-rate-caps.test.ts authored fail-first. Remaining refinement: release the lease when the queue consumer finishes a job (currently slots free via the 30-min lease TTL — fail-open, off by at most one slot for <30 min); needs frozen tests for the retry re-admission semantics before wiring.
• [x] Signup flow skeleton: install App → webhook installation.created → pending tenant (DONE) → dashboard claim via GitHub OAuth (DONE, P1 slice 3c). POST /api/tenants/claim (src/tenant-claim.ts): the claimant's OAuth code is exchanged server-side (with skipOrgGate: true — CLI_GITHUB_ORG restricts CLI sign-in only), then the claim is verified WITH THE CLAIMANT'S OWN TOKEN — either their /user id matches the stored installation account id or their /user login matches the installation account (user-account installs; the id comparison is rename-proof, the login comparison covers records provisioned before account ids were stored) or /user/memberships/orgs/<account> shows state=active, role=admin (org installs; the codebase otherwise never distinguishes account types). Tenant existence and fallback-name checks run before the single-use OAuth code is spent; GitHub fetch failures surface as 503 so outages are not mistaken for denials. claimTenant is idempotent for the same owner (logins compared case-insensitively), rejects conflicting owners (409), suspended tenants (403), and fallback-named installations (422 — skipped when a stored account id can verify). Org membership verdicts anchor on the stored account id: the membership response's organization.id must match, so a freed login re-registered as someone else's org cannot pass. Tenant gains owner { login, claimedAt } and accountId. 48 tests in src/__tests__/tenant-claim.test.ts plus org-gate tests in src/__tests__/cli-auth-org-gate.test.ts. Known concurrency boundary (accepted until tenant state becomes a gate): tenant records have two unsynchronized KV writers — claims and provision redeliveries (which refresh name/accountId, guarded to state === "pending" for exactly this reason) — and both are last-write-wins. Each writer serializes its own snapshot; the write-time re-check inside claimTenant catches anything already visible to the reading colo, and the single-use OAuth code protects a literal double-click, but two writers finishing within one read-put window both succeed and the last put wins. Before suspension or subscription enforcement turns tenant state into a gate, serialize per-tenant mutations behind a Durable Object per the RepoMemory convention (docs/architecture.md, State in KV). Update 2026-09-29: the claim path now does this — claimTenantThroughGovernor serializes the claim read-check-write through the tenant-keyed GitHubRateGovernor Durable Object (blockConcurrencyWhile); the KV claimTenant path remains as the binding-absent fallback. Filed follow-ups on the tenant domain (pre-existing, not from the claim slice): (1) [addressed 2026-09-29: enqueueWebhookReview now returns queued/unavailable/denied and a denial no longer falls back to the inline path] tenant cap denials previously fell back to an uncapped synchronous review run — enqueueWebhookReview false → inline runPullRequestReviewInRequestPath, so monthly/concurrent caps have no teeth on the webhook path; (2) re-provisioning an uninstalled installation overwrites the suspended record wholesale (owner, plan, state reset) while the monthly counter and per-tenant history keys survive — decide new-signup vs restore before billing lands; (3) suspendTenant/resume has no operator surface, so a suspended tenant is unsuspendable only by hand-written KV; wire or remove before suspension is set by production code; (4) provisioning is unconditional, so self-hosted single-installation operators are mapped to a capped free tenant contrary to the documented "self-host stays index-free" premise — gate provisioning on hosted mode or document the caps; (5) thread job.tenantId into completion recordReviewLog calls instead of re-resolving at log time (uninstall mid-review loses the counter increment); (6) renaming the installation account after a claim freezes claim verification (claimed records never self-heal) and returns a false 403 "not an admin" to the rightful owner — repair is a manual KV edit or a new installation; recover by refreshing verification inputs at claim time from GitHub by installation id rather than by provisioning redelivery; (7) a GitHub outage on GET /user inside the OAuth exchange maps to 401 "code rejected" although the code was accepted, misattributing outage windows to caller faults in ops metrics; (8) provisioning failure on installation.created is logged and answered 200 OK, so GitHub never redelivers and no tenant exists until reinstall. Contract notes for the hosted claim page: the OAuth App needs a web callback URL (the CLI registers none — it uses an ephemeral loopback callback per run), the same redirect_uri string must appear in the authorize URL, the app registration, and the claim body (the handler 400s when it is absent), and the page needs scopes read:org for the membership check. The claim client must use its own timeout longer than the shared 15s request() default (the endpoint can legitimately take ~35s) and must never auto-retry with the same single-use code; classify errors by the response envelope, not HTTP status, since claim's 401 means "code rejected", not "session expired". Hosted UI (DONE, P1 slice 3d): /login drives the OAuth authorize redirect (/api/auth/github/start → /api/auth/dashboard-callback, HttpOnly cb_session cookie backed by a KV web session that holds the user's GitHub token server-side), and the /app overview claims pending tenants through POST /api/tenants/claim-session, which enforces the same verification rules against the session token — no second OAuth redirect.
• [ ] Evals: tenant-isolation fixtures (tenant A can never read tenant B findings/keys); contract test additions for new endpoints.

Acceptance: two tenants reviewing concurrently share one deployment with isolated state, isolated rate limits, and no cross-tenant reads in tests.

P2 — BYOK vault + tenant dashboard

• [ ] tenant_providers store with envelope encryption: per-tenant DEK, master key in Worker secret, ciphertext in D1; masked display (sk-…abcd); keys decrypted only inside the provider call path.
• [ ] Key onboarding wizard (provider choice, key paste, model + timeout overrides, test call), key rotation + revocation, per-request audit row (key used, repo, tokens, estimated cost).
• [~] Tenant dashboard pages: overview + repos landed (P2 slice 3a: /login, /app workspace overview with claim CTA + monthly usage, /app/repos with per-repo auto-review toggles gated on installation-admin; GitHub sign-in per the auth split above). Remaining: usage + spend, seats, keys, settings; tenant role gates beyond the per-endpoint admin checks.
• [ ] CLI: session JWT carries tenant; review endpoints resolve tenant keys.
• [ ] Secrets never logged: extend logger redaction tests to the vault path.

Acceptance: a tenant can add, rotate, and delete an OpenRouter key without operator involvement; no plaintext key appears in logs, KV, or D1 (testable); audit row exists for every model call.

P3 — Monetization

• [~] Marketplace listing (free + per-seat paid plans); Marketplace purchase/cancel/plan-change webhooks update tenant billing state. Landed: marketplace_purchase webhook handling records marketplace-entitlement:v1:{login} and applies plan changes / cancellation suspension / payment resume to the matching tenant (matched by installation account or claim owner login — purchase events carry no installation). The public listing itself is an operator step; plan mapping comes from MARKETPLACE_PLAN_IDS.
• [~] Seat metering job: distinct committers per tenant per month; seat count sync to Marketplace. Landed: nightly 30 2 * * * cron meters tenant-seats:v1:{tenantId}:{month} over the tenant review index (bots excluded); counts surface in /api/tenant/workspaces and the dashboard usage page. Marketplace sync waits on the listing.
• [x] Enforcement: reuse tenant pause/resume state for subscription_required — reviews stop, bot posts one friendly comment (30-day sub-notice:v1:{tenantId} flag), resumable on payment; installation.suspend/unsuspend webhooks honor the same state and a cancelled entitlement keeps the tenant suspended across reinstalls. Operator suspend/resume surface: staff-only POST /api/tenants/state (closes follow-up 3).
• [~] Plan gating: Free = X reviews/month (monthly cap) and 1 repo/month (PLAN_REPO_LIMITS, active-repos set written by recordReviewLog, enforced at PR-review entry with a posted skip reason); Team = per-seat BYOK. Still open: gating learning rules / Jev / Vectorize search by plan where sensible.

Acceptance: cancel → suspension → reviews stop; re-subscribe resumes without data loss; seat counts match GitHub records.

P4 — Enterprise pack

• [ ] SAML SSO for the tenant dashboard (Cloudflare Access or WorkOS).
• [ ] Audit-log export, data-retention configuration (diff/context retention windows), region/residency notes for Workers.
• [ ] Dedicated-tenant provisioning: run scripts/selfhost-init.mjs in a Cloudflare account we manage; per-customer wrangler config; handover runbook.
• [ ] SLA + status page for the hosted tier; SOC2 controls roadmap.

Non-goals

• No port off Cloudflare (DOs/Queues/Vectorize/Workers AI are load-bearing). Self-host is "your Cloudflare account", which is also the enterprise data isolation story.
• No Stripe on day one.
• No managed-keys default; BYOK-only at launch keeps us out of the token-reselling compliance surface.

Risks

• Key vault trust is the product's biggest liability surface: P2 lands behind a security review pass (fail-closed, redaction tests) before the hosted tier accepts real customers.
• Marketplace seat semantics differ from raw committer counts (bot authors, dependabot); the metering job must exclude bot/automation authors.
• Self-host support burden: publish a supported-version policy (we support the latest release, security fixes only for older ones).
• Review queue fairness: one noisy tenant must not starve others; the per-tenant caps in P1 are the guardrail and need load tests before P3.