02 — Tenant Provisioning & Orchestration

← Index

How a new tenant comes into existence, who decides its deployment shape, and how its compute and database are brought up. This is a control-plane flow (core-api) that delegates database creation to the data plane (tenant-api).

Entry point: createTenant

TenantProvisioningService.createTenant() — src/core-api/src/services/tenant/TenantProvisioningService.ts:84.

The flow, in order:

1. Validate the request (validateTenantRequest).
2. Create the tenant record (createTenantRecord, :443) in the core DB. has_dedicated_database defaults to true.
3. Add the primary domain if provided (domainService.addCustomDomain).
4. Decide the deployment strategy — deploymentStrategy.determineDeploymentType(characteristics) (:125). The characteristics fed in (:116-123) are: planType, complianceRequired, memberCount, monthlyApiCalls, and region. → returns 'shared' or 'container_per_tenant'.
5. Provision the deployment (provisionDeployment, :484) — for container_per_tenant, create the dedicated compute (below).
6. Record deployment info (updateTenantDeploymentInfo, :525): set deployment_strategy, has_dedicated_database, and write a TenantDeployment row.
7. Generate + encrypt + store the license (generateAndEncryptLicense → storeLicense, :134-135). See 04.
8. Create the tenant database (createTenantDatabase, :1108) — but only when has_dedicated_database is true; shared tenants reuse the region DB. This step delegates to the tenant-api (below).
9. Create the bootstrap admin user (createBootstrapUser).
10. Notify the tenant-api to load the license into cache (fire-and-forget).

The deployment decision: shared vs container-per-tenant

determineDeploymentType maps tenant characteristics to one of two deployment types:

The two drivers that push a tenant to container_per_tenant are noisy-neighbour / scale and compliance / data residency.

Provisioning dedicated compute

provisionDeployment(tenantId, deploymentType, request) — :484.

• For shared: returns immediately, no compute to create (:485-488).
• For container_per_tenant (:490): builds a container config and calls orchestrationService.createTenantContainer(config) (:509). The config (:494-507):

  image:        ip4x/tenant-api:latest
  environment:  TENANT_ID        = <uuid>          # pins DB init + tenant scope
                TENANT_DB_NAME   = tenant_<uuid>   # the dedicated DB
                STORAGE_PREFIX   = <uuid>/
                NODE_ENV         = ...
  resources:    memory 512m, cpu 0.5               # per-tenant resource cap

It returns { containerId, endpoint }.

⚠️ Known gap. The container env above sets TENANT_ID, TENANT_DB_NAME, STORAGE_PREFIX, NODE_ENV but does not set REDIS_PREFIX. All instances share one Redis; BullMQ keys queues by name, so two container_per_tenant instances without distinct REDIS_PREFIX values would bind the same queue names and steal each other's jobs. Setting REDIS_PREFIX: tenantId in this config closes it. See 03 — queue isolation.

The orchestrator

createTenantContainer is implemented by KubernetesOrchestrationService — src/core-api/src/services/deployment/KubernetesOrchestrationService.ts. It uses @kubernetes/client-node and creates namespaced Kubernetes objects per tenant: a Deployment (the image + env + resource limits), a Service, and an Ingress (via NetworkingV1Api). It also exposes updateTenantContainer / read operations for the deployment lifecycle.

This means the per-tenant-instance model is not greenfield — it is implemented in skeleton. The remaining work to operate it at scale is (a) the REDIS_PREFIX gap above, (b) wiring per-region database hosts (below), and (c) confirming the Ingress/TLS/namespace/secret path end to end.

Database creation is delegated to the data plane

createTenantDatabase(tenantId) — :1108 — does not run CREATE DATABASE from core-api. Instead it:

1. Resolves the tenant's API endpoint (getTenantApiEndpoint → the region's tenant_api_url, or the dedicated container's endpoint).
2. Makes an internal HTTP call: POST {tenantApiEndpoint}/internal/database/initialize with header x-internal-secret and a 10-minute timeout (:1122-1134).
3. The tenant-api then runs CREATE DATABASE tenant_<uuid> and all migrations itself, via its init.ts initialize() (the same migrator behind npm run db:migrate:prod).

This is the crux: each tenant-api instance owns its own schema. A newly provisioned dedicated instance boots, is told to initialize, and brings up its own database. There is no central migration fan-out — the data plane is self-sufficient. See 03 for the migration mechanics.

Regions as the residency unit

A Region (src/core-api/src/database/schema.ts, region_key e.g. za-north, eu-west-1) carries a tenant_api_url (the regional data-plane endpoint) and a shared_database_name. A tenant's region_id decides which regional tenant-api serves it and where its data lives. For full data residency, a dedicated tenant's database should be created on the in-region Postgres and its instance run in that region's cluster — the per-region database host is noted in the schema as a future field (db_connection_string ... per region) and is the main residency seam still to wire.

Sequence summary

core-api  createTenant
  ├─ create tenant record (has_dedicated_database=true default)
  ├─ determineDeploymentType  →  shared | container_per_tenant
  ├─ provisionDeployment
  │     └─ (container_per_tenant) KubernetesOrchestrationService.createTenantContainer
  │            → Deployment + Service + Ingress, env TENANT_ID/TENANT_DB_NAME/...
  ├─ generate + encrypt + store license  (→ Tenant.encrypted_license)
  ├─ createTenantDatabase
  │     └─ HTTP POST {tenantApiEndpoint}/internal/database/initialize
  │            → tenant-api runs CREATE DATABASE tenant_<uuid> + migrations  (DATA PLANE)
  ├─ createBootstrapUser
  └─ notify tenant-api to cache license

Key source paths: src/core-api/src/services/tenant/TenantProvisioningService.ts, src/core-api/src/services/deployment/KubernetesOrchestrationService.ts, src/core-api/src/database/schema.ts (Region, Tenant, TenantDeployment), src/tenant-api/src/database/init.ts.

Next: 03 — Database topology & data isolation →