Infrastructure deployment guide for single tenants

Modified on Fri, 21 Aug at 9:30 AM

This guide provides the deployment procedure for installing the Cubyts platform in a single-tenant Kubernetes environment using kubectl and the provided legacy_k8s/v1 manifests. It covers the deployment prerequisites, environment and hostname configuration, Kubernetes resources, secrets and ConfigMaps, Vault and database initialization, platform exposure, and post-deployment verification. The guide assumes that the underlying infrastructure including the Kubernetes cluster, networking, DNS, certificates, MongoDB Atlas, model endpoints, email provider, and required access permissions has already been provisioned and validated.

Reference for prerequisites: Infrastructure Prerequisites

Repository layout

All paths below are relative to the root of the deployment repository.

deployment/

├── envs.config.example              template for the values you fill in

├── envs.config                      YOU CREATE THIS — copy of the above, filled in

├── update_envs.sh                   substitutes envs.config into every .env file

├── update_mongodb_uri.sh            imports the MongoDB seed collections

├── cubyts-v3/                       per-service .env.docker files  -> ConfigMaps

│   ├── gcp-key.json                 -> Secret gcp-key

│   ├── agents/.env.docker           -> Secret agents-env

│   ├── byts/api/.env.docker         -> ConfigMap byts-api-env

│   ├── v3/api/.env.docker           -> ConfigMap v3-api-env

│   ├── induct/api/.env.docker       -> ConfigMap induct-api-env

│   ├── ting/api/.env.docker         -> ConfigMap ting-api-env

│   ├── cubyts-dr/.env.docker        -> ConfigMap dr-api-env

│   ├── cubyts-dr/.env.loader.docker -> ConfigMap loader-api-env

│   ├── cubyts-dr/.env.mcp.docker    -> ConfigMap mcp-api-env

│   └── 1nt-e/<svc>/api/.env.docker  -> ConfigMaps axios-env, slack-api-env,

│                                       jira-api-env, common-api-env,

│                                       microsoft-api-env

├── envs/cubyts/

│   ├── ast/config/.env              -> ConfigMap ast-engine-config

│   ├── ast/java/config/.env         -> ConfigMap ast-java-config

│   ├── ast/jwt/ec_public_key.pem    -> Secret ast-jwt-pubkey

│   ├── tools/ide/config/.env        -> ConfigMap tools-ide-env

│   └── mosquitto/jwt/jwt-*.key      -> Secret mqtt-jwt

├── jira/jira_privatekey.pcks8       -> Secret jira-key

├── database/

│   ├── restore-k8s.sh               PostgreSQL restore

│   ├── db_backups.zip               PostgreSQL seed dumps

│   └── mongodb_backup/*.json        MongoDB seed collections

└── legacy_k8s/

    ├── update-configmaps.sh         creates every ConfigMap

    ├── create-app-secrets.sh        creates the file-based Secrets

    ├── vault-bootstrap-k8s.sh       initialises and unseals Vault

    ├── vault-check-secrets.sh       verifies Vault

    ├── nginx/nginx.conf             -> ConfigMap nginx-config

    ├── nginx/proxy_params           -> ConfigMap nginx-config

    ├── certs/*.crt                  -> ConfigMap ca-certs

    ├── ui-config/v3-config.js       -> ConfigMap v3-ui-config

    ├── ui-config/induct-config.js   -> ConfigMap induct-ui-config

    └── v1/*.yaml                    the manifests you apply

Before you start

•  Access to the cluster — kubectl auth can-i create deployment,secret,statefulset -n cubyts must return yes

•  Tooling: kubectl 1.28+, vault 1.13+, jqunzippython3opensslmongoimport, and your cloud CLI

•  The values from the completed prerequisite configuration: MongoDB URI, model keys and endpoints, SMTP settings

•  The container image tags for the release you are deploying


1) Fill in envs.config

This is the single place you enter credentials. update_envs.sh reads it and replaces the matching __<KEY>_PLACEHOLDER__ token in every .env file across the repository.

cp envs.config.example envs.config

$EDITOR envs.config

chmod 600 envs.config

Every key below must be non-empty — the script refuses to run otherwise.

Key

Value

MONGODB_URI

Full connection string. Must start with mongodb:// or mongodb+srv://

AZURE_GPT_API_KEY

Shared across all AZURE_GPT_* deployments

AZURE_GPT_MODEL_DOMAIN

Resource endpoint, e.g. https://<resource>.openai.azure.com

AZURE_GPT_LARGE_MODEL_DEPLOYMENT_NAME

Large / reasoning model deployment name

AZURE_GPT_LARGE_MODEL_API_VERSION

Its API version

AZURE_GPT_SMALL_MODEL_DEPLOYMENT_NAME

Small model deployment name

AZURE_GPT_SMALL_MODEL_API_VERSION

Its API version

GPT_TEXT_EMBEDDER_3_API_KEY

Embedding model key

GPT_TEXT_EMBEDDER_3_DEPLOYMENT_NAME

Default text-embedding-3-small

GPT_TEXT_EMBEDDER_3_API_VERSION

Default 2025-01-01-preview

GPT_TEXT_EMBEDDER_3_DOMAIN

Embedding endpoint

VOYAGE_API_KEY

Set a non-empty dummy value if no flow in scope uses Voyage


Then run it:

bash update_envs.sh

The script reports how many files it changed per key, and fails loudly if any placeholder is left unreplaced — do not continue past an error. To target a different tree, set CONFIG_FILE or SEARCH_ROOT.

envs.config holds live credentials. It is git-ignored. Keep it out of shared drives and delete your working copy after the deployment.

2) Set the hostnames

envs.config does not cover hostnames. Change these by hand.

grep -rnE 'cubyts\.com|localhost|host\.docker\.internal|REDIRECT|CORS_ORIGIN' \

  cubyts-v3/*/api/.env.docker cubyts-v3/1nt-e/*/api/.env.docker cubyts-v3/cubyts-dr/.env*.docker

File

Change

cubyts-v3/**/.env.docker

CUBYTS_BASEURL*_HOST_NAME*_API_URLCUBYTS_MCP_PUBLIC_URLCORS_ORIGIN_FOR_*, and the Google / Jira / Microsoft redirect URIs

legacy_k8s/nginx/nginx.conf

server_name on line 81 and line 234

legacy_k8s/ui-config/v3-config.js

API_HOSTINDUCT_HOSTPO_HOSTDR_HOSTSLACK_HOSTJIRA_HOSTCOMMON_HOSTONEDRIVE_HOSTLOADER_HOSTINDUCT_UI_URLLOCALHOST_URL

legacy_k8s/ui-config/induct-config.js

REACT_APP_API_HOSTREACT_APP_BYTS_API_HOSTREACT_APP_COMMON_API_HOSTREACT_APP_BYTS_UI_HOST


Register each redirect URI you set in the matching Google, Jira, Microsoft and Slack application. The reCAPTCHA keys in byts-api/.env.docker and the UI configs are Cubyts development keys and will fail on your domains — replace them with your own.

3) Update the manifests

The manifests in legacy_k8s/v1/ carry literal values, not placeholders — update_envs.sh does not touch them. Edit these before applying.

File

Change

v1/ingress.yaml

Hostnames (app-cubyts.<domain>.comid-cubyts.<domain>.com), the reserved address name cubyts-gateway-ip, and the certificate name cubyts-cert

v1/postgres.yaml

POSTGRES_PASSWORD ships as postgres in plaintext. Change it, and use the same value as PG_PASS in step 11

v1/postgres.yaml

storageClassName: standard — set to a class that exists in your cluster

All API / UI / AST manifests

Image registry and tag


4) Connect to the cluster

gcloud container clusters get-credentials <CLUSTER> --region=<REGION> --project=<PROJECT>

kubectl cluster-info

kubectl get storageclass

Confirm the egress address and that it is allow-listed in MongoDB Atlas:

kubectl create job egress-test --image=curlimages/curl -- curl -s https://ifconfig.me

kubectl wait --for=condition=complete job/egress-test --timeout=60s

kubectl logs job/egress-test

kubectl delete job egress-test

5) Grant image access

Give the cluster's node service account read access to the registry:

NODE_SA=$(gcloud container clusters describe <CLUSTER> --region=<REGION> \

  --project=<PROJECT> --format='value(nodeConfig.serviceAccount)')

 

gcloud artifacts repositories add-iam-policy-binding <REPO> \

  --location=<LOCATION> --project=<REGISTRY_PROJECT> \

  --member="serviceAccount:${NODE_SA}" --role="roles/artifactregistry.reader"

6) Create the namespace

kubectl apply -f legacy_k8s/v1/namespace.yaml

7) Create the ConfigMaps

NS=app bash legacy_k8s/update-configmaps.sh

Idempotent — it uses --dry-run=client | kubectl apply, so it is safe to re-run. It creates:

byts-api-envv3-api-envinduct-api-envdr-api-envloader-api-envmcp-api-envaxios-envslack-api-envjira-api-envting-api-envcommon-api-envmicrosoft-api-envast-engine-configast-java-configtools-ide-envnginx-configca-certsv3-ui-configinduct-ui-config.

Comments, blank lines and surrounding quotes are stripped from each .env source as it is converted.

kubectl get cm -n cubyts

8) Create the Secrets

NS=cubyts bash legacy_k8s/create-app-secrets.sh

Creates gcp-keyjira-keymqtt-jwtast-jwt-pubkey and agents-env from the repository files listed in the layout above. The script fails if any source file is missing.

mosquitto-config is not created by the script and its source file is not in the repository. Obtain mosquitto.conf from Cubyts, then:

kubectl create secret generic mosquitto-config \

  --from-file=mosquitto.config=/path/to/mosquitto.conf -n cubyts

The key name must be mosquitto.config — that is the subPath the Mosquitto pod mounts.

postgres-secret is created by v1/postgres.yamlvault-approle-credentials is created in step 10. Do not create either by hand.

kubectl get secret -n cubyts

9) Apply the manifests

kubectl apply -n cubyts -f legacy_k8s/v1/

kubectl get pods -n cubyts --watch

kubectl apply -f <dir> is not recursive, so v1/postgres/ is deliberately skipped — that directory is an older duplicate and must not be applied.

Before applying, remove or skip what does not apply to your cluster:

Manifest

When to skip

sc-gp3.yaml

Everywhere except AWS / EKS — it is an EBS StorageClass

vpa.yaml

Unless the VerticalPodAutoscaler CRD is installed

datadog.yamldatadog-agent.yaml

Unless Datadog is in scope; both need an API key

loader-api-apollo11.yaml

Always — it is the apollo11 variant of loader-api.yamlApplying both creates two conflicting Deployments

ingress.yaml

On this pass. Apply it in step 13, once the backend is healthy


API pods restart until Vault is bootstrapped. That is expected at this stage.

10) Bootstrap Vault

kubectl get pod -n cubyts -l app=vault

NAMESPACE=app bash legacy_k8s/vault-bootstrap-k8s.sh

kubectl rollout restart deployment -n cubyts

The script port-forwards Vault, initialises it (3 key shares, threshold 2), unseals it, enables KV v2 at cubyts/, enables AppRole auth, and writes vault-approle-credentials and vault-init-credentials.

Save the unseal keys and root token the script prints. They are not recoverable. Vault re-seals on every pod restart and must be unsealed again with kubectl exec -n cubyts vault-0 -- vault operator unseal <KEY>.

bash legacy_k8s/vault-check-secrets.sh

11) Restore PostgreSQL — one time only

Destructive. The script drops and recreates the cubyts and users databases. Never re-run it against a populated system.

kubectl get pod -n cubyts postgres-0

PG_NS=app PG_PASS='<POSTGRES_PASSWORD>' bash database/restore-k8s.sh

PG_PASS must match what you set in v1/postgres.yaml in step 3.

kubectl exec -n cubyts postgres-0 -- psql -U postgres -c '\l' | grep -E 'cubyts|users'

12) Seed MongoDB — one time only

Destructive. Each target collection is dropped before import.

$EDITOR update_mongodb_uri.sh #update the mongodb uri inside


bash update_mongodb_uri.sh

update_mongodb_uri.sh loops over the whole directory. Set SKIP_URI_UPDATE=1 to run only the import, since step 1 already substituted the URI.

13) Expose the platform

The reserved address come from the prerequisite guide. 

Update the ingress.yaml with the IP Address name reserved for the gateway.

Apply the Gateway, then point DNS at the address it reports:

kubectl apply -n cubyts -f legacy_k8s/v1/ingress.yaml

kubectl get gateway cubyts-gateway -n cubyts --watch


Non-GKE clusters: skip ingress.yaml. The nginx Service is a NodePort — put your own ingress controller or Application Gateway in front of it, terminate TLS there, route both hostnames to it and preserve the Host header.

14) Verify

kubectl get pods -n cubyts

kubectl get deploy,sts,svc,hpa,gateway,httproute -n cubyts

kubectl exec -n cubyts vault-0 -- vault status | grep Sealed     # false

 

curl -s https://<APP_HOST>/healthz                            # ok

curl -s https://<IDENTITY_HOST>/healthz                       # ok

Every pod must be Running with a stable restart count. Then sign in through the UI and confirm that a model-backed feature, an outbound email and each integration in scope all work.


What gets deployed

Namespace cubyts. Ports are the container ports in the manifests.

Manifest

Workload

Port

byts-api.yaml

Deployment + Service

5100

v3-api.yaml

Deployment + Service

5010

induct-api.yaml

Deployment + Service

5002

dr-api.yaml

Deployment + Service

3032

loader-api.yaml

Deployment + Service

3030

mcp-api.yaml

Deployment + Service

3031

axioshelper-api.yaml

Deployment + Service

3029

slack-api.yaml

Deployment + Service

5003

jira-api.yaml

Deployment + Service

6010

ting-api.yaml

Deployment + Service

5007

common-api.yaml

Deployment + Service

5005

microsoft-api.yaml

Deployment + Service

5130

ast-engine.yaml

Deployment + Service

8040

ast-java.yaml

Deployment + Service

8030

ast-dotnet.yaml

Deployment + Service

8080

ast-python.yaml

Deployment + Service

8080

v3-ui.yaml

Deployment + Service

8080

induct-ui.yaml

Deployment + Service

80

nginx.yaml

Deployment + NodePort Service

80

vault.yaml

StatefulSet + Service + PVC

8200

postgres.yaml

StatefulSet + Service + Secret + PVC

5432

mosquitto.yaml

StatefulSet + Service + PVC

1883

tools-ide.yaml

Deployment + Service

8280, 4883

observability.yaml

Loki, Grafana, Promtail

3100, 3000

hpa.yaml

18 HorizontalPodAutoscalers

vpa.yaml

5 VerticalPodAutoscalers

ingress.yaml

Gateway, 3 HTTPRoutes, HealthCheckPolicy

80, 443


MongoDB is external and is never created by these manifests.



Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons

Feedback sent

We appreciate your effort and will try to fix the article