Following up on our previous post where we automated a production-ready Kubernetes cluster, this guide walks through deploying a complete OpenStack cloud on top of it using OpenStack-Helm ↗ (2026.2) and a GitOps workflow managed by FluxCD ↗.
Why OpenStack?#
OpenStack is the world’s leading open-source cloud operating system, turning bare-metal servers into a full AWS-style private cloud with multi-tenant Virtual Machines (Nova), Software-Defined Networking (Neutron/OVN), Ceph block storage (Cinder/Glance), Load Balancers (Octavia), DNS (Designate), hardware accelerators (Cyborg), and managed Kubernetes (Magnum).
Running OpenStack gives organizations complete data sovereignty, zero vendor lock-in, and predictable infrastructure costs compared to hyperscalers or proprietary virtualization stacks—and deploying its control plane on Kubernetes via OpenStack-Helm + FluxCD brings self-healing pods, rolling upgrades, and declarative GitOps lifecycle management to the entire cloud.
Architecture & Network Topology#
FluxCD continuously reconciles the Kubernetes cluster with the declarative configuration stored in Git. Upstream OpenStack-Helm charts are pulled dynamically by tag (${OPENSTACK_RELEASE}.${OPENSTACK_CHART_PATCH}), while all customizations are applied through Helm value overrides.
All public endpoints are exposed through the Envoy Gateway VIP (192.168.64.100) at https://<service>.192.168.64.100.nip.io.
Node Network Layout#
The underlying Kubernetes nodes use the 4-NIC topology provisioned in the Kubernetes cluster post:
| Interface | Subnet | OpenStack & Cluster Role |
|---|---|---|
ens160 | 192.168.64.0/24 | Management, Kubernetes API, Calico VXLAN, and MetalLB VIP (192.168.64.100) |
ens192 | 10.0.0.0/24 | Internal node-to-node traffic and tenant overlay transport |
ens224 | L2 only (no IP) | Bridged into OVS/OVN br-ex for OpenStack external flat (provider) networks and FIPs |
ens256 | L2 only (no IP) | Worker nodes only — Octavia Load Balancer management network |
Key Design Decisions#
- Helm Hooks & Job Protection: OpenStack-Helm drops
helm3_hook, turning most bootstrap tasks into regular KubernetesJobs ordered bykubernetes-entrypointinit containers (dependencies.static.<component>.jobs). Because a few charts still use hard-codedpost-install,post-upgradehooks, a global Kustomize patch setsspec.install.disableWait: trueandspec.upgrade.disableWait: trueon allHelmReleases to avoid deadlocks, and applieshelm.sh/resource-policy: keepso completed Jobs are never deleted during Flux reconciliation. - Unified Gateway &
openstack-caPKI: TheGateway(gateway-defaultinenvoy-gateway-system) and the initialopenstack-cahierarchy are bootstrapped by Ansible. FluxCD adopts theopenstack-caresources seamlessly, issuesopenstack-gateway-tlsin theopenstacknamespace (so pods like Magnum and Heat trust the public HTTPS endpoints via one shared CA), and manages all OpenStackHTTPRoutes. - Single-Source Cluster Variables & Multi-Release Overrides:
clusters/prod/region1/flux-system/cluster-vars.yamldefinesGATEWAY_IP(192.168.64.100),PUBLIC_DOMAIN(192.168.64.100.nip.io),OPENSTACK_RELEASE(2026.1), andOPENSTACK_CHART_PATCH(0) once, which Flux injects across all manifests viapostBuild.substituteFrom. Release-specific chart schema differences between2026.1and2026.2live cleanly invalues-release/<release>/.
Repository Structure#
clusters/
└── prod/
├── flux-system/flux-system/ # Output of `flux bootstrap`
└── region1/
├── flux-system/ # Applied with `kubectl apply`
│ ├── cluster-vars.yaml # GATEWAY_IP, PUBLIC_DOMAIN, OPENSTACK_RELEASE
│ ├── gitrepository-openstack-helm.yaml
│ └── openstack-{sources,pki,kustomization}.yaml
├── sources/
│ └── openstack-helm-charts.yaml
└── openstack/
├── kustomization.yaml
├── patch-helmrelease-openstack-gitops.yaml
├── patch-helmrelease-timeouts.yaml
├── pki/ # selfsigned-bootstrap -> openstack-ca + gateway certs
├── releases/ # HelmRelease manifests (one per OpenStack component)
├── values/ # Common Helm value overrides + SOPS credentials.yaml
├── values-release/ # Per-release overrides (2026.1 / 2026.2)
└── gateway/
└── httproutes.yaml # HTTPS HTTPRoutes attached to gateway-default
scripts/
└── release-upgrade.sh # Automated release-by-release upgrade helper
.sops.yaml # SOPS age public key configurationtextDetailed Setup & Deployment Walkthrough#
Prerequisites#
1. Kubernetes Cluster Infrastructure#
Ensure your Kubernetes cluster (deployed via openstack-k8s-cluster ↗) has all add-ons running and 300 GiB of raw Ceph storage available across the 3 worker nodes:
| Component | Purpose |
|---|---|
| MetalLB | Assigns VIP 192.168.64.100 to Envoy Gateway |
| Envoy Gateway | Terminates HTTPS on :443 via gateway-default |
| cert-manager | Manages openstack-ca and per-service mTLS certificates |
| Rook-Ceph | Provides RBD block storage (general StorageClass) |
| FluxCD | Reconciles GitRepositories, Kustomizations, and HelmReleases |
Verify Rook-Ceph health before starting:
kubectl -n rook-ceph exec -it deploy/rook-ceph-tools -- ceph statusbash2. Admin Workstation Tools#
Install helm, the flux CLI, age, and sops on the host where you manage the cluster:
# Install Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
# Install Flux CLI
curl -s https://fluxcd.io/install.sh | sudo bash
# Install age and SOPS (arm64 example; use linux.amd64 on x86_64)
sudo apt update && sudo apt install -y age
curl -LO https://github.com/getsops/sops/releases/download/v3.13.3/sops-v3.13.3.linux.arm64
sudo install sops-v3.13.3.linux.arm64 /usr/local/bin/sopsbashStep 1: Node Preparation#
OpenStack services require higher inotify limits on worker nodes, role labels so OpenStack-Helm pods schedule onto the right nodes, the openstack namespace, and endpoint_pod_names enabled in CoreDNS (already configured if you used the openstack-k8s-cluster playbook):
# 1. Increase inotify limits on all worker nodes
for NODE in openstack-worker1 openstack-worker2 openstack-worker3; do
ssh root@$NODE "sysctl -w fs.inotify.max_user_watches=524288 && \
sysctl -w fs.inotify.max_user_instances=1024"
done
# 2. Label worker nodes for OpenStack control-plane, compute, and network roles
for NODE in openstack-worker1 openstack-worker2 openstack-worker3; do
kubectl label --overwrite nodes $NODE \
openstack-control-plane=enabled \
openstack-compute-node=enabled \
openvswitch=enabled \
l3-agent=enabled \
openstack-network-node=enabled
done
# 3. Create the target namespace
kubectl create namespace openstack
# 4. Ensure endpoint_pod_names is enabled in CoreDNS (required by OVN RAFT StatefulSets)
if ! kubectl get configmap coredns -n kube-system -o yaml | grep -q endpoint_pod_names; then
kubectl get configmap coredns -n kube-system -o yaml | \
sed '/pods insecure/a \ endpoint_pod_names' | \
kubectl apply -f -
kubectl rollout restart deployment/coredns -n kube-system
kubectl rollout status deployment/coredns -n kube-system
fibashStep 2: SOPS Credential Encryption#
All OpenStack service, database, messaging, and registry passwords are stored in clusters/prod/region1/openstack/values/credentials.yaml and encrypted with SOPS + age. Never commit this file unencrypted.
1. Generate an age Keypair and Create the Cluster Secret#
unset SOPS_AGE_KEY
age-keygen -o age.agekey
export SOPS_AGE_KEY_FILE=$(pwd)/age.agekey
# Copy the printed public key (age1...) into .sops.yaml
# Store the private key in the flux-system namespace so FluxCD can decrypt secrets
kubectl create namespace flux-system --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic sops-age \
--namespace=flux-system \
--from-file=age.agekey=./age.agekey \
--dry-run=client -o yaml | kubectl apply -f -bash2. Configure .sops.yaml#
Ensure .sops.yaml at the root of the repository references your age public key:
creation_rules:
- path_regex: clusters/prod/region1/openstack/values/credentials\.yaml
encrypted_regex: '^(data|stringData)$'
age: >-
age19u528x0n9cxvlrxs4rxu6ws0yhvev3ucgv5tcpnp4vr5nrqaq4uqxhmkulyaml3. Populate and Encrypt credentials.yaml#
Populate clusters/prod/region1/openstack/values/credentials.yaml using the structure below (replace Password1 with strong passwords):
# ============================================================
# SOPS ENCRYPTION REQUIRED BEFORE COMMITTING TO GIT
#
# Steps:
# 1. Fill in all Password1
# 2. Run: sops --encrypt --in-place clusters/prod/openstack/values/credentials.yaml
# 3. Only then: git add clusters/prod/openstack/values/credentials.yaml && git commit
#
# To edit after encryption:
# sops clusters/prod/openstack/values/credentials.yaml
# ============================================================
apiVersion: v1
kind: Secret
metadata:
name: openstack-credentials
namespace: openstack
stringData:
values.yaml: |
endpoints:
identity:
auth:
admin:
password: Password1
cinder:
password: Password1
glance:
password: Password1
nova:
password: Password1
swift:
password: Password1
service:
password: Password1
test:
password: Password1
keystone:
password: Password1
neutron:
password: Password1
barbican:
password: Password1
placement:
password: Password1
heat:
password: Password1
heat_trustee:
password: Password1
heat_stack_user:
password: Password1
magnum:
password: Password1
magnum_stack_user:
password: Password1
mariadb-server:
password: Password1
octavia:
password: Password1
designate:
password: Password1
ironic:
password: Password1
skyline:
password: Password1
user:
password: Password1
cyborg:
password: Password1
oslo_db:
auth:
admin:
username: root
password: Password1
keystone:
username: keystone
password: Password1
nova:
username: nova
password: Password1
nova_api:
username: nova_api
password: Password1
nova_cell0:
username: nova_cell0
password: Password1
neutron:
username: neutron
password: Password1
cinder:
username: cinder
password: Password1
glance:
username: glance
password: Password1
placement:
username: placement
password: Password1
heat:
username: heat
password: Password1
barbican:
username: barbican
password: Password1
magnum:
username: magnum
password: Password1
octavia:
username: octavia
password: Password1
designate:
username: designate
password: Password1
powerdns:
username: powerdns
password: Password1
sst:
username: sst
password: Password1
audit:
username: audit
password: Password1
exporter:
username: exporter
password: Password1
skyline:
username: skyline
password: Password1
cyborg:
username: cyborg
password: Password1
oslo_db_persistence:
auth:
admin:
username: root
password: Password1
octavia:
username: octavia
password: Password1
oslo_db_api:
auth:
admin:
username: root
password: Password1
nova:
username: nova
password: Password1
oslo_db_cell0:
auth:
admin:
username: root
password: Password1
nova:
username: nova
password: Password1
ceph_object_store:
auth:
glance:
username: glance
password: Password1
tempurlkey: supersecret
oslo_messaging:
auth:
admin:
username: rabbitmq
password: Password1
user:
username: rabbitmq
password: Password1
guest:
password: Password1
keystone:
username: keystone
password: Password1
nova:
username: nova
password: Password1
neutron:
username: neutron
password: Password1
cinder:
username: cinder
password: Password1
glance:
username: glance
password: Password1
heat:
username: heat
password: Password1
barbican:
password: Password1
magnum:
username: magnum
password: Password1
octavia:
username: octavia
password: Password1
designate:
username: designate
password: Password1
cyborg:
username: cyborg
password: Password1
powerdns:
auth:
admin:
password: Password1
oci_image_registry:
auth:
barbican:
username: barbican
password: Password1
cinder:
username: cinder
password: Password1
designate:
username: designate
password: Password1
etcd:
username: etcd
password: Password1
glance:
username: glance
password: Password1
heat:
username: heat
password: Password1
keystone:
username: keystone
password: Password1
libvirt:
username: libvirt
password: Password1
magnum:
username: magnum
password: Password1
mariadb:
username: mariadb
password: Password1
memcached:
username: memcached
password: Password1
neutron:
username: neutron
password: Password1
nova:
username: nova
password: Password1
octavia:
username: octavia
password: Password1
openvswitch:
username: openvswitch
password: Password1
ovn:
username: ovn
password: Password1
placement:
username: placement
password: Password1
powerdns:
username: powerdns
password: Password1
prometheus:
username: prometheus
password: Password1
prometheus-mysql-exporter:
username: prometheus-mysql-exporter
password: Password1
prometheus-openstack-exporter:
username: prometheus-openstack-exporter
password: Password1
rabbitmq:
username: rabbitmq
password: Password1
redis:
username: redis
password: Password1
skyline:
username: skyline
password: Password1
cyborg:
username: magnum
password: Password1
oslo_db_cell1:
auth:
admin:
username: root
password: Password1
nova:
username: nova
password: Password1
conf:
magnum:
keystone_auth:
password: Password1
barbican:
simple_crypto_plugin:
kek: 5tQw7jaIix7xMUiAEacF3zPDIdVvlwSEtU4xz6mLnxs=yamlEncrypt the file in place, verify decryption locally, and push to the fluxcd-arm branch:
sops --encrypt --in-place clusters/prod/region1/openstack/values/credentials.yaml
sops -d clusters/prod/region1/openstack/values/credentials.yaml > /dev/null
git add .sops.yaml clusters/prod/region1/openstack/values/credentials.yaml
git commit -m "chore: update sops key and encrypt credentials"
git push origin fluxcd-armbashStep 3: Bootstrap FluxCD & Apply Region1 GitOps Manifests#
Bootstrap FluxCD against your GitLab repository, create the GitLab repository and container registry secrets, and apply the region1 Flux Kustomizations:
# 1. Run pre-flight checks
flux check --pre
# 2. Bootstrap FluxCD controllers
flux bootstrap gitlab \
--owner=openstack7461002 \
--repository=openstack-helm \
--branch=fluxcd-arm \
--path=clusters/prod/flux-system \
--token-auth
# 3. Create the GitLab authentication secret for the openstack-helm GitRepository
export GITLAB_TOKEN=<your-gitlab-token>
kubectl create secret generic openstack-helm-auth \
--namespace=flux-system \
--from-literal=username=git \
--from-literal=password="${GITLAB_TOKEN}" \
--dry-run=client -o yaml | kubectl apply -f -
# 4. Create the GitLab Container Registry pull secret in the openstack namespace
kubectl create secret docker-registry gitlab-registry-auth \
-n openstack \
--docker-server=registry.gitlab.com \
--docker-username="<your-gitlab-username>" \
--docker-password="${GITLAB_TOKEN}" \
--dry-run=client -o yaml | kubectl apply -f -
# 5. Apply cluster variables, GitRepository, and region1 Flux Kustomizations
kubectl apply -f clusters/prod/region1/flux-system/cluster-vars.yaml
kubectl apply -f clusters/prod/region1/flux-system/gitrepository-openstack-helm.yaml
kubectl apply -f clusters/prod/region1/flux-system/openstack-sources-kustomization.yaml
kubectl apply -f clusters/prod/region1/flux-system/openstack-pki-kustomization.yaml
kubectl apply -f clusters/prod/region1/flux-system/openstack-kustomization.yaml
# 6. Trigger immediate reconciliation
flux reconcile source git openstack-helm -n flux-system
flux reconcile kustomization openstack-sources -n flux-system
flux reconcile source git openstack-helm-charts -n flux-system
flux reconcile kustomization openstack-pki -n flux-system
flux reconcile kustomization openstack -n flux-systembashStep 4: Verify PKI & Monitor the Deployment#
Confirm that the openstack-ca issuer and certificate are Ready, then watch the HelmReleases reconcile:
# Verify PKI status
kubectl get clusterissuer openstack-ca
kubectl get certificate openstack-ca -n cert-manager
# Watch all OpenStack HelmReleases
flux get helmreleases -n openstack --watch
# Stream helm-controller logs if needed
kubectl logs -n flux-system deploy/helm-controller -fbashkubernetes-entrypoint init containers enforce the following rollout sequence automatically:
mariadb, rabbitmq, memcached
→ keystone (db-init → db-sync → credential-setup → fernet-setup → api)
→ glance, cinder, placement
→ openvswitch → ovn → libvirt
→ neutron → nova
→ barbican → heat → magnum
→ octavia, designate (+powerdns)
→ skyline
→ prometheus exporterstextStep 5: Trust the openstack-ca Certificate & Verify via CLI#
Because both the Envoy Gateway (gateway-openstack-ca-tls) and internal OpenStack services use openstack-ca, exporting a single CA certificate is sufficient for your browser and the OpenStack CLI:
# 1. Export the openstack-ca root certificate
kubectl get secret openstack-ca-secret -n cert-manager \
-o jsonpath='{.data.ca\.crt}' | base64 -d > openstack-ca.crt
# Optional: trust system-wide on Ubuntu/Debian
sudo cp openstack-ca.crt /usr/local/share/ca-certificates/ && sudo update-ca-certificates
# 2. Install OpenStack CLI clients
pip install python-openstackclient python-magnumclient \
python-octaviaclient python-heatclient python-designateclient
# 3. Create ~/.config/openstack/clouds.yaml
mkdir -p ~/.config/openstack
cat > ~/.config/openstack/clouds.yaml << 'EOF'
clouds:
openstack_helm:
region_name: RegionOne
identity_api_version: 3
cacert: /path/to/openstack-ca.crt
auth:
username: admin
password: <your-admin-password>
project_name: admin
project_domain_name: default
user_domain_name: default
auth_url: https://keystone.192.168.64.100.nip.io/v3
EOF
# 4. Verify all OpenStack services
export OS_CLOUD=openstack_helm
openstack endpoint list
openstack compute service list
openstack network agent list
openstack volume service listbashService Catalog#
All public endpoints are served over HTTPS at https://<service>.192.168.64.100.nip.io (with automatic HTTP-to-HTTPS redirect on port 80):
| Service | Public URL | Notes |
|---|---|---|
| Skyline | https://skyline.192.168.64.100.nip.io | Web Dashboard |
| Keystone | https://keystone.192.168.64.100.nip.io/v3 | Identity & Authentication |
| Nova | https://nova.192.168.64.100.nip.io/v2.1 | Compute API |
| NoVNC | https://novnc.192.168.64.100.nip.io | VM Console Access |
| Cinder | https://cinder.192.168.64.100.nip.io/v3 | Block Storage (Ceph RBD) |
| Glance | https://glance.192.168.64.100.nip.io | Image Service (Ceph RBD) |
| Neutron | https://neutron.192.168.64.100.nip.io | Networking (OVN) |
| Placement | https://placement.192.168.64.100.nip.io | Resource Inventory |
| Heat | https://heat.192.168.64.100.nip.io/v1 | Orchestration |
| Heat CFN | https://heat-cfn.192.168.64.100.nip.io | CloudFormation-Compatible API |
| Barbican | https://barbican.192.168.64.100.nip.io | Key & Secret Manager |
| Magnum | https://magnum.192.168.64.100.nip.io/v1 | Kubernetes Cluster as a Service |
| Cyborg | https://cyborg.192.168.64.100.nip.io/v2 | Accelerator Management |
| Octavia | https://octavia.192.168.64.100.nip.io | Load Balancer as a Service (OVN provider) |
| Designate | https://designate.192.168.64.100.nip.io/v2 | DNS as a Service (PowerDNS backend) |
Deployment in Action#
Once logged into Skyline (https://skyline.192.168.64.100.nip.io), you can provision the external flat network (external-net on ens224), create tenant routers, and launch instances or Magnum Kubernetes clusters:



