Skip to content

Strategy

The previous page fixes the six layers and their dependencies. This one explains why that order cannot be permuted, what a single command applies and what is applied by hand, the place of ArgoCD, two pairs of mechanisms that are easy to confuse (liveness and readiness probes, HPA and metrics-server), and how to roll back after an update.

Why the order of the six layers cannot be permuted

Four concrete dependencies, each of them already observed or already verified in this repository, justify the order fixed on the previous page.

  • Secrets before certificates, even though nothing hints at it. The ClusterIssuer of the DNS-01 challenge resolves its Cloudflare token through the ExternalSecret cloudflare-dns-token, fed by Vault through the ClusterSecretStore vault-store. Issuing certs/ before that path works leaves the ClusterIssuer without a token: the DNS-01 challenge never resolves, the wildcard certificate stays False indefinitely, and nothing tells the operator that the cause is a link one layer further away. None of this makes the order circular: Vault (secrets/vault.yaml, namespace vault) is covered by kubectl kustomize ., hence available from the very first command, and its Service vault.vault.svc:8200 resolves through the standard internal DNS of Kubernetes, depending neither on the customised CoreDNS nor on any certificate. It is that route, not an Ingress, that is used to unseal Vault and to write the Cloudflare token into it.
  • The certificate before the Ingresses. The Traefik TLSStore references the wildcard certificate issued by cert-manager. An Ingress applied before that certificate is Ready still starts, but Traefik can only serve it a default certificate, not the one for the customer's domain.
  • Secrets before applications. External Secrets Operator injects the connection credentials (application database, SMTP, MinIO) into the Kubernetes Secrets consumed by the .NET pods and the portals. An application started before those Secrets exist restarts in a loop with no explicit message on the Kubernetes side, the same symptom as an unreachable external data endpoint.
  • CoreDNS before the .NET pods. The resolution of *.${TV_DOMAIN} by CoreDNS conditions the resolution of the name for the .NET services. The trap already hit on this repository, a CNAME answer that glibc does not follow, replays itself if a .NET pod starts before CoreDNS is in place. TLS trust no longer depends on anything: since the internal CA was removed on 2026-09-10, *.${TV_DOMAIN} is signed by Let's Encrypt, already present in the default store of every image.

Placing the Cloudflare token by hand does not exempt you from populating Vault

It is possible to create the cloudflare-dns-token Secret by hand, without going through Vault or ESO, in order to issue the wildcard without waiting. That shortcut does not hold over time: as soon as ESO starts, the ExternalSecret of the same name takes that Secret over. If the corresponding Vault path has not been populated beforehand, the token placed by hand disappears at that moment, with no message to signal it. The failure then shows up only at the next renewal of the certificate, weeks later, far from its real cause. That is why the recommended order remains Vault and ESO first, never the other way round.

What a single command covers, what is applied per directory

The repository is not entirely kustomize, for a historical reason: the apps, ingress, externalsecrets, keycloak and mailpit modules were added after the base layer, without being reintegrated into the root kustomize. kubectl kustomize . therefore only covers that base layer: namespace, network, cert-manager, cluster-internal data layer, one observability stack, Vault, Traefik, ArgoCD. Everything else is applied per directory. The inventory below is ordered by layer, it is not an execution order and it is not the installation procedure: it names what the base command alone leaves aside. Playing it from top to bottom would fail, if only because kube-vip and the Cloudflare token must come before certs/. The exact commands, their order and their prerequisites live in the Step by step.

Layer What the root kustomize does not lay down Effect of the omission
2, secrets secrets/eso/vault-store.yaml no ExternalSecret synchronises
3, certificates core/cert-manager/cloudflare-dns-token.yaml, then certs/ the ClusterIssuer has no token, the wildcard is never issued
4, network and ingress core/kube-vip/ no LoadBalancer service obtains an address
4 core/traefik/traefik-rbac.yaml, core/traefik/traefik-ingressclass.yaml Traefik runs and reads no Ingress, hence serves no route, with no visible error
4 core/traefik/tlsstore-default.yaml Traefik serves its self-signed certificate on every host
4 core/coredns/coredns-configmap.yaml *.${TV_DOMAIN} does not resolve from the pods
4 the two named Ingresses of core/platform-access/, core/lan-dns/ no access to the operating interfaces, no LAN or VPN resolution
4 core/metrics-server/metrics-server.yaml kubectl top silent, ten HPA inert at <unknown>
4 core/monitoring/node-exporter.yaml, which step 10 of the step by step adds outside its main block no host metric (CPU, memory, disk of the nodes) reaches Prometheus; the tradeval-ressources dashboard stays empty on those series
6, applications apps/api/, apps/keycloak/, apps/web/ neither Keycloak, nor microservices, nor portals
6 the four files of core/cloudflare/, in order no public exposure

The node-exporter line is the only one in the table that the base layer did not lay down and that no inherited manifest called for: it is step 10 of the Step by step that adds it, with its command and its verification, outside its main block so that this status stays visible. Do not confuse it with metrics-server: both serve metrics, but only metrics-server feeds kubectl top and the HPA, and only node-exporter feeds Prometheus.

core/vpn/ does not appear in this table, and that is not an oversight: what that directory contains is not applied by the procedure. VPN access goes through a gateway VM outside the cluster, the option retained by the repository; wireguard-incluster.yaml is option B, its header carries Non appliqué, and it mounts a Secret that no applied manifest creates. Step 13 of the Step by step.

An operator who does not know about this split risks a silent omission: applying only kubectl kustomize . leaves the cluster without a VIP of the customer's own, without a wildcard, without a customised CoreDNS, without CA trust, without a ClusterSecretStore hence without a single synchronised secret, without LAN or VPN resolution, without active HPA, without Keycloak, without microservices or portals, and without public exposure, with no error to signal it at the time of the apply.

Two observability stacks, only one actually wired up

The root kustomize deploys a Prometheus, Grafana, Kibana stack in the observability namespace. It runs, but nothing connects to it: the stack actually used to operate the instance (Prometheus, Grafana, Loki, Alloy) is installed separately by Helm, in the monitoring namespace, outside the scope of kubectl kustomize .. An operator who applies the root kustomize gets the first stack and sees no useful metric go through it, with no message to explain why. The Observability page details which of the two to install and how.

The place of ArgoCD

The installation remains driven by the kubectl apply commands of this page and of the previous one, run by hand. ArgoCD comes after it: installed by Helm from platform/argocd/, it then follows the main branch of this repository for the application directories (apps/), and application updates go through git. The base layers (Vault, External Secrets, network) stay applied by hand. See platform/README.md.

Liveness and readiness probes, why they are distinct

A readiness probe (readinessProbe) removes a pod from the Service as long as it does not answer, without restarting it. A liveness probe (livenessProbe) restarts the pod if it fails for too long. Confusing the two, or pointing either of them at an unsuitable path, produces very different symptoms: a pod that restarts in a loop with no application reason in one case, a pod that is Running but unreachable for hours in the other, with no classic dashboard signalling it since it counts Running pods.

It is exactly that second symptom that occurred on the reference instance: a portal stayed unavailable for several hours, without restarting and without any error appearing on the Kubernetes side, because its readiness probe followed a redirect towards an authentication page whose rendering time exceeded the delay granted to the probe. The lesson to retain: a probe path must never redirect, nor authenticate, nor depend on an external service. On the ten deployments of this repository that expose an application HTTP route, the readiness and liveness probes are already separated, each on a direct path (/health and /health/ready on the .NET side, /api/health on the Next.js portals side): an operator who adds a service will follow that same scheme rather than point a probe at a page that can redirect.

The second half of the rule, the delay granted to the probe, is not respected everywhere in the delivered repository, however: the readiness probe of hub-portail does not set timeoutSeconds and therefore takes the default of one second, the very value that the incident incriminates. To be fixed before go-live, detail at Troubleshooting, entry 11.

HPA and metrics-server

The ten HPA of this repository, applied with the rest of apps/api/ and apps/web/, react to nothing as long as metrics-server is not installed: they stay inert, and signal it only by an <unknown> value in kubectl get hpa. The manifest is core/metrics-server/metrics-server.yaml, applied at layer 4 in the block of commands above: it does not live in kubectl kustomize ., and an operator who does not apply it ends up with ten inert autoscalers with no message to indicate it. Installing metrics-server wakes up all the existing HPA at once, with their configuration as it is written in the repository, with no running-in period. That is what happened during the reference installation: the HPA climbed to their maximum within a few minutes, saturating the CPU of the cluster and blocking every deployment under way. The mechanism, the delivered values and the way to check that they have not drifted are at Troubleshooting, entry 10; the lesson to retain is that the CPU and memory requests values, and the maxReplicas bounds, must be confronted with real usage before installing metrics-server, not after.

Updating a service and rolling back

Rolling back an application deployment uses kubectl rollout undo, as for any standard Kubernetes Deployment.

kubectl rollout undo deployment/<nom> -n <namespace>

That mechanism is enough for the .NET microservices, whose configuration is read when the container starts. It is not enough for the Next.js portals: their NEXT_PUBLIC_* variables are frozen at the moment the image is built, not at startup. If the rollback follows a domain change, going back to the previous image restores code that still calls the old ${TV_DOMAIN}, even after the rollback. In that specific case, a complete rollback requires rebuilding the portal image with the right variables, not merely changing the deployed version.