Wave on Kubernetes.
Posted on :: 935 Words :: Source Code
This post was orignially written as a guest blog somewhere else. It is mirrored here for safe keeping. It has not been modifier, so broken links are still broken etc.
https://blog.winter-software.com/2024/02/25/wave-on-kubernetes

Kubernetes is an increasingly more popular deployment option for services. Originally developed by Google0, Kubernetes provides functionality for multi host containerized deployment of applications.

Today, i will take a look at deploying wave onto my Kubernetes.[1] This does not explain how to interact with Kubernetes

Wave depends on some other services to be fully functional, namely I. A PostgreSQL server for persistent data storage II. A redis server for session storage and synchronization III. A reverse proxy for TLS termination IV. An SMTP server for sending email.

Dependencies

Because we want to focus on running wave, let's just get the other stuff out of the way. All our stuff will be happening in the wave-dev namespace, so let's create that first: kubectl create namespace wave-dev

PostgreSQL

Deployed via helm from the bitnami/postgresql chart using the following values:

# postgres.yaml
---
auth:
  username: "wave"
  password: "wavepw12345"
  database: "wave"

using

$ helm install postgres bitnami/postgresql --values postgres.yaml --namespace wave-dev

After a successful installation helm informs us that our Postgres is reachable on postgres-postgresql.wave-dev.svc.cluster.local. Because wave will be running in the same namespace (wave-dev), we can connect to the database server with just postgres-postgresql

Redis

Same story here, deployed via helm from the bitnami/redis chart. Values:

---
architecture: standalone
auth:
  password: "wave-redis-pw-12345"
commonConfiguration: |-
  # Enable AOF https://redis.io/topics/persistence#append-only-file
  appendonly yes
  # Disable RDB persistence, AOF persistence already enabled.
  save ""

Using

helm install redis bitnami/redis --values redis.yaml --namespace wave-dev

With now redis being available as redis-master.wave-dev.svc.cluster.local

Reverse proxy

We will be using standard Kubernetes ingresses, more on that later.

Email

I won't be configuring email right now, but one can use any normal email server.

Wave

Wave is already provided in form of a container, so we don't need to do any extra work on that front.

Kubernetes provides 3 basic types for deploying services:

  • Deployments Basic type, runs 1..n copies of a pod[2]
  • Statefullsets Like deployments, but with more consistency guarantees. Used e.g., for the
  • Daemonsets Runs one copy of a pod per Kubernetes node, mostly used for Kubernetes internal services.

Therefore, we will be using a deployment. The basic variant of a deployment looks like this:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wave-deployment
  labels:
    app: wave
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wave
  template:
    metadata:
      labels:
        app: wave
    spec:
      containers:
        - name: wave
          image: docker.io/miawinter/wave:alpha-13
          ports:
            - containerPort: 80

This would work, but we are still missing some important things, namely:

  • Configuring waves' access to redis and Postgres
  • Have a place for wave to store its files

Luckily, wave allows configuration via environment variables, so we can just add some of these to the containers spec:

env:
  - name: WAVE_ConnectionStrings__DefaultConnection
    value: "Host=postgres-postgresql; Username=wave; Password=wavepw12345"
  - name: WAVE_ConnectionStrings__Redis
    value: redis-master,password=wave-redis-pw-12345

By default, Kubernetes does not allow files to be written to non-persistent locations, to prevent you from accidentally losing data. Therefore, we need to configure some storage for wave. For this, Kubernetes provides so called persistant volumes, commonly shortened to PV[3]. Persistent volumes can either be configured by hand (bääh), or created automatically by Kubernetes. To have Kubernetes create these automatically, we need to tell it what exactly we need, by issuing a persistent volume claim

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: wave-files
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

When applying this claim, the Kubernetes control plane prepares a storage volume for us that:

  • has at least 10Gi of free space
  • is at least ReadWriteOnce capable. Other modes exist, e.g., ReadManyWriteOnce or ReadWriteMany, but we don't need these here because we only want to run one wave instance. Checking a few seconds later with kubectl get pvc wave-files shows the PVC state is Bound. We can now just add this volume to our wave container:
# [...]
spec:
  containers:
    - name: wave
      # [...]
      volumeMounts:
        - name: wave-files
          mountPath: /app/files
  volumes:
    - name: wave-files
      persistentVolumeClaim:
        claimName: wave-pvc

Our deployment now looks like this:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wave-deployment
  labels:
    app: wave
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wave
  template:
    metadata:
      labels:
        app: wave
    spec:
      containers:
        - name: wave
          image: docker.io/miawinter/wave:alpha-13
          ports:
            - containerPort: 8080
          env:
            - name: WAVE_ConnectionStrings__DefaultConnection
              value: "Host=postgres-postgresql; Username=wave; Password=wavepw12345"
            - name: WAVE_ConnectionStrings__Redis
              value: redis-master,password=wave-redis-pw-12345
          volumeMounts:
            - name: wave-files
              mountPath: /app/files
              subPath: files/
      volumes:
        - name: wave-files
          persistentVolumeClaim:
            claimName: wave-files
    securityContext:
      # Wave runs as this user, ensure the fs is writable by it
      fsGroup: 1654

Only one more thing (well technically 2). We need to define an Ingress[4], and matching service. Let's start with the service:

---
apiVersion: v1
kind: Service
metadata:
  name: wave
spec:
  selector:
    app: wave
  ports:
    - name: wave-http
      protocol: TCP
      port: 8080

We basically tell Kubernetes that we want to access wave under the dns-name wave, and that it should map port 8080 to the wave containers' port 8080. So within Kubernetes wave would now be accessible as http://wave:8080/ We can test this by using kubectls' port-forward option.

$ kubectl port-forward services/wave 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Wave should now be accessible on our localhost at port 8080. XXX check.

Since this works, the last thing needed is to expose it properly with TLS. For this, we use an Ingress[4]. I don't really want to go into detail of how ingresses work, but the gist is Everything that arrives with the Host header set to wave.example.com will get connected to the wave container. If the wave container is not available, the ingress presents an error page. Also, we instruct the ingress to enable TLS for wave.example.com, using the letsencrypt acme certificate source. This does basically the same thing as certbot would for a normal reverse proxy, but all automated. This of course requires proper DNS records to be set up already.

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wave-ingress
  annotations:
    # Replace this with a production issuer once you've tested it
    cert-manager.io/issuer: letsencrypt
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  rules:
    - host: "wave.example.com"
      http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: wave
                port:
                  number: 8080
  tls:
    - hosts:
        - wave.example.com
      secretName: wave-ingress-tls

Shortly after applying this, wave should be accessible on https://wave.example.com

Create an account, and to elevate it to the admin role, check the logs of the wave pod using kubectl logs deployments/wave-deployment. There should be a line like There is currently no user in your installation with the admin role, go to /Admin and use the following password to self promote your account: [password] Enter the password on https://wave.example.com/admin to promote yourself to admin.

You're done now, wave should be running in Kubernetes

[1]: A three node cluster deployed with kubespray [2]: A pod is a collection of one or more containers. [3]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/ [4]: Basically a reverse proxy on steroids. Even does TLS without any further configuration needed [5]: How the ingress finds the wave containers.