So i've been thinking. Specifically, i need a local kubernetes with proper services of type: LoadBalancer. On the (not laptop local) clusters, i use metallb, and sometimes with bgp for this.

So it can't be that hard to get this to work on my laptop, right?

Turns out i was right, and it wasnt that hard at all!

Software

For my local kubernetes, i'll be using minikube. Specifically, i create the cluster with minikube start -p minikube-metallb --driver=kvm

I'll be deploying metallb with argocd, but that really doesnt matter. If you wanna follow along, install metallb with helm[1]:

helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb --namespace infra-metallb --create-namespace

I'm not setting any extra values here, as the defaults are fine, and all configuration will be created later manually.

Numbers and stuff

These will likely be different on your setup.

for me, the minikube VM lives at 192.116.50.181 and thhe host lives at 192.168.50.1

We will be using AS numbers from the private range (RFC 6996)

Metallb will be 4200042002, the host will be 4200042001.

frr installation

On the host, we will be using frr. No good reason why frr, i just tried it first.

Install frr with your package manager of choice. For me, that means from the aur.

Edit /etc/frr/daemons, change bgpd=no to bgpd=yes. Save and restart frr.

The rest of the configuration we will be doing with the vtysh cli. So, start vtysh

$ sudo vtysh

Enter configure mode

host# configure

And configure the bgp router

1
# We create a bgp router with our local as number
2
router bgp 4200042001
3
 # eBGP required a policy to be conform with RFC-8212.
4
 # There is/was a bug that this setting also applied to iBPG in some versions of frr.
5
 # As we dont do eBGP, we can just disable it to work around that.
6
 no bgp ebgp-requires-policy
7
 # we configure one neighbor, with the ip of the minikube node and the metallb asn
8
 neighbor 192.168.50.181 remote-as 4200042002
9
 neighbor 192.168.50.181 description Metallb
10
 !
11
 address-family ipv4 unicast
12
  # for ipv4, we want to apply all routes recieved from the metallb peer to our hosts
13
  neighbor 192.168.50.181 soft-reconfiguration inbound
14
 exit-address-family
15
exit
16
!

That should be it on the frr side.

You can check the current config with sh run. If you wanna persist the config, save it with write mem

The bpg status can be checked with:

sh bgp sum
sh bgp neighbors 
sh ip bpg

And on the host, ip r should show the routes

It is expected that nothing is shown yet, as we havent configure the metallb side.

Minikube config

You need to ensure your firewall allows minikube to access resources on your host. In my case, using firewalld this was as simple as

sudo firewall-cmd --zone=trusted --change-interface=virbr1

You will likely need to adapt this to your host.

For verifying it works, you can use python -m http.server on the host, and try to curl host.minikube.internal:8000 from the minikube vm (minikube ssh) or a pod (kubectl run --image=archlinux -it -- curl host.minikube.internal:8000). Check also https://minikube.sigs.k8s.io/docs/handbook/host-access/

Metallb

For metallb to announce via BPG, we will need to create 3 configuration objects.

• IPAdressPool: Which addresses metallb should use
• BGPPeer: Were to connect to for the bpg session
• BGPAdvertisement: Which IPAddressPools should be announced to which bgp peers

For the IPAddressPool, chose some addresses that are not already in use on your host. You MUST avoid these coliding with anything else you need, e.g. your internet connection, your docker daemon etc.

I choose 172.16.0.0/24, so the full object looks like this

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: bgp-pool1
  namespace: infra-metallb
spec:
  addresses:
  - 172.16.0.1/24

The BGPPeer looks like this:

apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: host
  namespace: infra-metallb
spec:
  myASN: 4200042002
  peerASN: 4200042001
  peerAddress: 192.168.50.1

And finally, the BGPAdvertisement:

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bpg-laptop
  namespace: infra-metallb

We dont configure anything in here, because we just want all addresspools be announced by bgp by default. But, it needs to be present anyways.

Testing it

This should be it, now it should work.

Check with sudo vtysh -c 'sh bgp sum' if the connection btween frr and metallb worked.

Create a test deployment and service

kubectl create deployment nginx --image=nginx --replicas 1
kubectl expose deployment nginx --type LoadBalancer --port 80
# Check what ip metallb chose for your loadbalancer
kubectl get svc

You should now see a route to that ip on the host, using ip r

curl it, or open it in your browser to test.