Using MetalLB And Traefik for Load balancing on your Bare Metal Kubernetes Cluster – Part 1
Running a Kubernetes Cluster in your own data center on Bare Metal hardware can be lots of fun but also can be challenging.
One of the changeless are exposing your service to an external Load Balancer, Kubernetes does not offer an implementation of an external load-balancer for bare metal cluster implementations.
The Kubernetes implementations of Network Load Balancers are only available on specific Cloud Providers like AWS, GCP, Azure, etc.. and is enabled by specifying the –cloud-provider= option. leaving a gap/challenge if you create your own Bare Metal Kubernetes Cluster.
While their are many workarounds to address this issue most of them are not perfect and have their own list of challenges. while I was looking for a for a solution, most recently came across a project called MetalLB, at first I was quite sceptical if its going to work, but after testing this for a while I have to say, I am quite happy with the solution.
Below is a quote from the MetalLB website.
Kubernetes does not offer an implementation of network load-balancers (Services of type LoadBalancer) for bare metal clusters. The implementations of Network LB that Kubernetes does ship with are all glue code that calls out to various IaaS platforms (GCP, AWS, Azure…). If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created.
Note: On the MetalLB project website they warn you as follows. MetalLB is a young project. You should treat it as a beta system. The project maturity page explains what that implies., however I using this for a while and it seems quite mature.
One last note, I am continuing to use Traefik for internal micro services / load balancing which works great, however MetalLB helps exposing your service for external use.
Now that we have that out of the way, lets see what it takes to install and configure MetalLB on your Kubernetes cluster.
Installing MetalLB on your Kubernetes cluster
You have two options to install MetalLB on your Kubernetes cluster.
- Use the a per-packged MetalLBHelm chart.
- Install / configure manually by applying a YAML configuration file.
Installing MetalLB with Helm
First, lets create a configmap which will be using at helm install time, the below configmap was used with a flannel configuration.
configInline: address-pools: - name: default protocol: layer2 addresses: - 10.90.10.190-10.90.10.199
Note: There are a number of networking options you can specify likeprotocol: BGP, etc.. please take a look at the MetalLB website for a full list of options.
Once helm is available on your cluster, just run the below to install MetalLB (omit the -tls if your are not using a secure configuration).
Note: The below configuration installs MetalLB with helm including a configmap. (otherwise you get the errors below)
helm install --name=metallb --namespace=metallb-system -f values.yaml stable/metallb --tls # If all is ok, you shuld see something like the below message at the end of the helm deploy. MetalLB is now running in the cluster. LoadBalancer Services in your cluster are now available on the IPs you defined in MetalLB's configuration. To see IP assignments, try `kubectl get services`. # Without specifying a configmap. helm install stable/metallb --name=metallb --namespace=metallb-system --tls
Errors you will see if you don’t specify / use a configmap to list your network range, etc..
MetalLB is now running in the cluster. WARNING: you specified a ConfigMap that isn't managed by Helm. LoadBalancer services will not function until you add that ConfigMap to your cluster yourself.
You can also check the logs to make sure all looks good, by running the below
kubectl logs -l component=speaker -n metallb-system
If your cluster is not configured to use RBAC, you might run somthing like the below.
helm install \ --set rbac.create=false \ -f values.yaml \ stable/metallb --name=metallb --namespace=metallb-system --tls
An example configmap if using a BGP overlay system.
# cat values.yaml configInline: peers: - peer-address: 10.0.0.1 peer-asn: 64512 my-asn: 64512 address-pools: - name: default protocol: bgp addresses: - 10.90.10.190-10.90.10.199
If all works correctly, you should now have a MetalLB working configuration.
You can also check what helm report about the installation.
helm list --tls NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE metallb 1 Mon Feb 11 19:52:17 2019 DEPLOYED metallb-0.8.4 0.7.3 metallb-system
To remove MetalLB from your cluster, just run the below.
helm delete --purge metallb --tls
Installing MetalLB manually
To install MetalLB manually, you would do so in cases ware you don’t have a helm configuration or you would like to install a version not available in the helm repository.
To MetalLB from the web just run the below.
kubectl apply -f https://raw.githubusercontent.com/google/metallb/master/manifests/metallb.yaml
You can also create a deploy file, like the one below.
cat deploye_metal.yaml apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: metal-lb-ip-space protocol: layer2 addresses: - 10.90.10.190-10.90.10.199
Then to install just run the below, this will already include your IP address definitions.
kubectl apply -f deploye_metal.yaml
A quick check on your pods will reveal something like the below, note the metallb-controller and metallb-speaker.
kubectl get po --all-namespaces metallb-system metallb-controller-776cdd69dc-4lzcx 1/1 Running 0 9d metallb-system metallb-speaker-4xrnm 1/1 Running 0 9d metallb-system metallb-speaker-g59h6 1/1 Running 0 9d metallb-system metallb-speaker-jzh4g 1/1 Running 0 9d metallb-system metallb-speaker-k6spd 1/1 Running 0 9d
MetalLB in action
To test MetalLB working, you can deploy any pod(s) to you cluster, and specify the Cluster Type as LoadBalancer.
An example Nginx deployment is below.
# cat nginx_load_balance.yaml apiVersion: apps/v1beta2 kind: Deployment metadata: name: nginx spec: selector: matchLabels: app: nginx replicas: 2 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1 ports: - name: http containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx spec: ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
Now, just run the below to deploy to your cluster.
kubectl apply -f nginx_load_balance.yaml
Now take a look on your Kubernets Cluster services, note the EXTERNAL-IP that got assigned automatically by MetalLB.
kubectl get svc --all-namespaces NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.30.0.1
443/TCP 14d kube-system tiller-deploy ClusterIP 10.30.4.154 44134/TCP 13d default nginx LoadBalancer 10.30.2.56 10.90.10.190 80:30827/TCP 2s ...
In the next post Part 2 – I will show you how to combine this with Traefik for internal Micro-services Load Balancing.
I hope you enjoyed reading Using MetalLB And Traefik Load Balancing For Your Bare Metal Kubernetes Cluster, give it a thumbs up by rating the article or by just providing feedback.
You might also like – realted to Docker Kubernetes / micro-services.