Deploying Helm / Tiller, Prometheus, AlertManager, Grafana, Elasticsearch On Your kubernetes Cluster – Part 2

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)

Installing, configuring Prometheus and Grafana

Below I am continuing, with options on installing Prometheus and Grafana.

This is Part 2. in Part 1 I am describing what it takes to install Helm, Tiller as well as SSL/TLS configuration.

It has been a while, I didn’t had a chance to complete the Prometheus & Grafana installation steps.

lets first upgrade helm to v2.11(In my original testing, v2.1.1 had bugs which are now supposed to be fixed).

Upgrading Helm and Tiller

Next, lets upgrade the repo.

Lets try to look on the pkg list.

So lets upgrade tiller.

Hooray, it works again.

Installing / configuring Prometheus and Grafana

There are many ways you can install Prometheus and Grafana.
Below I am focusing on two methods.

  1. Simple easy installation by using the CoreOS Operator helm workflow.
  2. Semi simple, a bit more involved, manual separate stock prometheus and grafana installation.

Simple easy by using CoreOS Operator

This method is relay simple (if all works ok)

First we have to add the CoreOS repo.

Ok, lets move on to the installation.
The below instillation will install the CoreOS prometheus operator.

Next, lets install the stack.
To keep things simple, you can just run the below, meaning no persistent volumes and no ingress configuration (good for a lab, etc).

You can run something like the below.
Note: The below configuration is using an ingress controller, as well as setting persistent volumes, more on this below in the manual configuration (creating persistent volumes).

After a little while, you shuld be up and running, you can check by running the below.

Now all you got a do is forward you port.
The way I like to do this, is use the port-forward to port forward port 3000
Then use ssh to forward to my local pc to test if all works, I can then go in my browser to localhost:3000

While at this, you can also install heapster and the kubernetes dashboard like the below.

Of course all of them can be port-forward or use an ingress controller.
Below is a confirmation of the installation.

Cleaning up.

Below are a few screen shots of the included CoreOS Dashboards.
Kubernetes cluster status

Kubernetes nodes

Kubernetes capacity planing

Manual stock installation of Prometheus/stable and Grafana/stable

By going this route we have to pre-stage a persistent volume.
So lets get that out of the way first.
You shuld use a NAS mount or something like CEPH, DRDB, etc..
For demonstration purpose this is good enough, lets create a few directory’s on each node.

Next, lets create the persistent volume, create the below file.
cat volume_create.yaml

Then run the below to create the pv volume.

To verify run the below.
Note: This volumes can only be used once, after one use you have to delete and re-create the volume.

We are now ready to move to the next step.

Installing Prometheus

Now, lets first install the stable/prometheus, you do so by running the below.

Verify that all services are up, before continuing to grafana.

Note: You can access prometheus by forwarding the port 9090, something like the below.

To see how the prometheus server allocated the pv volumes , run the below.

Installing Grafana

Lets fetch the stable/grafana, we do so, so we can modify the installation options while still using/installing the stable/grafana from the repo.

Now lets inspect the grafana/values.yaml, and make some modifications.
Lets modify the persistence: section, I used something like the blow.
Note: Without these change, all metrics will be lost once the pod reboots.

To set a grafana admin user and password, make sure to set something like the below.

To get the local kubernetes prometheus cluster url, run something like the below (this is needed in the next section).

Lets also modify the datasources: section, I used something like the blow.
Note: Without the below change, no datasources will be configured in Grafana once it comes up.

To add the grafana kubernetes dashboards plugins, also modify the plugins: section, I used something like the blow.

We are now ready to install the grafana chart, you do so by running the below.

Cleaning up.
Removing Prometheus and Grafana.

Helpful links.
List of Grafana properties
Install Prometheus Operator
Install Prometheus and Grafana
Kubernetes monitoring with Prometheus

You might also like – Other related articles to Docker and Kubernetes / micro-service.

Like what you’re reading? please provide feedback, any feedback is appreciated.

Leave a Reply

Notify of