Configuring Kubernetes 3 Node Cluster On CoreOS Without Tectonic – Part 1

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)

Configuring a 3 node(master) Kubernetes cluster

Below I am going to describe the process to install/configure a 3 node Kubernetes cluster. the configuration will contain 3 Kubernetes master nodes (i.e. controller, api, scheduler, proxy), I will also add 1 worker node(additional workers can be added as needed).

Most of the examples below are still relevant for the overall kubernetes configuration. However, some of the YAML file examples below are outdated and will only work with kubernetes v1.8.x.
For a copy of the most recent CoreOS examples kubernetes version v.1.13.x+, is available on my GitHub repository by going here.
You can Also generate your own by using the generator also available on my GitHub repository by going here.
A more up-to-date write up is also available Installing a 3 master node Kubernetes cluster on CentOs.

Why not use Tectonic?
Two reasons for that.

  1. While Tectonic is great, I am still learning how Kubernetes works, and the only way to understand is, by creating your own cluster from scratch.
  2. Tectonic comes with a 10 node license limitation.

I divided the configuration into parts outlined below (still working progress).

Note: An up-to-date example is available on my GitHub project page, or generate your own Kubernetes configuration with the Kubernetes generator available here on my GitHub page.

This is part 1 – Initial setup – getting CoreOS and prepare SSL certificates

What items, versions are in the configuration below, and what not.
Items versions used.

  1. Openssl or Cfssl to generate certificates.
  2. CoreOS: 1576.1.0
  3. Kubernetes/kubelet/kubectl: 1.8.1
  4. Etcd: version 3
  5. Flannel: version 0.90-0.30
  6. Default Container engine Rkt: version 1.29.0 (no Docker is being used)
  7. Use SSL for all communication if possible

Items not included.

  1. Matchbox configuration

Note: This setup is still in progress, I will be updating on further development.

With that said, let’s jump right in.

OS configuration – CoreOS preparation

Lets start with the OS, I opted in using CoreOS which can be download form here – CoreOS ISO images.
The reason for selecting CoreOS is obvious, CoreOS runs the process in a container(rkt) and has out of the box excellent Kubernetes support plus many more great features.

A few notes about recent CoreOS development.

  1. CoreOS used to have this so called cloud-config configuration file, in the recent versions they moved to something they call container linux config (ignition config).
    If you are migrating from a cloud config to a container linux config, you might find migrating to ignition and ignition config refrence helpful.
  2. To create an ignition config you will need the ct utility, which you can download from here and a helpfull video how to use it from here
  3. The ct utility (transpiler) will generate an ignition config which you can then use to configure your nodes.

An example on how to use the ct utility is below,

You can skip and jump directly to a complete ct/ignition ready files for a 3 Node cluster. Click here for Node 1, Node 2 and Node 3.

In my configuration below I am using 3 Virtualbox VM’s, to see how to configure those VM’s you can read this.. Note: that using physical hardware would apply the same or similar process and can be fully automated with matchbox.

Last, CoreOS has also a project called matchbox, matchbox is an HTTP and gRPC service that renders signed Ignition configs, cloud-configs, network boot configs, and metadata to machines to create Container Linux clusters. matchbox helps fully automate the Ignition config. For more details you can check out CoreOS matchbox documentation here.

Prepare SSL certificates by using Openssl or Cfssl

Since Several components of Kubernetes require SSL, to simplify the certificate creation I created only two certificates.

  1. CA certificate
  2. General certificate – used for all components
  3. Optional create worker certificates

Since the certificate is shared i.e. re-used by all components i.e. Etcd, Docker, Rkt, Kubelet and all Manifests, etc.. the certificate has to contain all the names and ip’s, more is described below.

The Kubernetes node names, IP address are in the table below.

CoreOS Cluster IP Address
 Name  IP Addrss

Generating the Certificates by using Cfssl

Cfssl Reference
Since many OS’s do not include the cfssl application, the installation steps are below.

Now, create the ca-config.json json files. (I modified the expire time)

Next, create the ca-csr.json file – used for CA configuration.

Next, create the etcd-node-csr.json – used for etcd and all other components.
Note: Replace with your domain

Create the below if you would like to use a separate key for the kub-proxy communication.
Note: Replace with your domain

Now, just run the below which will generate the certificate and keys used latter in the kubernetes configuration.

You should now have all the files below.

Generating the Certificates by using OpenSSL

Since most linux distros include Openssl, I am including steps to use that below.

Create your config file cert.conf
Note: Replace with your domain

Run the below to generate the certificates CA and general.

You should now have the below files.

You are now ready to move to the next step, configuring Etcd – in part 2.

You might also like – Other articles related to Docker Kubernetes / micro-service.

Like what you’re reading? please provide feedback, any feedback is appreciated.

Leave a Reply

1 Comment threads
1 Thread replies
Most reacted comment
Hottest comment thread
2 Comment authors
Eli KleinmanDan Recent comment authors
newest oldest most voted
Notify of

Good Morning

Would it be possible to provide an example of “3.Optional create worker certificates”

This guide has been really useful in understanding how to secure K8s in my own test environment, had been struggling to find any relevant information on-line previously.