Creating, Hosting your own CoreOS Rkt (ACI) Images

Creating your own (ACI) package

The document below uses the ACI image format since we are dealing with Rkt. in the future Rkt might switch to the OCI format. to note Docker is using the OCI image format, which is part of the opencontainer initiative. The steps below, show you how to create/host and sign your own Rkt package/images, you can also always download/convert any Docker package to the ACI Rkt format. To create an ACI package, you will need the actool utility included in the CoreOS build, or you can use docker2aci to create/convert your images. First, Lets create a directory structure, similar to one below. An example directory layout is below.

mkdir -p app1/rootfs/bin app1/rootfs/lib64

# copy required library's
cd app1/rootfs/lib64
cp /usr/lib/libncurses.so.5 .
cp /usr/lib/libncurses.so.5.9 .
cp /usr/lib/libc.so.6 .
cp /usr/lib/libc-2.23.so .
cp /usr/lib/libncurses.so.5.9 .
cp /usr/lib/ld-2.23.so .
cp /usr/lib/ld-linux-x86-64.so.2 .
cp /usr/lib/libreadline.so.6 .
cp /usr/lib/libreadline.so.6.3 .
cp /usr/lib64/libdl.so.2 .
cp /usr/lib64/libdl-2.23.so .
ln -s /usr/lib/libncurses.so.5.9 libtinfo.so.5

# copy bash
cd app1/rootfs/bin
cp /usr/bin/bash .

Next, create something to run, can be a simple as echo hello.

chmod +x app1/rootfs/bin/app1

cat app1/rootfs/bin/app1
echo hello
# Or
printf hello"\n"

Now, lets create our manifest, something like the below should work. Note: For a full list of available options click here.

{
    "acKind": "ImageManifest",
    "acVersion": "0.0.1",
    "name": "devtech101.com/app1",
    "labels": [
        {"name": "os", "value": "linux"},
        {"name": "arch", "value": "amd64"},
        {"name": "version", "value": "v0.0.1"}
    ],
    "app": {
        "environment": [
          {
            "value": "/usr/sbin:/usr/bin:/sbin:/bin",
            "name": "PATH"
          }
        ],
        "eventHandlers": null,
        "exec": [
            "/bin/bash",
            "-c",
            "/bin/app1"
        ],
        "user": "0",
        "group": "0"
    }
}

Next, lets compile the code to create the aci, run the below to do so.

# add --overwrit if the aci exist.
actool build /var/tmp/rkt_aci/app1 /var/tmp/rkt_aci/v0.0.1-linux-amd64.aci

Now, Lets publish the code so we can use it. Note: The below fetch method is with no security i.e. without a key, if you like to use / configure your own gpg key, you can just follow the below gpg configuration.

rkt fetch --insecure-options=image v0.0.1-linux-amd64.aci

To run the application app1, just run the below.

rkt --insecure-options=image run (--interactive) app1:0.0.1
# Or debug
rkt --debug --insecure-options=image run --interactive app1:0.0.1
# It should return an hello.
[520510.712704] app1[6]: hello

Now, we can distribute / publish the aci to our local image registry servers.

Securing / signing / setting your GPG key on your ACI images

To distribute an aci application, the application image would usually to be secured by using a GPG key to sign the image. To sign the image we first need to create our own GPG key. Lets create our own key.

# Create a working dir.
mkdir gpg_keys

Create a gpg file, something like the below. cat gpg-batch

%echo Generating a default key
Key-Type: RSA
Key-Length: 2048
Subkey-Type: RSA
Subkey-Length: 2048
Name-Real: Eli
Name-Comment: ACI signing key
Name-Email: eli@devtech101.com
Expire-Date: 0
Passphrase: rkt
%pubring rkt.pub
%secring rkt.sec
%commit
%echo done

Now, lets generate gpg key

gpg --batch --gen-key gpg-batch

Note: If gpg key generating hangs, like the one below. stop the process, and try to install rng-tools. like below

gpg --gen-key --batch gpg-batch
gpg: Generating a default key
^C
gpg: signal Interrupt caught ... exiting

apt install -y rng-tools
...
rngd -r /dev/urandom

# Now re-run the gpg application.

Next, lets output/get the key (required for next step).

gpg --no-default-keyring \
--secret-keyring ./rkt.sec --keyring ./rkt.pub --list-keys
---------
pub   2048R/0AC4E6F2 2017-12-27
uid       [ultimate] Eli (ACI signing key) 
sub   2048R/9D140E0E 2017-12-27

Now, lets trust our key, by running the below (use the key output from above).

gpg --no-default-keyring \
--secret-keyring ./rkt.sec \
--keyring ./rkt.pub \
--edit-key 0AC4E6F2 \
trust
...
# Select option 5
5 = I trust ultimately

# Then trust the key, like so.
Do you really want to set this key to ultimate trust? (y/N) y

Now, lets export the public key to use latter.

gpg --no-default-keyring --armor \
--secret-keyring ./rkt.sec --keyring ./rkt.pub \
--export eli@devtech101.com > pubkeys.gpg

Last

Finally, lets sign the aci image with the GPG key.

gpg --no-default-keyring --armor \
--secret-keyring ./rkt.sec --keyring ./rkt.pub \
--output v0.0.01-linux-amd64.aci.asc \
--detach-sig v0.0.01-linux-amd64.aci

Enter the key password when prompted which in our case is rkt.

We should now have 3 files, the public GPG key, the aci(app file), and the asc(signature file). we will be using this in next steps to publish the package.
# Example aci, asc, pubkeys
-rw-r--r-- 1 root root 1845467 Dec 27 15:19 v0.0.01-linux-amd64.aci
-rw-r--r-- 1 root root     473 Dec 27 15:35 v0.0.01-linux-amd64.aci.asc
-rw-r--r-- 1 root root    2108 Dec 27 15:35 pubkeys.gpg

Creating your own (rkt) registry
One of the great benefits of using RKT is the image distribution(hub/registry) is super simple.

You will need a running web server on to host that contains all the images.

 	
Generate web server Certificate (self signed or public)
 	
Create directory structure
 	
Copy aci and gpg keys
 	
Create ac-discovery rules

Generate a certificate
The example below uses a self signed certificate, lets first Generate a CA certificate, so we can sign our certificate.

To do so create a file called cert.conf, like the one below.
cat cert.conf
[req] 
default_bits       = 2048
prompt             = no
default_md         = sha256
distinguished_name = dn
req_extensions     = v3_req
x509_extensions    = v3_ca
 
[ dn ]
C                  = US
ST                 = NY
L                  = New York
O                  = Company1
OU                 = Ops
CN                 = CA
 
[ v3_ca ]
keyUsage = critical,keyCertSign, cRLSign
basicConstraints = critical,CA:TRUE
subjectKeyIdentifier = hash

[ v3_req ]
keyUsage = critical,digitalSignature, keyEncipherment, nonRepudiation
extendedKeyUsage = clientAuth, serverAuth
basicConstraints = critical,CA:FALSE
subjectKeyIdentifier = hash
subjectAltName = @alt_names
 
[ alt_names ]
DNS.1              = devtech101.com
DNS.2              = rkt.devtech101.com
DNS.5              = localhost
DNS.8              = rkt
IP.1               = 127.0.0.1
IP.2               = 0.0.0.0
email              = admin@devtech101.com

Next, run the below openssl commands to generate the certificates.
# Generate the CA private key.
openssl genrsa -out ca-key.pem 2048
 
# Generate the CA certificate.
sed -i 's/^CN.*/CN                 = CA/g' cert.conf
openssl req -x509 -new -extensions v3_ca -key ca-key.pem -days 3650 \
-out ca.pem \
-subj '/C=US/ST=New York/L=New York/O=devtech101.com/CN=CA' \
-config cert.conf

# Generate the server/client private key.
openssl genrsa -out rkt-reg-key.pem 2048
 
# Generate the server/client certificate request.
sed -i 's/^CN.*/CN                 = rkt/g' cert.conf
openssl req -new -key rkt-reg-key.pem \
-newkey rsa:2048 -nodes -config cert.conf \
-subj '/C=US/ST=New York/L=New York/O=devtech101.com/CN=rkt' \
-outform pem -out rkt-reg-req.pem -keyout rkt-reg-req.key
 
# Sign the server/client certificate request.
openssl x509 -req -in rkt-reg-req.pem -CA ca.pem -CAkey ca-key.pem -CAcreateserial \
-out rkt-reg.pem -days 3650 -extensions v3_req -extfile cert.conf

You should see many files create, we only need 3 of these files.
# CA file
cp ca.pem /etc/ssl/certs/

# Save for the we server configuration
rkt-reg-key.pem
rkt-reg.pem

On CoreOS, run update-ca-certificates, on Ubuntu rename to .crt and run dpkg-reconfigure ca-certificates (if update-ca-certificates is not working).
Web server and directory structure
There are many options you can use for a web server, like flask, etc...
I will use a simple Python HTTPS web server.

Create a file web_server.py with the below content.
#!/usr/bin/env python

import BaseHTTPServer, SimpleHTTPServer
import ssl

httpd = BaseHTTPServer.HTTPServer(('rkt.devtech101.com', 443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, keyfile='rkt-reg-key.pem',
                                certfile='rkt-reg.pem', ca_certs='ca.pem', server_side=True)
httpd.serve_forever()

chmod +x web_server.py

Create a index.html , like the one below.
cat index.html 

  
    
   
   
   


In the above index.html configuration I am using a separate directory for every application.this

For example in our case, the application is named app1.
Lets create an app1 directory, and copy the app1 application files.
mkdir app1

# app1 files
ls -l app1
total 2580
-rw-r--r-- 1 root root 2636322 Jan 11 12:20 v0.0.1-linux-amd64.aci
-rw-r--r-- 1 root root     455 Jan 11 12:20 v0.0.1-linux-amd64.aci.asc

The last configuration is to copy the pubkeys.gpg to the root directory.
Now lets start the web server, by running.
./run_web.py &

Now, we are ready to test the rkt app discovery by running the below.
rkt fetch rkt.devtech101.com/app1:v0.0.1
pubkey: prefix: "rkt.devtech101.com/app1"
key: "https://rkt.devtech101.com/pubkeys.gpg"
gpg key fingerprint is: 9E90 CD8C A814 88F2 86CB  6683 9823 67D1 E8F7 6CF7
    Subkey fingerprint: 3645 CCAA 8232 C62A B52E  89FF 63EB 0CAD 7333 2C89
	Eli (ACI signing key) 
Are you sure you want to trust this key (yes/no)?
yes
Trusting "https://rkt.devtech101.com/pubkeys.gpg" for prefix "rkt.devtech101.com/app1" after fingerprint review.
Added key for prefix "rkt.devtech101.com/app1" at "/etc/rkt/trustedkeys/prefix.d/rkt.devdtech101.com/app1/9e90cd8ca81488f286cb6683982367d1e8f76cf7"
10.10.10.10 - - [11/Jan/2018 14:24:49] "GET /app1/v0.0.1-linux-amd64.aci.asc HTTP/1.1" 200 -
Downloading signature: [=======================================] 455 B/455 B
10.10.10.10 - - [11/Jan/2018 14:24:49] "GET /app1/v0.0.1-linux-amd64.aci HTTP/1.1" 200 -
Downloading ACI: [=============================================] 2.64 MB/2.64 MB
image: signature verified:
  Eli (ACI signing key) 
sha512-af012352c31579df90db605baaa140de

A quick verification should show something like the below.
rkt image list
ID			NAME					SIZE	IMPORT TIME	LAST USED
sha512-c8c21115c820	coreos.com/rkt/stage1-host:1.21.0	94 MiB	2 hours ago	2 hours ago
sha512-af012352c315	rkt.devtech101.com/app1:v0.0.1		5.5 MiB	2 minutes ago	2 minutes ago

You can also try to run the app something like the below.
rkt run rkt.devtech101.com/app1:v0.0.1
[520510.712704] app1[6]: hello

Helpful link
CoreOs Signing and Verification Guide
App Container Image Discovery
Rkt meta discovery tag
Adding public packages to your local registry/repository
Below I am going to add etcd to my local registry.

First lets fetch the image from one of the public repository.
rkt fetch quay.io/coreos/etcd:v3.2.9

# List the images
rkt image list
ID			NAME						SIZE	IMPORT TIME	LAST USED
sha512-141f988f5450	quay.io/coreos/etcd:v3.2.9			69MiB	2 weeks ago	12 minutes ago
...

Next lets extract the image locally
rkt image export quay.io/coreos/etcd:v3.2.9 etcd.aci

# You can also run the below to see the manifest
rkt image cat-manifest quay.io/coreos/etcd:v3.2.9  

Now, lets create a tmp directory and extract the etcd image.
mkdir etcd_tmp
cd etcd_tmp
tar xvf ../etcd.aci

Now modify the etcd_tmp/manifest file
Replace all the quay.io/etcd/ with rkt.devtech101.com (approximately 3 times), for example.
# From
"name": "quay.io/coreos/etcd"

# To
"name": "rkt.devtech101.com/etcd"

Now lets re-package and sign the aci.
# Create aci
actool build /var/tmp/rkt_aci/etcd_tmp /var/tmp/rkt_aci/v3.2.9-linux-amd64.aci

# Sign aci
gpg --no-default-keyring --armor \
--secret-keyring ./rkt.sec --keyring ./rkt.pub \
--output v3.2.9-linux-amd64.aci.asc \
--detach-sig v3.2.9-linux-amd64.aci

Next, lets update the index.html with the etcd location.
 
    
   
   
   
   
  


We are almost done, lets create the etcd directory and copy the files.
# In the web server root
mkdir etcd
cp v3.2.9-linux-amd64.aci.asc v3.2.9-linux-amd64.aci etcd/

Finally, lets fetch the image from/to the local repo
pubkey: prefix: "rkt.devtech101.com/etcd"
key: "https://rkt.devtech101.com/pubkeys.gpg"
gpg key fingerprint is: 9E90 CD8C A814 88F2 86CB  6683 9823 67D1 E8F7 6CF7
    Subkey fingerprint: 3645 CCAA 8232 C62A B52E  89FF 63EB 0CAD 7333 2C89
	Eli (ACI signing key) 
Are you sure you want to trust this key (yes/no)?
yes
Trusting "https://rkt.devtech101.com/pubkeys.gpg" for prefix "rkt.devtech101.com/etcd" after fingerprint review.
Added key for prefix "rkt.devtech101.com/etcd" at "/etc/rkt/trustedkeys/prefix.d/rkt.devtech101.com/etcd/9e90cd8ca81488f286cb6683982367d1e8f76cf7"
10.10.10.10 - - [11/Jan/2018 15:46:12] "GET /etcd/v3.2.9-linux-amd64.aci.asc HTTP/1.1" 200 -
Downloading signature: [=======================================] 455 B/455 B
10.10.10.10 - - [11/Jan/2018 15:46:12] "GET /etcd/v3.2.9-linux-amd64.aci HTTP/1.1" 200 -
Downloading ACI: [=============================================] 12 MB/12 MB  
image: signature verified:
  Eli (ACI signing key) 
sha512-ffa46f2f338b0182770fb9a3e7f03108

Below you can see both the coreos(quay.io) and the local copy.
rkt image list
ID			NAME					SIZE	IMPORT TIME	LAST USED
sha512-c8c21115c820	coreos.com/rkt/stage1-host:1.21.0	94 MiB	3 hours ago	3 hours ago
sha512-af012352c315	rkt.devtech101.com/app1:v0.0.1		5.5 MiB	1 hour ago	1 hour ago
sha512-578ee83955c9	quay.io/coreos/etcd:v3.2.9		63 MiB	15 minutes ago	11 minutes ago
sha512-ffa46f2f338b	rkt.devtech101.com/etcd:v3.2.9		69 MiB	5 seconds ago	4 seconds ago

Helpful link
Rkt image layout

This concludes the Rkt/Appc application discovery/registry setup, the options are endless once you host your own application registry.

You might also like - Articles related to Docker Kubernetes / Micro-services.

Like what you're reading? please provide feedback, any feedback is appreciated.