Installing and managing Docker by Using the Shipyard API’s

There are many options to manage Docker. Below I will show you how to use Shipyard to manage your Docker environment One of the great things about Shipyard is the web Interface, while that is true(and I will show that too). but this article is mainly focused on the befits and strength of using the API’s .

Installing Shipyard

First we need to install Shipyard. The process is rather simple, you can do it manually (which I will go through below), or full automated.

Installing Shipyard – fully automated

To quickly get up and running, install Shipyard fully automated, just run the below. Note: There are a few options you can specify like TLS,etc.., for full details, please check the shipyard website

curl -sSL https://shipyard-project.com/deploy | bash -s

Installing Shipyard – manual method

Note: If you installed Shipyard the fully automated way, you can just skip this section. The process below installs Shipyard in to your regular Docker (Swarm) environment. Note: The process is almost identical to the process on the shipyard web site, with a few miner modifications which I will outline below. The below docker run will install the Datastore, this is the database (Rethink DB) which stores all Shipyard activity.

docker run \
    -ti \
    -d \
    --restart=always \
    --name shipyard-rethinkdb \
    -p 49153:8080 \
    rethinkdb

Note: I added the port 49153 with the line -p 49153:8080, to expose the Rethink DB port. The reason for this was, to be able to explore the Rethink DB web UI, screen captures are below. Shipyard RethinkDB – Main Screen Shipyard Server Overview Next, we need to install the Discovery to enable Swarm leader elections.

docker run \
    -ti \
    -d \
    -p 4001:4001 \
    -p 7001:7001 \
    --restart=always \
    --name shipyard-discovery \
    microbox/etcd -name discovery

You might also need to install the Proxy, this is a lightweight container that simply forwards requests from TCP to the Unix socket that Docker listens on.

docker run \
    -ti \
    -d \
    -p 2375:2375 \
    --hostname=docker-api \
    --restart=always \
    --name shipyard-proxy \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -e PORT=2375 \
    shipyard/docker-proxy:latest

Next, we need to install the Swarm Manager and Swarm agent.

# Swarm Manager
docker run \
    -ti \
    -d \
    --restart=always \
    --name shipyard-swarm-manager \
    swarm:latest \
    manage --host tcp://0.0.0.0:3375 etcd://10.10.10.11:4001

# Swarm Agent
docker run \
    -ti \
    -d \
    --restart=always \
    --name shipyard-swarm-agent \
    swarm:latest \
    join --addr 10.10.10.11:2375 etcd://10.10.10.11:4001

Finally, we need to install the Shipyard controller.

docker run \
    -ti \
    -d \
    --restart=always \
    --name shipyard-controller \
    --link shipyard-rethinkdb:rethinkdb \
    --link shipyard-swarm-manager:swarm \
    -p 8080:8080 \
    shipyard/shipyard:latest \
    server \
    -d tcp://swarm:3375

Once completed we can now test to make sure shipyard is up and running, by going to.

# shipyard
http://docker-api.domain.com:8080/

# shipyard db
http://docker-api.domain.com:49153/

The default login is admin and password is shipyard, please make sure to change the password once you login.

Managing Docker With Shipyard

Once Shipyard is up and running, it will auto-discover all your Docker instances, images, etc… You can manage most of your Docker environment form with-in the Web UI, or by using the API’s. Note: Shipyard also has the option to be managed with the Shipyard CLI, by connecting to the Shipyard-Docker container.

Managing By Using the Web UI

To manager Docker with the shipyard web UI, just login to the web UI(in our case by going to port 8080). Using the Web UI is quite simple, I will show a few examples below. Shipyard login screen Shipyard Deploy – Options Shipyard Container Stats Shipyard Pulling Images

Managing Shipyard By Using the API’s

Shipyard has a simple API which is great to manage docker. The example below uses Curl to create / add a new account to the system, by using the Shipyard API’s. Below is a simple shell script to add a new account. Note: Make sure to change the account names to your desire.

#!/bin/bash

userid="elik"
firstname="Eli"
lastname="Kleinman"
password="password"

# Get Auth token
token=`curl -s -H "content-Type: application/json" -X POST -d \
'{"username": "admin", "password": "shipyard"}' http://localhost:8080/auth/login | jq -r [.auth_token][0]`

curl -H "X-Access-Token: admin:$token" -X POST -d \
'{"first_name": "$firstname", "last_name": "$lastname", "username": "$userid", "password": "$password", "roles": ["Administrator"]}' \
http://localhost:8080/api/accounts

The next example lists all system accounts(including the newly created account). Just added the below code to end of the above script.

curl -s -H "X-Access-Token: admin:$token" http://localhost:8080/api/accounts

To get a list of all roles. Just added the below code to end of the above script.

curl -s -H "X-Access-Token: admin:$token" http://localhost:8080/api/roles

Shipyard API – Python example The example below, uses Python to list all accounts in the system. Note: Replace username and password, with the account used to authenticate to the system.

#!/usr/bin/python

import requests
from requests.auth import HTTPDigestAuth
import sys, json, os
import yaml

username = 'elik'
password = 'password'
ContentType = 'application/json'

# Remove proxy if set (as it might block reqest)
if "http_proxy" in os.environ:
    del os.environ['http_proxy']
if "https_proxy" in os.environ:
    del os.environ['https_proxy']

def getReqType( reqType ):
     if reqType == "login":
        reqType = 'http://localhost:8080/auth/login'
     elif reqType == "roles":
        reqType = 'http://localhost:8080/api/roles'
     elif reqType == "accounts":
        reqType = 'http://localhost:8080/api/accounts'
     return reqType

# Set values
headers                 = {}
authdata                = {}
headers['Content-Type'] = ContentType
authdata['username']    = username
authdata['password']    = password
url                     = getReqType("login")
reqType                 = "accounts"

r = requests.post(url, json=authdata,
                              headers=headers)
auth_token = yaml.load(r.text)
url = getReqType(reqType)
headers['X-Access-Token'] = username + ":" + auth_token['auth_token']
response = requests.get(url, json=authdata,
                              headers=headers)
data = json.loads(response.text)
print json.dumps(data, sort_keys=True,
                 indent=4 )

To list all roles in the system, just replace the code on line 34, like the below.

# From
reqType                 = "accounts"
# To
reqType                 = "roles"

Example of the Python script output of account listing.

./api.py 
[
    {
        "first_name": "Abc", 
        "id": "19f3961f-2044-4576-8b46-e3e3739caecd", 
        "last_name": "Abc", 
        "password": "$2a$10$0dJ6FkiVfAcFI2tUwzz9ouicts9GLx62CIN19I11iuJRwu26zyIy6", 
        "roles": [
            "admin"
        ], 
        "username": "abc"
    }, 
    {
        "first_name": "Shipyard", 
        "id": "86708856-2a41-425e-8b7c-0a1530f8e37a", 
        "last_name": "Admin", 
        "password": "$2a$10$6QI3hEB7jUTWPHy8vOlbkuZj5w5y3E0lupOl/dY/vgVetlbTtg7Mq", 
        "roles": [
            "admin"
        ], 
        "username": "admin"
    }, 
    {
        "first_name": "Eli", 
        "id": "5fefbe3b-d7a5-48f6-a783-1f90ae3933d5", 
        "last_name": "Kleinman", 
        "password": "$2a$10$g13NUV2ZUpG23LIJHa4PfOI583Kp7i5RP2Lxe/O8blhO6zIzgk35S", 
        "roles": [
            "admin"
        ], 
        "username": "elik"
    }
]

Note: The Python script can easily be extended with additional functionality, helping in fully automate your Docker DevOps life-cycle. In the Shipyard UI, you can also see the elik account created. this was created by running the Python script (using the reset API’s). You can also see in the Web UI > Events. all API’s that ran in the past, for example the creation of the elik Account (screen capture below). In summery, using Shipyard is a great and simple way to manage Docker. however, there are a list of yet desirable / noticeable missing features. I will list some of them below.

1. Using a Dockerfile by API
2. Using a dokcer-compose.yml by API
3. Intergeneration with a cloud provider, like AWS, GCP, Azure, etc…

What tools/programs are you using to manage Docker API’s? You might also like Managing Docker Using Docker-Compose to Create Multi-Container Docker Applications Using ZFS For The Docker COW Storage Layer(s) In Ubuntu 17.04 Gotchas / Tips Creating Your Own Private Docker Registry With Self Signed Certificate Using Chef Kitchen / Docker Build Behind a Corporate Proxy or Firewall