What to configure if Docker build or the Kitchen-Docker driver is behind a Corporate Proxy.
While working with Docker builds and more specific Kitchen(using the Docker driver, not the default vagrant) behind a Corporate Proxy, I wasn’t able to complete a successful docker/kitchen image build.
The problem was clearly related to Docker (or Kitchen) build process – not being able to reach the outside world i.e. the Ubuntu, docker, etc.. related network repository.
The article below address options that worked for me, how to configure your proxy/firewall, which will then enable to build your own images by using Docker with a DockerFile or Chefs Kitchen Docker-Driver with a kitchen.yml configuration file.
Note: All the configurations below were tested using the latest Ubuntu 17.04 and Docker version 17.03.1-ce.
Ubuntu 17.04 Firewall, Iptables, ufw
Before we start with any Docker (or kitchen) configurations, lets adjust or disable the Ubuntu firewall.
First lets verify the current active firewall, this can be running the below.
sudo iptables -L -n -v
If using ufw, you might wont to disabled that, by running the below.
sudo ufw disable # or sudo systemctl stop ufw.service sudo systemctl disable ufw.service
And if using iptables, run the below.
sudo iptables -X sudo iptables -t nat -F sudo iptables -t nat -X sudo iptables -t mangle -F sudo iptables -t mangle -X sudo iptables -P INPUT ACCEPT sudo iptables -P FORWARD ACCEPT sudo iptables -P OUTPUT ACCEPT # Verify your changes. sudo iptables -L -n -v
Setting the basic http proxy
First, lets set the simplest proxy setting(used in many instances), the shell environment variable http_proxy, https_proxy and no_proxy.
Note: Make sure to set the no_proxy as well, as it can cause issues in some instances if not set.
http_proxy=http://your-proxy.domain.com:3172/ https_proxy=http://your-proxy.domain.com:3172/ no_proxy=you-host-name,you-host-name.domain.com,127.0.0.0/8,127.0.1.1,127.0.1.1*,[other-ips*] export http_proxy https_proxy no_prox
Note: A good option might be to add the above in a place ware it gets sourced when you login, like your .profile, .bashrc or even in /etc/profile.
Anther place to set your proxy and good practice, is in the /etc/apt/apt.conf, mainly used in the apt application, like the below.
cat /etc/apt/apt.conf Acquire::http::Proxy "http://your-proxy.domain.com:3172"; Acquire::https::Proxy "http://your-proxy.domain.com:3172";
Working with the Docker Daemon
Now ,that we enabled system related outgoing/incoming proxy configurations, lets move on and configure the Docker Daemon.
Check and make sure the DOCKER_OPTS are set properly
First lets set the Docker dns and the ip-masq, you can use something like the below (which uses the Google DNS servers)
Modify the settings in the /etc/default/docker configuration file
DOCKER_OPTS="--dns 220.127.116.11 --dns 18.104.22.168 --ip-masq=true"
While there make sure to also configure your http and https proxy, like the below.
export http_proxy="http://your-proxy.domain.com:3172/" export https_proxy="https://your-proxy.domain.com:3172/"
Note: This is one of the places you can set you proxy being used in docker.
You will also need to reload and restart the docker service, by doing this.
systemctl daemon-reload /etc/init.d/docker restart
Add your proxy to the Docker service
Adding your proxy to the systemd Docker service startup, is a good idea. this might not work/help in some instances (I believe there are a number of bugs related to this), but in any case a good idea doing so.
Adding the below to /etc/systemd/system/docker.service.d/http-proxy.conf will add this to the systemd configuration files.
Note: Since some programs use HTTP_PROXY all CAPS, you might wont to add that too.
[Service] Environment="HTTP_PROXY=http://your-proxy.domain.com:3172/" Environment="HTTPS_PROXY=http://your-proxy.domain.com:3172/" Environment="http_proxy=http://your-proxy.domain.com:3172/" Environment="https_proxy=http://your-proxy.domain.com:3172/" Environment="NO_PROXY=localhost,127.0.0.1"
You might also need to reload and restart, to take effect.
systemctl daemon-reload /etc/init.d/docker restart
Docker build Proxy configuration
Next, lets move to the Docker build process, which requires external access, especially when used with apt install curl, etc..
Note: Below only address the Docker build process in relation to the proxy options, I will have a separate article on all the other Docker build related options.
First, lets pull a Docker SSH image, this will be used in most of the testing.
docker pull rastasheep/ubuntu-sshd
Now, lets discuss the Docker proxy configuration options.
Make sure to set a variable something like to below, is anyway a good practice to have that.
http_proxy=http://your-proxy.domain.com:3172/ https_proxy=http://your-proxy.domain.com:3172/ no_proxy=http://127.0.0.1/
Create a Dockerfile, like the one below.
FROM rastasheep/ubuntu-sshd MAINTAINER Eli (email@example.com) RUN apt-get update && apt-get install -y curl vim
Then lets create a new alias called docker_build like the one below.
alias docker_build="docker build \ --build-arg http_proxy=$http_proxy \ --build-arg https_proxy=$https_proxy \ --build-arg no_proxy=$no_proxy \ --build-arg HTTP_PROXY=$http_proxy \ --build-arg HTTPS_PROXY=$https_proxy \ --build-arg NO_PROXY=$no_proxy "
Finally, lets use that for the docker build process, like the one below.
docker_build -t myimage1:latest .
Very similar to option 1, but instead we will modify the Dockerfile to set a proxy, we will also use the actual docker instead of the docker_build alias we created above.
Modify your Dockerfile to look like the one below.
FROM rastasheep/ubuntu-sshd MAINTAINER Eli (firstname.lastname@example.org) ENV http_proxy=http://your-proxy.domain.com:3172/ ENV https_proxy=http://your-proxy.domain.com:3172/ RUN apt-get update && apt-get install -y curl vim RUN echo root:newpasswd | chpasswd
You should see something like the below, apt install should completed correctly without any issues.
docker build --rm -t myimage2:latest . Sending build context to Docker daemon 2.048 kB Step 1/5 : FROM rastasheep/ubuntu-sshd ---> c11d694026ce Step 2/5 : MAINTAINER Eli (email@example.com) ---> Using cache ---> da769fb41eff Step 3/5 : ENV http_proxy http://your-proxy.domain.com:3172/ [...] snip Get:1 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB] Get:2 http://archive.ubuntu.com trusty-security InRelease [65.9 kB] Hit http://archive.ubuntu.com trusty Release.gpg [...] snip
Now to use the image just run, something like the below.
docker run -d --name test_sshd1 -p 2022:22 myimage2 ssh -p 2022 localhost ...
Now that Docker is out of the way, lets begin working on the Kitchen docker-driver
Chef Kitchen / Kitchen-test Docker driver Proxy Configuration Options
Lets create our first kitchen configuration directory.
mkdir mykitchen1 cd mykitchen1
Now, lets initialize docker by using kitchen.
Note: Make sure to specify the Kitchen driver name as docker, because the default kitchen driver will configure/use Vagrant Virtual Box which is a full VM.
kitchen init --driver=kitchen-docker --create-gemfile create .kitchen.yml create chefignore create test/integration/default create Gemfile append Gemfile append Gemfile You must run `bundle install' to fetch any new gems.
Note: There are many kitchen driver you can use like AWS/EC2, GCP, etc.. to get a full list of drivers, just run kitchen driver discover
Next, lets run bundle install, you should see something like the below.
Note: This will also generate an initial .kitchen.yml
bundle install Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. Fetching gem metadata from https://rubygems.org/......... Fetching version metadata from https://rubygems.org/.. Fetching dependency metadata from https://rubygems.org/. Resolving dependencies... Using artifactory 2.8.1 Using mixlib-shellout 2.2.7 Using mixlib-versioning 1.1.0 : look in the genarted .kitchen.yml
Kitchen Configuration options
Now comes the fun part.
Below is the .kitchen.yml with the proxy configuration already set, make sure to modify yours the same way.
Note: This only address the Kitchen/Docker build process in relation to the proxy options, I will have a separate article on all the other Kitchen automation related options.
--- driver: name: docker provisioner: name: chef_solo platforms: - name: ubuntu-14.04 - name: centos-7.2 driver_config: memory: 20m http_proxy: http://your-proxy.domain.com:3172 https_proxy: http://your-proxy.domain.com:3172 provision_command: - echo "root:password" | chpasswd - sed -i 's/Defaults requiretty/#Defaults requiretty/g' /etc/sudoers - sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/g' /etc/ssh/sshd_config suites: - name: default run_list: attributes:
If you look closely on on the configuration above, you can see the section driver_config:.
To set your proxy all you need to do is add the driver_config: with http_proxy and https_proxy set.
Note: A quick note on Kitchen documentation. I had challenges getting to work the examples in the Chef kitchen yml documentation. but had a much better experience in getting it to work with this documentation.
For Solaris specific notes check this out – Solaris chef kitchen docker quick notes
You might also like Managing Docker On Ubuntu 17.04 Using Rancher Or Portainer.
What was you experience and challenges with Kitchen, Docker behind a Corporate Proxy or Firewall? Please let me know in the comments below.