Running a Private Docker Registry - Céondo Technology Notes

Created: Sat 15 November 2014 / Last updated: Thu 08 January 2015

What is a Docker Registry?

A Docker registry is a repository of Docker images. It is a bit like an APT repository, but instead of storing applications, it is storing full Docker images. When you pull an image with Docker, you retrieve an image from a registry, usually the main Docker registry.

Why Our Own Docker Registry?

Sometimes it is nice to share, sometimes it is better to keep your code private. A simple case is that you have production ready images which include creditentials to access remote services. You can build these images using public images as base, but you do not want to share your production images outside of your company.

The present step-by-step tutorial is using Ubuntu LTS 14.04 as base Linux distribution. The registry will not run inside a Docker container. Our approach is to run the registry on a virtual server hosted outside of our infrastructure. We have very good experience with Gandi where even a small virtual machine is good enough to run the registry but also the company website and services. Some of the steps will be specific to Gandi but can be easily translated to any other provider like AWS, DigitalOcean, Rackspace, etc. as it is basically a bare metal installation.

Base Ubuntu LTS Installation

You will see that the installation is really easy as nearly everything is well packaged. So, lets go and as root enjoy the installation of a lot of packages:

apt-get update
apt-get install build-essential python-dev libevent-dev python-pip
apt-get install liblzma-dev nginx nginx-extras apache2-utils
pip install docker-registry

Notice that the docker registry is installed using pip, this is to be sure to get the latest stable. This is important to follow the development of Docker itself.

Now, you have already installed all the needed software to run your private registry, it is time to configure it and setup the startup scripts.

Configuration of the Registry

The registry is pretty simple to configure, for our Gandi VM, we configured it with two drives, one system drive with only 5GB and one data drive with 25GB available as /srv/datadisk01. If you do not have two drives like that, you can create a /home/data folder and use it where /srv/datadisk01 is used in the rest of this note.

Create the data folder where the registry data will be put:

mkdir -p /srv/datadisk01/docker-registry/data
chown www-data:www-data /srv/datadisk01/docker-registry/data

a place to put the configuration of the registry:

mkdir -p /etc/docker

and finally a place to put the registry log files:

mkdir -p /var/log/docker-registry
chown www-data:www-data /var/log/docker-registry

The configuration of the registry for a simple use like ours is really simple, simply create the file /etc/docker/config.yml with the following content:

# All other flavors inherit the `common' config snippet
common: &common
    issue: 'docker-registry server'
    # Default log level is info
    loglevel: _env:LOGLEVEL:info
    # Enable debugging (additional informations in the output of the _ping endpoint)
    debug: _env:DEBUG:false
    # By default, the registry acts standalone (eg: doesn't query the index)
    standalone: _env:STANDALONE:true
    search_backend: sqlalchemy
    sqlalchemy_index_database: sqlite:////srv/datadisk01/docker-registry/docker-registry.db

local: &local
    <<: *common
    storage: local
    storage_path: /srv/datadisk01/docker-registry/data

This file must be readable by the www-data user.

We know have the registry configured to store the data on disk, you can also use Amazon S3 for the storage, but it will also force you to run a cache for the small files and metadata. The cache is using Redis.

Startup Script for the Registry

The upstart startup script for the registry is pretty simple, what is really important is to inform which flavor of the registry we run, here local and where is the configuration file. This is done through environment variables. So, create the /etc/init/docker-registry.conf file with this content:

description "Docker Registry"
version "0.9.0"
author "Docker, Inc."

start on runlevel [2345]
stop on runlevel [016]

respawn
respawn limit 10 5

# set environment variables
env REGISTRY_HOME=/srv/datadisk01/docker-registry
env SETTINGS_FLAVOR=local
env DOCKER_REGISTRY_CONFIG=/etc/docker/config.yml

setuid www-data

script
cd $REGISTRY_HOME
exec gunicorn -k gevent --max-requests 100 --graceful-timeout 3600 -t 3600 -b 127.0.0.1:5000 -w 2 --access-logfile /var/log/docker-registry/access.log --error-logfile /var/log/docker-registry/server.log docker_registry.wsgi:application
end script

The exec is pretty long, but basically it is launching 2 worker processes and putting the logs at the right place.

You can now start the registry:

service docker-registry start

A good thing to do is to check the output of the error log:

tail /var/log/docker-registry/server.log

You should not have any errors, if you have some, they are pretty explicit and will help you fixing the problems.

Now, your registry is configured but will only be available on the current server on 127.0.0.1:5000 but what you want is the ability to access it from registry.yourcompany.com or similar. For this, you need to install Nginx as front end with authentication, to only allow access to the registry to your coworkers.

Configuration of Nginx

The configuration of Nginx is also pretty standard and simple, the only particular case is that the registry requires you to run it over SSL. This is really good for security, but you will need a certificate.

Following the instructions of your SSL certificate provider, you must end up with a certificate, a file ending with .crt, for example certificate-registry.yourcompany.com.crt. In our case, we have a wildcard certificate for *.ceondo.com. And a private key, maybe registry.yourcompany.com.key. If your provider as an intermediate certificate, you will have to concatenate it at the end of your certificate to merge them. At the end you get a merged-registry.yourcompany.com.crt looking like that:

So, copy the merged-registry.yourcompany.com.crt as /etc/ssl/registry.yourcompany.com.crt and registry.yourcompany.com.key as /etc/ssl/registry.yourcompany.com.key.

Now, you can create the registry virtual host configuration file as /etc/nginx/sites-available/docker-registry:

upstream docker-registry {
 server 127.0.0.1:5000;
}

server {
 listen 443 ssl;
 server_name registry.yourcompany.com;

 ssl on;
 ssl_certificate /etc/ssl/registry.yourcompany.com.crt;
 ssl_certificate_key /etc/ssl/registry.yourcompany.com.key;

 proxy_set_header Host       $http_host;   # required for Docker client sake
 proxy_set_header X-Real-IP  $remote_addr; # pass on real client IP

 client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

 # required to avoid HTTP 411: see Issue #1486 (https://github.com/dotcloud/docker/issues/1486)
 chunked_transfer_encoding on;

 location / {
     # let Nginx know about our auth file
     auth_basic              "Restricted Docker Registry";
     auth_basic_user_file    /etc/nginx/docker-registry.htpasswd;
     proxy_pass http://docker-registry;
 }
 location /_ping {
     auth_basic off;
     proxy_pass http://docker-registry;
 }  
 location /v1/_ping {
     auth_basic off;
     proxy_pass http://docker-registry;
 }

}

Then you need enable the site, it will be enabled at restart:

ln -s /etc/nginx/sites-available/docker-registry /etc/nginx/sites-enabled/docker-registry

Setup the Access Rights with the htpasswd File

You have seen in the configuration file that the basic authentication is enabled, so you want to create a password file, here /etc/nginx/docker-registry.htpasswd with a login and a password for you to use. This is done with the htpasswd utility:

htpasswd -c /etc/nginx/docker-registry.htpasswd USERNAME

Replace USERNAME with your username and provide the password you want. You can then test that everything is ok by running:

htpasswd -v /etc/nginx/docker-registry.htpasswd USERNAME

it will ask for the password of the user USERNAME. If you want to add another user, do not put the -c flag or it will overwrite the current file:

htpasswd /etc/nginx/docker-registry.htpasswd OTHERUSER

You can restart Nginx:

service nginx restart

Time to Test your Installation

Now, all the commands are from your laptop/personal computer as the server is running nicely far away from you.

docker login https://registry.yourcompany.com

provide your username and password, the same you provided when creating the htpasswd file. It will normally also ask your email address and at the end you will get something like:

Login Succeeded

if not, open https://registry.yourcompany.com with your browser and ensure that you can access it with your login and password. You should see:

"docker-registry server"

as output. If not, take a look at the registry logs in /var/log/docker-registry and the Nginx logs in /var/log/nginx.

Now, we are going to pull a small image from the main public registry, change it, create a new image out of it and upload it to our own private registry. The image is official Debian image as it is less than 100MB in size:

docker run -t -i debian:stable /bin/bash
root@aefb1234:/# touch /SUCCESS
root@aefb1234:/# exit

Here we started the debian:stable image, created the empty /SUCCESS file and exited. We can now create the test-debian-stable image out of it:

docker commit $(docker ps -lq) test-debian-stable

and to push it to our own registry, we need first to tag it with our registry path:

docker tag test-debian-stable registry.yourcompany.com/test-debian-stable

and push it:

docker push registry.yourcompany.com/test-debian-stable

You uploaded your first image to your registry. From another computer with Docker installed your run:

docker login registry.yourcompany.com
docker pull registry.yourcompany.com/test-debian-stable
docker run -t -i registry.yourcompany.com/test-debian-stable /bin/bash
root@12abcaf86:/# ls /SUCESSS
/SUCCESS

Congratulations, you can now enjoy your private registry!

You can even search your registry using your web browser, access https://registry.yourcompany.com/v1/search?q=test and you should get as answer something like:

{"num_results": 1, "query": "test",
 "results": [{"description": "", "name": "library/test-debian-stable"}]}

References

Changelog

Thu Jan 8, 2015: correctly use the www-data user to run Gunicorn.