Save bandwidth and time with Docker pulls

I’m lucky - my home office is served by Webpass. Network connectivity and speed is almost never an issue. I pay little attention to my download habits, unless the server is slow…

The case is significantly different, though, when I’m on the road. Long BART rides (or plane rides) provide great opportunity to focus on a single task and make serious progress on a project. On a recent trip while tethered to my phone, I created a 3-host swarm with docker-machine, and then pulled down several images with docker pull (by default in a docker swarm, each host will run the docker pull, so I was repeatedly downloading the same image). After a few minutes later, my phone chirped with an alert from Verizon: I was suddenly at 95% of my monthly data budget.

Whoops.

This is also handy and time-saving when you’re doing dev work - building machines, tearing them down, re-deploying. If the image is in the next VM on your laptop, things spin up much faster on a fresh docker engine with no images.

As I expect to be traveling more this year, and I also expect to be doing more work with containers, this isn’t going to cut it. I already use the awesome vagrant-cachier plugin to cache OS repositories outside of vagrant VMs. Now I needed similar for container images.

Matt Bently over at Docker wrote a blog post showing how to set up a registry proxy cache - that helped me quickly get things up. But I like a little more reproducability so I could easily share this with coworkers and others. So I coded this into docker-registry-cache - now available on GitHub.

How does this work?

In a nutshell, a caching proxy sits inline between your docker engines, and the registry they connect to to pull container images. Before attempting to pull an image, the cache first checks to see if it has already downloaded that version of the image. If so, it serves out that cached version of the image, instead of downloading it over the Internet from Docker’s registry.

It’s pretty simple to use, actually. Clone the git repo git@github.com:jlk/docker-registry-cache.git, cd into that directory, and run “vagrant up.” Presuming you have Vagrant and VirtualBox installed, vagrant will look for the ubuntu-trusty64-docker vagrant box image locally, probably not find it, and then download it. Once it’s downloaded that box will be used to create a new VM, and then a provisioning script will download and run version 2 of the Docker registry container.

If you’re not familiar with Vagrant, there’s an easy tutorial at https://www.vagrantup.com/docs/getting-started/.

Once the VM and containers are running, the registry cache is listening by default at http://192.168.99.50:5000. Point your docker engines at this, and you’ll see significant time and bandwidth savings on repeated docker pulls. As you pull different images, checking http://192.168.99.50:5000/v2/_catalog will display what images the registry is caching.

More details are available in the README.md at GitHub. Would love to hear any use stories, suggestions for improvement, etc. I’d like to expand the cache beyond just Docker Registry - will see what I can do with CoreOS’s Quay in coming weeks…