docker-arm-chip.jpg

Making your Docker images ARM compatible for Raspberry Pi

The big promise of Docker and containers in general; portability. A Docker container runs on a system that supports the Docker container's runtime environment. Applications are no longer tied to the host its operating system, that's great! But, it still needs to be compiled for the host its system architecture.

But how does that affect our Docker images? Does it require a lot of changes? And why on earth would you even want to compile Docker images for ARM! Nobody runs Docker on ARM devices, right?

raspberrypi3bplus.jpg

Think again! Some people, like myself, do use Docker in production on devices like a Raspberry Pi! In fact, this very website you're on right now is running in Docker on my Raspberry Pi at home. It used to run with some of my other personal projects on one of Digital Ocean cheapest droplets ($5/month), but why pay $5 a month for something while you can achieve the same thing for free?

So I plugged in a Raspberry Pi that I still had laying around in the back of my router (usb-powered, no external power source, super handy!) and got to work. Turns out it's only a couple of changes!

Example project

Let's do the changes with an example project. The website we'll use as example, ikhaat.be, has a pretty basic setup that consists out of 2 containers; an nginx service & a php-fpm service.

screenshot-ikhaat.be.jpg

nginx

Starting with nginx, the Dockerfile currently looks like this.

FROM nginx:1.15-alpine
LABEL maintainer="Wouter De Schuyter <[email protected]>"

# Copy config files
COPY ./.docker/nginx/default.conf /etc/nginx/conf.d/default.conf

# Copy project
COPY ./public /code/public

WORKDIR /code

In this case the only thing we really need to change is to extend from an ARM compatible variant of the nginx image, e.g. arm32v6/nginx:1.15-alpine. And that's it for the nginx service, DONE 😎!

php-fpm

Our php-fpm service is slightly more complex as we need to install/build a couple of dependencies and extensions here, the Dockerfile currently looks like this.

FROM php:7.3-fpm-alpine
LABEL maintainer="Wouter De Schuyter <[email protected]>"

# Install extensions
RUN apk add --no-cache \
      freetype-dev libpng-dev libjpeg-turbo-dev \
      freetype libpng libjpeg \
    && docker-php-ext-configure gd \
      --with-freetype-dir=/usr/include/ \
      --with-png-dir=/usr/include/ \
      --with-jpeg-dir=/usr/include/ \
    && NPROC=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || 1) \
    && docker-php-ext-install -j${NPROC} gd \
    && apk del freetype-dev libpng-dev libjpeg-turbo-dev

# Copy project
COPY ./public/index.php /code/public/index.php
COPY ./src /code/src
COPY ./resources /code/resources
COPY ./vendor /code/vendor

WORKDIR /code

Let's see if that's as easy as nginx and we can just extend the image from an arm32v6 variant. Change the base image to arm32v6/php:7.3-fpm-alpine and try to build. Yay, looks like it works 🤩!

Great success?

Yes and no. If you're running macOS like I am, it looks like we can now build both our ARM compatible images. And in fact we can, they will work if deployed on an ARM system. But then I saw the little red cross in the commit history, followed by that email from CircleCI that my build had failed 😢..

Screen Shot 2019-02-12 at 22.20.56.jpg

The logs

In the logs we can see it crashes at standard_init_linux.go:185: exec user process caused "exec format error". A similar error you would see when trying to run non ARM compatible images on ARM systems. It turns out that CircleCI can't build our php-fpm service by just changing the base image.

Step 1/8 : FROM arm32v6/php:7.3-fpm-alpine
7.3-fpm-alpine: Pulling from arm32v6/php
Digest: sha256:228b6a24152e359ee4ba0e58f5cc72071aa11d4e6eca80563126f8a45367fa87
Status: Downloaded newer image for arm32v6/php:7.3-fpm-alpine
 ---> adc10e09db98
Step 2/8 : LABEL maintainer "Wouter De Schuyter <[email protected]>"
 ---> Running in 2508bbbc4924
 ---> 66b1cb2a5405
Removing intermediate container 2508bbbc4924
Step 3/8 : RUN apk add --no-cache       freetype-dev libpng-dev libjpeg-turbo-dev       freetype libpng libjpeg     && docker-php-ext-configure gd       --with-freetype-dir=/usr/include/       --with-png-dir=/usr/include/       --with-jpeg-dir=/usr/include/     && NPROC=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || 1)     && docker-php-ext-install -j${NPROC} gd     && apk del freetype-dev libpng-dev libjpeg-turbo-dev
 ---> Running in 1bc9695d0271
standard_init_linux.go:185: exec user process caused "exec format error"
The command '/bin/sh -c apk add --no-cache       freetype-dev libpng-dev libjpeg-turbo-dev       freetype libpng libjpeg     && docker-php-ext-configure gd       --with-freetype-dir=/usr/include/       --with-png-dir=/usr/include/       --with-jpeg-dir=/usr/include/     && NPROC=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || 1)     && docker-php-ext-install -j${NPROC} gd     && apk del freetype-dev libpng-dev libjpeg-turbo-dev' returned a non-zero code: 1
make: *** [.build-php-fpm] Error 1
Exited with code 2

The fix

I can't explain why it works on macOS but doesn't work on Linux (CircleCI), I don't know the technicals behind it. But I do know what's the issue and how to fix it. To crosscompile an ARM image on an x86 system we'll need to emulate an ARM environment (normally, again: I'm not sure why it works out of the box on macOS). This can be done quite easily thanks to guys of multiarch/qemu-user-static!

One of the things you need to do is to run the following command before you start building.

⚠️ Note: this needs to be done only once, but because of how CircleCI works you will need to put the command somewhere in your build script so it runs during every job before the building phase.

docker run --rm --privileged multiarch/qemu-user-static:register --reset

And the other thing is to copy the qemu-arm-static binary to /usr/bin into your php-fpm image at the very beginning (you will also need to download it & set permissions on it, outside of the Dockerfile).

And that's it basically it! Just for reference I created a PR so you can easily go through the diff and see clearly what changed exactly. Go check it out here: https://github.com/wouterds/ikhaat.be/pull/22.