How to Build a Docker Container on an Apple Silicon M-series Machine
TL;DR version (slow way) - example:
Impatient? Here:
Method 1: Use docker buildx build
QEMU way
docker buildx build --platform linux/amd64 --tag my-container:latest --load .
Details:
buildx build
tell Docker to build using BuildKit.buildx
appears in Docker version >= 19.03.--platform linux/amd64
- abuildx
option to set the target architecture of our build--tag my-container:latest
- The normal tag image option. Replace with your own value--load
- thebuildx
option will not automatically load the image into the cache (unlike normal build) - so if you plan on using the image you need this.
The directory context for docker. We are just using the current directory - replace as needed.
Example with extra options:
docker buildx build -f Dockerfile --platform linux/amd64 --tag my-container:latest --load .
Tag & Push?
Works the same.
docker tag my-container:latest myregistry.com/somewhere/my-container:latest
docker push
But’s It’s So Slow!
Yes it is. This is because buildx
command uses QEMU. The buildx Docker build system will start up QEMU, emulating a linux/amd64
(aka x86_64
) architecture. In this emulated mode it will run the entire build process. So running toolchains such as the go compiler will be slow.
Here is how to build images faster:
- Obviously - but we will say it anyway. Just build on the native architecture. Your CI / CD flow could (should) do this.
- Use a cross-build toolchain. This way the compiler runs native - without QEMU. This still uses
buildkit
Let’s talk about the later - for the sake of building on your laptop or workstation - but much faster.
Using --platform
to do fast native cross-compile builds
Method 2: Roll your own multi-arch Dockerfile - Golang example:
As you may know, Go’s compiler supports many architectures out of the box. For projects with straightforward build procedures, we can make some simple modifications to our Dockerfile and get a much faster build.
Let’s say your Golang project’s Dockerfile looked like this:
FROM golang:1.20
LABEL org.opencontainers.image.source=https://github.com/IzumaNetworks/trivial-golang-k8s-deployment
LABEL org.opencontainers.image.description="Trivial Go K8s example"
LABEL org.opencontainers.image.licenses=Apache-2.0
ADD app /app
ENV CGO_ENABLED=0
RUN cd /app && go build -o app .
WORKDIR /app
ENTRYPOINT [ "/app/app" ]
To make Docker tell Go to target a different arch (use it’s cross compiler toolchain):
FROM --platform=$BUILDPLATFORM golang:1.20 AS build
LABEL org.opencontainers.image.source=https://github.com/IzumaNetworks/trivial-golang-k8s-deployment
LABEL org.opencontainers.image.description="Trivial Go K8s example"
LABEL org.opencontainers.image.licenses=Apache-2.0
ADD app /app
# Automatic platform ARG variables produced by Docker
ARG TARGETOS TARGETARCH
# No CGO!
ENV CGO_ENABLED=0
WORKDIR /app
# Use the args to make Go build for the target arch
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o app .
ENTRYPOINT [ "/app/app" ]
(Example code is on Github.)
Now run the same build command:
docker buildx build -f Dockerfile --platform linux/amd64 --tag my-container:latest --load .
Orders of magnitude faster.
But what about Cgo?
In the above example, commenting out
# ENV CGO_ENABLED=0
Will result in an error such as:
#0 1.665 # runtime/cgo
#0 1.665 gcc: error: unrecognized command-line option '-m64'
------
Dockerfile-cross:9
--------------------
7 | ENV CGO_ENABLED=1
8 | WORKDIR /app
9 | >>> RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o app .
10 | ENTRYPOINT [ "/app/app" ]
11 |
--------------------
error: failed to solve: process "/bin/sh -c GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o app ." did not complete successfully: exit code: 1
--progress=plain
with Docker
Why? Well because the version of gcc
included in the golang
image that the Dockerfile is using for the build, lacks a gcc cross-compiler targeting x86-64.
Using a Cross Toolchain
We need to install the right cross-compile version of gcc
and then tell Go to use it:
FROM --platform=$BUILDPLATFORM golang:1.20 AS build
LABEL org.opencontainers.image.source=https://github.com/IzumaNetworks/trivial-golang-k8s-deployment
LABEL org.opencontainers.image.description="Trivial Go K8s example"
LABEL org.opencontainers.image.licenses=Apache-2.0
# get gcc cross toolchain
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install g++-x86-64-linux-gnu libc6-dev-amd64-cross
RUN DEBIAN_FRONTEND=noninteractive apt-get -y clean
ADD app /app
ARG TARGETOS TARGETARCH
ENV CGO_ENABLED=1
WORKDIR /app
# RUN gcc -dumpmachine with --progress=plain here to see what gcc is used by default
RUN CC=x86_64-linux-gnu-gcc GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o app .
ENTRYPOINT [ "/app/app" ]
Method 3: Dockerfile for multi-arch using xx
(Best way)
But… Our Dockerfile above is now specific to x86-64. This kind of sucks - for instance let’s say you want to build and run locally on your Apple Silicon Mac, and then you decide to run it on an Arm 64-bit instance on AWS. Or you just build it on an x86-64 machine where you don’t need the cross-compiler. You don’t want to maintain multiple Dockerfiles. What then?
Use the tonistiigi/xx tools. These are written by the same author who built much of the buildx
subsystem and who works at Docker. Docker’s own documentation suggests using them.
Using tonistiigi/xx
:
FROM --platform=$BUILDPLATFORM tonistiigi/xx AS xx
FROM --platform=$BUILDPLATFORM golang:1.20
LABEL org.opencontainers.image.source=https://github.com/IzumaNetworks/trivial-golang-k8s-deployment
LABEL org.opencontainers.image.description="Trivial Go K8s example"
LABEL org.opencontainers.image.licenses=Apache-2.0
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install build-essential
RUN DEBIAN_FRONTEND=noninteractive apt-get -y clean
COPY --from=xx / /
ADD app /app
ARG TARGETPLATFORM
RUN xx-apt install -y libc6-dev gcc
ENV CGO_ENABLED=1
WORKDIR /app
RUN xx-go --wrap
RUN go build -o app . && xx-verify app
ENTRYPOINT [ "/app/app" ]
Our Dockerfile is no longer glued to a specific architecture.
Once again, the same build command…
docker buildx build -f Dockerfile --platform linux/amd64 --tag my-container:latest --load .
Also, for reference, here is the same build but using Alpine as the underlying image:
FROM --platform=$BUILDPLATFORM tonistiigi/xx AS xx
FROM --platform=$BUILDPLATFORM golang:1.20-alpine
LABEL org.opencontainers.image.source=https://github.com/IzumaNetworks/trivial-golang-k8s-deployment
LABEL org.opencontainers.image.description="Trivial Go K8s example"
LABEL org.opencontainers.image.licenses=Apache-2.0
# get xx tools to build image
RUN apk add clang lld
COPY --from=xx / /
ADD app /app
ARG TARGETPLATFORM
RUN xx-apk add musl-dev gcc
ENV CGO_ENABLED=1
WORKDIR /app
# RUN gcc -dumpmachine
RUN xx-go --wrap
RUN go build -o app . && xx-verify app
ENTRYPOINT [ "/app/app" ]
What is going on with the xx-
tools?
The tools in the xx
image wrap common build tools, but with an understanding of Docker’s built-in BUILDPLATFORM
and other variables.
So when you run docker build buildx --platform=linux/amd64
the linux/amd64
value is understood by xx-go
, xx-gcc
and xx-apk
etc. These will then call the appropriate platform-specific command for your target architecture.
Conclusion: Use xx
When Possible
- Use the QEMU method for a one-off build or to use an old Dockerfile
- Use the manual cross toolchain method if you have a very specific build environment or needs
- Most of the time, just use the multi-arch method using the
xx-
tools fromtonistiigi/xx
Troubleshooting
What if the buildx
command is missing?
Docker Desktop
By default, the buildx
docker command is included with Docker Desktop. Any version past 19.03 should have it. If you have some other Docker installation or are not using MacOS you might need to install the buildx plugin:
docker plugin install buildx
Installation on Linux:
Download the latest docker buildx plugin from here:
https://github.com/docker/buildx/releases
And place it in ~/.docker/cli-plugins
Then verify the install.
Example:
# Download buildx
curl -L "https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-amd64" -o ~/.docker/cli-plugins/docker-buildx
# Make it executable
chmod a+x ~/.docker/cli-plugins/docker-buildx
# Verify installation
docker buildx version