Commit 81090c5e authored by noplanman's avatar noplanman

Add base image building script and let GitLab make use of the prebuilt images.

parent 151470fb
variables:
VERSION: $CI_COMMIT_TAG
BBB_HOT: oh-yeah!
stages:
- build
......
......@@ -27,7 +27,7 @@ RUN set -ex; \
# install borg + dependencies
RUN set -ex; pip install -r requirements.d/development.txt;
RUN set -ex; pip install -r requirements.d/fuse.txt;
RUN set -ex; pip install -e .;
RUN set -ex; pip install -e .[fuse];
# create standalone binary
RUN set -ex; \
......
# Base image to build the binaries from, based on BorgBackup version 1.1.0
ARG BORG_ARCH
FROM resin/${BORG_ARCH}-python:3.6-slim
ARG BORG_ARCH
ENV BORG_DIR /borg
ENV BORG_BUILD_DIR ${BORG_DIR}/dist
WORKDIR ${BORG_DIR}
# install dev requirements
RUN set -ex; \
apt-get update; \
apt-get install -y --no-install-recommends \
build-essential git \
fuse libfuse-dev pkg-config \
libssl-dev libacl1-dev \
zlib1g-dev liblz4-dev; \
rm -rf /var/lib/apt/lists/*;
# fetch borg source code for version 1.1.0 and update pip
RUN set -ex; \
git clone -b 1.1.0 https://github.com/borgbackup/borg.git .; \
pip install --upgrade pip;
# install all requirements
RUN set -ex; \
pip install -r requirements.d/development.txt; \
pip install -r requirements.d/fuse.txt; \
pip install pyinstaller;
ARG BORG_ARCH
FROM noplanman/borg-binary-builder:base-1.1.0-${BORG_ARCH}
ARG BORG_ARCH
ARG BORG_VERSION=1.1.7
ENV BORG_DIR /borg
ENV BORG_BUILD_DIR ${BORG_DIR}/dist
WORKDIR ${BORG_DIR}
# fetch latest borg source code for this version
RUN set -ex; \
git fetch --all; \
git checkout ${BORG_VERSION};
# install borg + dependencies
RUN set -ex; \
pip install -r requirements.d/development.txt; \
pip install -r requirements.d/fuse.txt; \
pip install -e .[fuse];
# create standalone binary
RUN set -ex; \
pip install pyinstaller; \
pyinstaller --log-level DEBUG -F -n borg-${BORG_ARCH} src/borg/__main__.py;
RUN ln -s "${BORG_BUILD_DIR}/borg-${BORG_ARCH}" /usr/bin/borg
......@@ -30,6 +30,19 @@ See the [list of available architectures] ([on GitHub][available-architectures-g
- Extract borg `1.1.5` binary for Raspberry Pi 3 to `/homes/lemoi/borg`
`./extract.sh raspberrypi3 1.1.5 /homes/lemoi/borg`
### Build base image
To speed up the building process, certain architectures have a prebuilt base image with the development requirements already installed.
These base images are based on BorgBackup version 1.1.0.
Adding a new architecture to the base image list and pushing it to docker hub:
`./build-base.sh <arch>`
*Examples:*
- Build base image for `aarch64`
`./build-base.sh aarch64`
## How this works
### Build
......@@ -42,6 +55,9 @@ Then, the docker image is built for the requested architecture.
This is basically done by following the flow of setting up a [BorgBackup development environment] and [creating the standalone binary]. (For more info on this, just have a look at the `Dockerfile`)
When using the base images created by `build-base.sh`, instead of using the normal `Dockerfile`, you can set the `BBB_HOT` environment variable to any value, which makes the build script use `Dockerfile.hot`.
The build then starts off with the prebuilt base image, speeding up the build tremendously.
### Extract
The `extract.sh` script extracts the built binary from the image created using `build.sh`.
......
#!/usr/bin/env sh
# Usage: ./build-base.sh <arch>
# This script is used to build a base image for the given architecture, to make future building much faster.
# User needs to be logged in via "docker login" for the push to work.
# For available architectures see:
# https://docs.resin.io/reference/base-images/base-images/
BORG_ARCH=${1}
if [ -z "${BORG_ARCH}" ]; then
echo "Usage: ./build-base.sh <arch>";
echo "No arch provided";
exit 1;
fi
DOCKER_IMAGE=noplanman/borg-binary-builder:base-1.1.0-${BORG_ARCH}
echo "Building ${DOCKER_IMAGE}"
# https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build \
--build-arg "BORG_ARCH=${BORG_ARCH}" \
-t "${DOCKER_IMAGE}" \
-f Dockerfile.base .
echo "Pushing to docker hub"
docker push "${DOCKER_IMAGE}"
......@@ -8,15 +8,23 @@
BORG_ARCH=${1}
BORG_VERSION=${2:-1.1.7}
DOCKER_IMAGE=borg:${BORG_VERSION}-${BORG_ARCH}
if [ -z "${BORG_ARCH}" ]; then
echo "Usage: ./build.sh <arch> [<version>]";
echo "No arch provided";
exit 1;
fi
echo "Building BorgBackup ${BORG_VERSION} for ${BORG_ARCH}"
DOCKER_FILE=Dockerfile
DOCKER_IMAGE=borg:${BORG_VERSION}-${BORG_ARCH}
# For building the hot binary, make use of the prebuilt base docker image to speed things up.
# Note: Only a limited number of architectures are supported, check here:
# https://hub.docker.com/r/noplanman/borg-binary-builder/tags/
if [ ! -z "${BBB_HOT}" ]; then
DOCKER_FILE=Dockerfile.hot
fi
echo "Building ${DOCKER_IMAGE}"
# https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
docker run --rm --privileged multiarch/qemu-user-static:register --reset
......@@ -24,4 +32,5 @@ docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build \
--build-arg "BORG_ARCH=${BORG_ARCH}" \
--build-arg "BORG_VERSION=${BORG_VERSION}" \
-t "${DOCKER_IMAGE}" .
-t "${DOCKER_IMAGE}" \
-f "${DOCKER_FILE}" .
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment