Currently we have some license issues. We are working on it.

Verified Commit 81090c5e authored by noplanman's avatar noplanman
Browse files

Add base image building script and let GitLab make use of the prebuilt images.

parent 151470fb
variables:
VERSION: $CI_COMMIT_TAG
BBB_HOT: oh-yeah!
stages:
- build
......
......@@ -27,7 +27,7 @@ RUN set -ex; \
# install borg + dependencies
RUN set -ex; pip install -r requirements.d/development.txt;
RUN set -ex; pip install -r requirements.d/fuse.txt;
RUN set -ex; pip install -e .;
RUN set -ex; pip install -e .[fuse];
# create standalone binary
RUN set -ex; \
......
# Base image to build the binaries from, based on BorgBackup version 1.1.0
ARG BORG_ARCH
FROM resin/${BORG_ARCH}-python:3.6-slim
ARG BORG_ARCH
ENV BORG_DIR /borg
ENV BORG_BUILD_DIR ${BORG_DIR}/dist
WORKDIR ${BORG_DIR}
# install dev requirements
RUN set -ex; \
apt-get update; \
apt-get install -y --no-install-recommends \
build-essential git \
fuse libfuse-dev pkg-config \
libssl-dev libacl1-dev \
zlib1g-dev liblz4-dev; \
rm -rf /var/lib/apt/lists/*;
# fetch borg source code for version 1.1.0 and update pip
RUN set -ex; \
git clone -b 1.1.0 https://github.com/borgbackup/borg.git .; \
pip install --upgrade pip;
# install all requirements
RUN set -ex; \
pip install -r requirements.d/development.txt; \
pip install -r requirements.d/fuse.txt; \
pip install pyinstaller;
ARG BORG_ARCH
FROM noplanman/borg-binary-builder:base-1.1.0-${BORG_ARCH}
ARG BORG_ARCH
ARG BORG_VERSION=1.1.7
ENV BORG_DIR /borg
ENV BORG_BUILD_DIR ${BORG_DIR}/dist
WORKDIR ${BORG_DIR}
# fetch latest borg source code for this version
RUN set -ex; \
git fetch --all; \
git checkout ${BORG_VERSION};
# install borg + dependencies
RUN set -ex; \
pip install -r requirements.d/development.txt; \
pip install -r requirements.d/fuse.txt; \
pip install -e .[fuse];
# create standalone binary
RUN set -ex; \
pip install pyinstaller; \
pyinstaller --log-level DEBUG -F -n borg-${BORG_ARCH} src/borg/__main__.py;
RUN ln -s "${BORG_BUILD_DIR}/borg-${BORG_ARCH}" /usr/bin/borg
......@@ -30,6 +30,19 @@ See the [list of available architectures] ([on GitHub][available-architectures-g
- Extract borg `1.1.5` binary for Raspberry Pi 3 to `/homes/lemoi/borg`
`./extract.sh raspberrypi3 1.1.5 /homes/lemoi/borg`
### Build base image
To speed up the building process, certain architectures have a prebuilt base image with the development requirements already installed.
These base images are based on BorgBackup version 1.1.0.
Adding a new architecture to the base image list and pushing it to docker hub:
`./build-base.sh <arch>`
*Examples:*
- Build base image for `aarch64`
`./build-base.sh aarch64`
## How this works
### Build
......@@ -42,6 +55,9 @@ Then, the docker image is built for the requested architecture.
This is basically done by following the flow of setting up a [BorgBackup development environment] and [creating the standalone binary]. (For more info on this, just have a look at the `Dockerfile`)
When using the base images created by `build-base.sh`, instead of using the normal `Dockerfile`, you can set the `BBB_HOT` environment variable to any value, which makes the build script use `Dockerfile.hot`.
The build then starts off with the prebuilt base image, speeding up the build tremendously.
### Extract
The `extract.sh` script extracts the built binary from the image created using `build.sh`.
......
#!/usr/bin/env sh
# Usage: ./build-base.sh <arch>
# This script is used to build a base image for the given architecture, to make future building much faster.
# User needs to be logged in via "docker login" for the push to work.
# For available architectures see:
# https://docs.resin.io/reference/base-images/base-images/
BORG_ARCH=${1}
if [ -z "${BORG_ARCH}" ]; then
echo "Usage: ./build-base.sh <arch>";
echo "No arch provided";
exit 1;
fi
DOCKER_IMAGE=noplanman/borg-binary-builder:base-1.1.0-${BORG_ARCH}
echo "Building ${DOCKER_IMAGE}"
# https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build \
--build-arg "BORG_ARCH=${BORG_ARCH}" \
-t "${DOCKER_IMAGE}" \
-f Dockerfile.base .
echo "Pushing to docker hub"
docker push "${DOCKER_IMAGE}"
......@@ -8,15 +8,23 @@
BORG_ARCH=${1}
BORG_VERSION=${2:-1.1.7}
DOCKER_IMAGE=borg:${BORG_VERSION}-${BORG_ARCH}
if [ -z "${BORG_ARCH}" ]; then
echo "Usage: ./build.sh <arch> [<version>]";
echo "No arch provided";
exit 1;
fi
echo "Building BorgBackup ${BORG_VERSION} for ${BORG_ARCH}"
DOCKER_FILE=Dockerfile
DOCKER_IMAGE=borg:${BORG_VERSION}-${BORG_ARCH}
# For building the hot binary, make use of the prebuilt base docker image to speed things up.
# Note: Only a limited number of architectures are supported, check here:
# https://hub.docker.com/r/noplanman/borg-binary-builder/tags/
if [ ! -z "${BBB_HOT}" ]; then
DOCKER_FILE=Dockerfile.hot
fi
echo "Building ${DOCKER_IMAGE}"
# https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
docker run --rm --privileged multiarch/qemu-user-static:register --reset
......@@ -24,4 +32,5 @@ docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build \
--build-arg "BORG_ARCH=${BORG_ARCH}" \
--build-arg "BORG_VERSION=${BORG_VERSION}" \
-t "${DOCKER_IMAGE}" .
-t "${DOCKER_IMAGE}" \
-f "${DOCKER_FILE}" .
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment