Merge pull request #7910

4571d36 Add simple script for running gitian-build (Howard Chu)

3 files changed, 220 insertions, 0 deletions

diff --git a/contrib/gitian/DOCKRUN.md b/contrib/gitian/DOCKRUN.md
new file mode 100644
index 000000000..fa5e13c82
--- /dev/null
+++ b/contrib/gitian/DOCKRUN.md
@@ -0,0 +1,85 @@
+Quick Gitian building with docker
+=================================
+
+*Setup instructions for a Gitian build of Monero using Docker.*
+
+Gitian supports other container mechanisms too but if you have a Debian or
+Ubuntu-based host the steps can be greatly simplified.
+
+Preparing the Gitian builder host
+---------------------------------
+
+The procedure here will create a docker container for build preparation, as well as
+for actually running the builds. The only items you must install on your own host
+are docker and apt-cacher-ng. With docker installed, you should also give yourself
+permission to use docker by adding yourself to the docker group.
+
+```bash
+sudo apt-get install docker.io apt-cacher-ng
+sudo usermod -aG docker $USER
+su $USER
+```
+
+The final `su` command is needed to start a new shell with your new group membership,
+since the `usermod` command doesn't affect any existing sessions.
+
+If you want Mac binaries included in your build, you need to obtain the MacOS SDK:
+
+```bash
+curl -O https://bitcoincore.org/depends-sources/sdks/MacOSX10.11.sdk.tar.gz
+```
+
+Other User Preparation
+----------------------
+
+The final step will be to `gpg` sign the results of your build and upload them to GitHub.
+Before you can do that, you'll need
+* a GitHub account.
+If your GitHub account name is different from your local account name, you must
+set your GitHub account name for the script to use:
+
+```bash
+export GH_USER=<github account name>
+```
+
+* PGP keys - if you don't have one already, you can use `gpg --quick-gen-key` to generate it.
+* a fork of the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo on your GitHub account.
+Please follow the directions there for uploading your key first.
+
+**Note:** Please ensure your gpg public key is available to check signatures by adding it to the [gitian.sigs/gitian-pubkeys/](https://github.com/monero-project/gitian.sigs/tree/master/gitian-pubkeys) directory in a pull request.
+
+
+Building the Binaries
+---------------------
+
+The dockrun.sh script will do everything to build the binaries. Just specify the
+version to build as its only argument, e.g.
+
+```bash
+./dockrun.sh v0.17.2.3
+```
+
+The build should run to completion with no errors, and will display the SHA256 checksums
+of the resulting binaries. You'll be prompted to check if the sums look good, and if so
+then the results will be signed, and the signatures will be pushed to GitHub.
+
+You can also look in the [gitian.sigs](https://github.com/monero-project/gitian.sigs/) repo and / or [getmonero.org release checksums](https://web.getmonero.org/downloads/hashes.txt) to see if others got the same checksum for the same version tag.  If there is ever a mismatch -- **STOP! Something is wrong**.  Contact others on IRC / GitHub to figure out what is going on.
+
+
+Other Options
+-------------
+
+This script just runs the [gitian-build.py](gitian-build.py) inside a container named `gitrun`.
+You can set other options for that script by setting the OPT variable when running `dockrun.sh`
+e.g.
+
+```bash
+OPT="-j 8" ./dockrun.sh v0.17.2.3
+```
+
+You can also examine the build and install logs by running a shell in the container, e.g.
+
+```bash
+docker exec -it gitrun /bin/bash
+more builder/var/install-linux.log
+```
diff --git a/contrib/gitian/README.md b/contrib/gitian/README.md
index 0a40d4608..c922a2373 100644
--- a/contrib/gitian/README.md
+++ b/contrib/gitian/README.md
@@ -31,6 +31,8 @@ This guide explains how to set up the environment, and how to start the builds.
 
 * Gitian gives you the option of using any of 3 different virtualization tools: `kvm`, `docker` or `lxc`. This documentation will only show how to build with `lxc` and `docker` (documentation for `kvm` is welcome). Building with `lxc` is the default, but is more complicated, so we recommend docker your first time.
 
+* For a shortcut using `docker` follow the instructions in [DOCKRUN.md](DOCKRUN.md) instead
+of following the rest of this document..
 
 ## Create the gitianuser account
 
diff --git a/contrib/gitian/dockrun.sh b/contrib/gitian/dockrun.sh
new file mode 100755
index 000000000..015c411fd
--- /dev/null
+++ b/contrib/gitian/dockrun.sh
@@ -0,0 +1,133 @@
+#!/bin/sh
+
+if [ $# -ne 1 ]; then
+	echo "usage: $0 <version>"
+	exit 1
+fi
+VERSION=$1
+
+DOCKER=`command -v docker`
+CACHER=`command -v apt-cacher-ng`
+
+if [ -z "$DOCKER" -o -z "$CACHER" ]; then
+	echo "$0: you must first install docker.io and apt-cacher-ng"
+	echo "  e.g. sudo apt-get install docker.io apt-cacher-ng"
+	exit 1
+fi
+
+GH_USER=${GH_USER-$USER}
+
+TAG=gitrun-bionic
+TAG2=base-bionic-amd64
+IMAGE=`docker images | grep $TAG`
+
+WORKDIR=/home/ubuntu
+
+if [ -z "$IMAGE" ]; then
+GID=`getent group docker`
+mkdir -p docker
+cd docker
+
+# container for running gitian-build.py
+cat <<EOF > ${TAG}.Dockerfile
+FROM ubuntu:bionic
+
+ENV DEBIAN_FRONTEND=noninteractive
+RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
+RUN echo "$GID" >> /etc/group
+RUN apt-get update && apt-get --no-install-recommends -y install lsb-release ruby git make wget docker.io python3 curl
+
+RUN useradd -ms /bin/bash -U ubuntu -G docker
+USER ubuntu:docker
+WORKDIR $WORKDIR
+
+RUN	git clone https://github.com/monero-project/gitian.sigs.git sigs; \
+  git clone https://github.com/devrandom/gitian-builder.git builder; \
+  cd builder; git checkout c0f77ca018cb5332bfd595e0aff0468f77542c23; mkdir -p inputs var; cd inputs; \
+  git clone https://github.com/monero-project/monero
+
+CMD ["sleep", "infinity"]
+EOF
+
+docker build --pull -f ${TAG}.Dockerfile -t $TAG .
+
+cd ..
+docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
+if [ -f MacOSX10.11.sdk.tar.gz ]; then
+  docker cp MacOSX10.11.sdk.tar.gz gitrun:$WORKDIR/builder/inputs/
+else
+  echo "No MacOS SDK found, Mac builds will be omitted"
+fi
+
+fi
+
+IMAGE=`docker images | grep $TAG2`
+if [ -z "$IMAGE" ]; then
+mkdir -p docker
+cd docker
+
+# container for actually running each build
+cat <<EOF > ${TAG2}.Dockerfile
+FROM ubuntu:bionic
+
+ENV DEBIAN_FRONTEND=noninteractive
+RUN echo 'Acquire::http { Proxy "http://172.17.0.1:3142"; };' > /etc/apt/apt.conf.d/50cacher
+RUN apt-get update && apt-get --no-install-recommends -y install build-essential git language-pack-en \
+  wget lsb-release curl gcc-7 g++-7 gcc g++ binutils-gold pkg-config autoconf libtool automake faketime \
+  bsdmainutils ca-certificates python cmake gperf
+
+RUN useradd -ms /bin/bash -U ubuntu
+USER ubuntu:ubuntu
+WORKDIR $WORKDIR
+
+CMD ["sleep", "infinity"]
+EOF
+
+docker build --pull -f ${TAG2}.Dockerfile -t $TAG2 .
+
+cd ..
+
+fi
+
+RUNNING=`docker ps | grep gitrun`
+if [ -z "$RUNNING" ]; then
+  BUILT=`docker ps -a | grep gitrun`
+  if [ -z "$BUILT" ]; then
+    docker run -v /var/run/docker.sock:/var/run/docker.sock -d --name gitrun $TAG
+  else
+    docker start gitrun
+  fi
+fi
+docker cp gitian-build.py gitrun:$WORKDIR/
+docker exec -t gitrun ./gitian-build.py -d -b -D -n $OPT $GH_USER $VERSION
+RC=$?
+if [ $RC != 0 ]; then
+	exit $RC
+fi
+echo "\nBuild Results:\n"
+docker exec gitrun sh -c "sha256sum out/$VERSION/*"
+echo "\nIf these results look correct, type \"sign\" to sign them, otherwise ^C to stop now."
+read check
+if [ "$check" != "sign" ]; then
+	echo "Not signing, bye."
+	exit 1
+fi
+
+if [ ! -d sigs ]; then
+	git clone https://github.com/monero-project/gitian.sigs.git sigs
+	cd sigs
+	git remote add $GH_USER git@github.com:$GH_USER/gitian.sigs.git
+	cd ..
+fi
+
+DIRS=`docker exec gitrun sh -c "echo sigs/$VERSION-*"`
+for i in $DIRS; do
+	docker cp gitrun:$WORKDIR/$i sigs
+	gpg --detach-sign $i/$GH_USER/*.assert
+done
+
+cd sigs
+git checkout -b $VERSION
+git add $VERSION-*
+git commit -S -m "Add $GH_USER $VERSION"
+git push --set-upstream $GH_USER $VERSION