66 lines
4.3 KiB
Markdown
66 lines
4.3 KiB
Markdown
# repliqate - reproducible builds
|
|
|
|
|
|
|
|
Repliqate is a tool for testing and confirming reproducible builds processes based on Qemu, and a Debian Cloud image.
|
|
|
|
## Step 0: Install Qemu for your System
|
|
|
|
Qemu is usually available via your package manager, of can be compiled from source.
|
|
|
|
## Step 1: Obtain a Debian Cloud Image
|
|
|
|
Official cloud images can be found here: https://cloud.debian.org/images/cloud/ - you will need a `nocloud` to work with Qemu e.g. **debian-11-nocloud-amd64-20221219-1234.qcow2**
|
|
|
|
### Optional Step: Resize Cloud Images
|
|
|
|
Depending on how many packages need to be installed as part of your build script you may need to resize the disk size of the provided image e.g. `qemu-img resize debian-11-nocloud-amd64-20221219-1234.qcow2 +10G`
|
|
|
|
## Step 2: Define a Build Script
|
|
|
|
A build script sets up dependencies, folder locations, and source code repositories needed to produce a builds. See the Cwtch [repliqate-scripts repository](https://git.openprivacy.ca/cwtch.im/repliqate-scripts) folder for examples and inspiration.
|
|
|
|
## Step 3: Run Repliqate
|
|
|
|
Basic example: `repliqate cloud-img.qcow2 build.script`
|
|
|
|
## Writing a Build Script
|
|
|
|
By default, each line is interpreted as a shell command to run in the build environment e.g. `echo "Hello World` will result in `echo "Hello World"` being run in the VM.
|
|
|
|
Lines starting with `#` are treated as comments and ignored.
|
|
|
|
Lines starting with `@%` e.g. `@%echo "Hello World"` indicate to repliqate that it should output the result of the command. This is useful when debugging build steps.
|
|
|
|
Lines starting with `@!` are treated as repliqate meta commands, these commands extend the execution environment with functionality useful for constructing and checking reproducible builds:
|
|
|
|
* `@!check <filename> <sha512hash to check>` - executes a `sha512sum` command on the given file, checks the result against the given hash, and terminates the build on failure.
|
|
* `@!preserve` - repliqate will skip deleting temporary disk images at shutdown (see FAQ for more information)
|
|
* `@!setup-secondary` - repliqate will format and mount the secondary disk image at `/mount`
|
|
* `@!extract <filename>` - repliqate will extract the given file from the virtual machine and into the current working directory. `netcat` needs to be installed on the virtual machine for this to work (this can be done by adding `apt install -y --no-install-recommends netcat` to your build script (assuming you have also run `apt update` previously).
|
|
|
|
|
|
|
|
## FAQ
|
|
|
|
Answers to a few possible questions:
|
|
|
|
### What are the `inuse-*qcow2` and `vd.img` files
|
|
|
|
These are temporary disk images that are used while repliqate is running. By default, repliqate deletes these once the script has run, however adding the metacommand
|
|
`@!preserve` anywhere in the build script will cause repliqate to skip the cleanup - these disk images can then be mounted and explored further (e.g. to debug an issue)
|
|
|
|
### Why not use Docker?
|
|
|
|
We do use Docker as part of our continuous build processes. However, we have chosen to develop repliqate to meet two specific requirements:
|
|
|
|
1. Standlone and Unprivileged - Our goal with repliqate was to provide a standalone way of reproducing builds. [Docker requires a whole host of permissions](https://docs.docker.com/engine/install/linux-postinstall/) to be granted, and privileged services to be setup, in order for containers to be run by non-root users.
|
|
2. Redundancy - Having a tool that is entirely separate from our build containers (and the Docker infrastructure that supports them) allows us have both keep each other in check. Many of the containers we rely on in our build process are managed and maintained by other teams. By maintaining our own reproducible build scripts we can narrow down the exact environments needed while still taking advantage of open source infrastructure for our day-to-day builds.
|
|
|
|
### Builds are slow. Can they be sped up?
|
|
|
|
Virtualization is inherently slower than running on bare metal. We consider the speed overhead worth it for complete isolation of the build environment. However, if you are considering using repliqate for
|
|
something more speed-sensitive then you may want to consider enabling and using qemu with [kvm](https://wiki.qemu.org/Features/KVM) enabled.
|
|
|
|
Note: This is now enabled by default. To disable kvm acceleration see the note in `src/main.rs`
|