Final Fixes

blog/2023-01-20-reproducible-builds-bindings.md

@@ -37,11 +37,9 @@ The Cwtch Bindings build pipeline results in four compiled libraries:
 
 These compiled libraries eventually make their way into Cwtch-based applications, like the Cwtch UI.
 
-
 ## Making libCwtch Reproducible
 
-Docker containers alone aren't enough to guarantee reproducibility. On inspection of several builds of the same source tree,
-we noticed a few elements that were distinct to each build:
+Docker containers alone aren't enough to guarantee reproducibility. On inspection of several builds of the same source tree, we noticed a few elements that were distinct to each build:
 
 * **Go Build ID**: By default, Go includes a build ID as part of compiled binaries. When using CGO this build ID is non-deterministic and differs for every build. We made the decision to override this build ID for all outputs, setting it to the version of the code being built.
 * **Build Paths and Go Environment Variables**: By default, Go includes full filesystem paths, and many Go-specific environment variables in the compiled binary – ostensibly to aid with debugging. These can be removed using the `trimPath` option, which we now specify for all bindings builds.
@@ -64,13 +62,16 @@ Fortunately this field can be zeroed out through passing `-Xlinker –no-insert-
 
 With that, and the universal Go fixes outlined above, Windows bindings are now reproducible using the same standalone Linux environment.
 
+
 ### Android Specific Considerations
 
-With the above universal Go fixes, Android build artifacts become repeatable. However, achieving full reproducibility for Android builds requires a number of specific dependencies:
+With the above universal Go fixes, Android build artifacts become almost repeatable. And on certain setups they appear to be reproducible. However,achieving full reproducibility for Android builds requires a number of specific environment dependencies, and considerations:
 
-* Cwtch makes use of [GoMobile](https://github.com/golang/mobile) for compiling Android libraries. We pin to a specific version `43a0384520996c8376bfb8637390f12b44773e65` in our Docker containers.
-* We still use [sdk-tools](https://developer.android.com/studio/releases/sdk-tools) instead of the new [commandline-tools](https://developer.android.com/studio/command-line). The latest version of sdk-tools is `4333796` and available from: https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip
-* Finally, official Cwtch Android builds use OpenJDK 8.
+* Cwtch makes use of [GoMobile](https://github.com/golang/mobile) for compiling Android libraries. We pin to a specific version `43a0384520996c8376bfb8637390f12b44773e65` in our Docker containers. Unlike `go build`, the `trimpPath` parameter passed to GoMobile does not strip all development environment paths. This means that the build environment needs consistent directory structures.  We have noticed inconsistencies in the detail stripped between setups e.g. cwtch.aar files build by our Docker and Repliqate builds still contain randomized `/tmp/go-build*` references that developer builds do not. We are still in the process of tracking down how these inconsistencies are introduced.
+* We still use [sdk-tools](https://developer.android.com/studio/releases/sdk-tools) instead of the new [commandline-tools](https://developer.android.com/studio/command-line). The latest version of sdk-tools is `4333796` and available from: [https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip](https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip). As part of our plans for Cwtch Stable we will be updating this dependency.
+* Cwtch Android builds currently use OpenJDK 8, unchanged from the earliest prototypes when Android development required Java 8. There is no nice way of obtaining this JDK version anymore, our Docker Containers are based on the now deprecated `openjdk:8` image. As with sdk-tooks, as part of our plans for Cwtch Stable we will be updating this dependency. 
+
+All of the above mean that we cannot consider Android builds to be reproducible yet, but we believe this is an achievable goal within the next couple of release cycles.
 
 ### OSX Specific Considerations
 
@@ -82,11 +83,11 @@ In order to fully guarantee reproducibility, OSX libraries need to be built on t
 
 In an ideal world we would be able to cross-compile OSX libraries on Linux the same way we do for Windows and Android. While there are no technical limits,  compiling for OSX is dependent on a [proprietary SDK](https://www.apple.com/legal/sla/docs/xcode.pdf). There is no way to trustfully obtain this SDK from anyone except Apple, and the license appears to strictly prohibit transferring the SDK to non-Apple hardware.
 
-Because of these limitations we cannot yet offer a way to automatically verify OSX builds, in the same way that we can for Linux, Windows, and Android. We will continue to look for ways to bring OSX builds to the same level as the rest of our distributions.
+Because of these limitations we cannot yet offer a way to automatically verify OSX builds, in the same way that we can for Linux, Windows, and Android. We will continue to look for ways to bring OSX builds to the same level as the rest of our Windows and Linux distributions.
 
 ## Introducing Repliqate!
 
-With all the above changes, Cwtch Bindings are fully reproducible!
+With all the above changes, **Cwtch Bindings for Linux and Windows are fully reproducible!**
 
 That alone is great, but we also want to make it easier for **you** to check the reproducibility of our builds yourself! As we noted in the introduction, the whole point of reproducible builds is that you no longer have to trust binaries provided by the Cwtch Team.
 
@@ -96,7 +97,9 @@ Repliqate makes it easy to construct isolated build environments, powered by Qem
 
 Repliqate runs [build-scripts](https://git.openprivacy.ca/openprivacy/repliqate#writing-a-build-script) to perform actions like downloading the specific versions of Go used in Cwtch official builds, grabbing a copy of the source code for Cwtch bindings, compiling the latest tagged version, and checking the hash against the same version that is available from [builds.openprivacy.ca](https://build.openprivacy.ca/files/).
 
-We now provide [Repliqate build-scripts](https://git.openprivacy.ca/cwtch.im/repliqate-scripts) for checking Linux, Windows, and Android Builds of libCwtch bindings.
+We now provide [Repliqate build-scripts](https://git.openprivacy.ca/cwtch.im/repliqate-scripts) for reproducible both [Linux libCwtch.so builds](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/libcwtch.v1.10.2-linux.script), [Windows libCwtch.dll builds](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/libcwtch.v1.10.2-windows.script)!
+
+We also have a partially repeatable [Android cwtch.aar build](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/libcwtch.v1.10.2-android.script) script that reproduces the official build environment, which we will be using to complete Android reproducible builds as detailed in the last section.
 
 You can (and I want to highly encourage you to) perform all these steps yourself (either via Repliqate, or a setup with the same specifications) and report back. We want to know if there are any other barriers to reproducing cwtch bindings, and anything that we can do to make the process easier.