Androdi Repro

blog/2023-02-10-android-reproducibility.md

@@ -0,0 +1,54 @@
+---
+title: Making Cwtch Android Bindings Reproducible
+description: "In this devlog we revisit reproducible builds and make Cwtch Android bindings reproducible"
+slug: cwtch-android-reproducibility
+tags: [cwtch, cwtch-stable, reproducible-builds, bindings, repliqate]
+image: /img/devlog6_small.png
+hide_table_of_contents: false
+toc_max_heading_level: 4
+authors:
+- name: Sarah Jamie Lewis
+  title: Executive Director, Open Privacy Research Society
+  image_url: /img/sarah.jpg
+---
+
+In this development log, we continue our previous work on [reproducible cwtch bindings](https://docs.cwtch.im/blog/cwtch-bindings-reproducible), uncovering the final few sources of variation between our [repliqate](https://git.openprivacy.ca/openprivacy/repliqate) scripts and our docker/drone builds, leading to fully reproducible builds for Cwtch Android bindings!
+
+![](/img/devlog6.png)
+
+<!--truncate-->
+
+## Changes Necessary for Reproducible Android Bindings
+
+After a thorough investigation of the build artifacts produced by repliqate and drone we uncovered three additional sources of variation:
+
+- **Insufficient path stripping introduced by Android NDK tools** - it turns out that Android builds using NDK versions below 22 are not reproducible as they produced randomized artifacts (through unstripped temporary directory paths appearing in compiled binares). NDK 22 [changed the binutils and default linker](https://github.com/android/ndk/wiki/Changelog-r22) to versions that correctly strip such paths from build artifacts. As such it was necessary for us to update the NDK version we used. We chose the technically outdated NDK 22 rather than the more modern NDK 25 to minimize Android OS compatibility changes during this switch. However, per our [long term support plan](https://docs.cwtch.im/blog/cwtch-platform-support), we will be moving towards adopting the latest NDK in the future. 
+- **Paths in DWARF entries** - while we have been unable to track down exactly where these are being introduced, we did track the final difference in the produced bindings to DWARF debug lines embedded in compiled ELF binaries. These entries encoded the actual location of the NDK on the disk of the build machine, instead of the symbolic link that we believed should have been followed. By physically placing the NDK at same location in repliqate as in our Docker container we were able to get these entries to be consistent - however there is still work to do to understand exactly why they are being introduced at all.
+
+<figure>
+
+![](/img/aar-diff.png)
+
+<figcaption>Vimdiff comparing the decoded (<code>readelf --debug-dump=line</code>) DWARF debug section of drone-produced Android bindings v.s. repliqate produced. The difference in paths are highlighted.</figcaption>
+</figure>
+
+- **Go Compiler Acquisition** - our Docker container was compiling the Go compiler from source, while the repliqate was downloading a pre-compiled version. During debugging we changed the Dockerfile to also download the pre-compiled version in order to eliminate the difference as a potential reproducibility issue. Our tests indicated that there *was* a difference between artifacts produced by the precompiled compiler v.s. one build from source - this  is likely explained by introduced environmental differences cuased by the compilation of the compiler itself e.g. the contents/versions of modules in the Go package cache which we have seen as having an impact on other produced binaries.
+
+## Repliqate Scripts
+
+With those issues now fixed, Cwtch Android bindings are **officially reproducible**. The first version that officially met this requirement was 1.10.5, and you can find the repliqate script under [cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script) in the [cwtch repliqate scripts repository](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/):
+
+This is another big milestone towards our ultimate goal of full reproducibility for Cwtch releases.
+
+## Help us go further!
+
+We couldn't do what we do without all the wonderful community support we get, from [one-off donations](https://openprivacy.ca/donate) to [recurring support via Patreon](https://www.patreon.com/openprivacy).
+
+If you want to see us move faster on some of these goals and are in a position to, please [donate](https://openprivacy.ca/donate). If you happen to be at a company that wants to do more for the community and this aligns, please consider donating or sponsoring a developer.
+
+Donations of **$5 or more** can opt to receive stickers as a thank-you gift!
+
+For more information about donating to Open Privacy and claiming a thank you gift [please visit the Open Privacy Donate page](https://openprivacy.ca/donate/).
+
+![A Photo of Cwtch Stickers](/img/stickers-new.jpg)
+