style comments
This commit is contained in:
parent
9efc00a290
commit
eea163acab
|
|
@ -12,7 +12,7 @@ authors:
|
|||
image_url: /img/sarah.jpg
|
||||
---
|
||||
|
||||
In this development log, we continue our previous work on [reproducible cwtch bindings](https://docs.cwtch.im/blog/cwtch-bindings-reproducible), uncovering the final few sources of variation between our [repliqate](https://git.openprivacy.ca/openprivacy/repliqate) scripts and our docker/drone builds, leading to fully reproducible builds for Cwtch Android bindings!
|
||||
In this development log, we continue our previous work on [reproducible Cwtch bindings](https://docs.cwtch.im/blog/cwtch-bindings-reproducible), uncovering the final few sources of variation between our [Repliqate](https://git.openprivacy.ca/openprivacy/repliqate) scripts and our docker/drone builds, leading to fully reproducible builds for Cwtch Android bindings!
|
||||
|
||||
|
||||
|
||||
|
|
@ -20,23 +20,23 @@ In this development log, we continue our previous work on [reproducible cwtch bi
|
|||
|
||||
## Changes Necessary for Reproducible Android Bindings
|
||||
|
||||
After a thorough investigation of the build artifacts produced by repliqate and drone we uncovered three additional sources of variation:
|
||||
After a thorough investigation of the build artifacts produced by Repliqate and Drone we uncovered three additional sources of variation:
|
||||
|
||||
- **Insufficient path stripping introduced by Android NDK tools** - it turns out that Android builds using NDK versions below 22 are not reproducible as they produced randomized artifacts (through unstripped temporary directory paths appearing in compiled binares). NDK 22 [changed the binutils and default linker](https://github.com/android/ndk/wiki/Changelog-r22) to versions that correctly strip such paths from build artifacts. As such it was necessary for us to update the NDK version we used. We chose the technically outdated NDK 22 rather than the more modern NDK 25 to minimize Android OS compatibility changes during this switch. However, per our [long term support plan](https://docs.cwtch.im/blog/cwtch-platform-support), we will be moving towards adopting the latest NDK in the future.
|
||||
- **Insufficient path stripping introduced by Android NDK tools** - it turns out that Android builds using NDK versions below 22 are not reproducible as they produce randomized artifacts (through unstripped temporary directory paths appearing in compiled binares). NDK 22 [changed the binutils and default linker](https://github.com/android/ndk/wiki/Changelog-r22) to versions that correctly strip such paths from build artifacts. As such it was necessary for us to update the NDK version we used. We chose the technically outdated NDK 22 rather than the more modern NDK 25 to minimize Android OS compatibility changes during this switch. However, per our [long term support plan](https://docs.cwtch.im/blog/cwtch-platform-support), we will be moving towards adopting the latest NDK in the future.
|
||||
- **Paths in DWARF entries** - while we have been unable to track down exactly where these are being introduced, we did track the final difference in the produced bindings to DWARF debug lines embedded in compiled ELF binaries. These entries encoded the actual location of the NDK on the disk of the build machine, instead of the symbolic link that we believed should have been followed. By physically placing the NDK at same location in repliqate as in our Docker container we were able to get these entries to be consistent - however there is still work to do to understand exactly why they are being introduced at all.
|
||||
|
||||
<figure>
|
||||
|
||||
|
||||
[](/img/aar-diff.png)
|
||||
|
||||
<figcaption>Vimdiff comparing the decoded (<code>readelf --debug-dump=line</code>) DWARF debug section of drone-produced Android bindings v.s. repliqate produced. The difference in paths are highlighted.</figcaption>
|
||||
<figcaption>Vimdiff comparing the decoded (<code>readelf --debug-dump=line</code>) DWARF debug section of Drone-produced Android bindings v.s. Repliqate-produced. The difference in paths is highlighted.</figcaption>
|
||||
</figure>
|
||||
|
||||
- **Go Compiler Acquisition** - our Docker container was compiling the Go compiler from source, while the repliqate was downloading a pre-compiled version. During debugging we changed the Dockerfile to also download the pre-compiled version in order to eliminate the difference as a potential reproducibility issue. Our tests indicated that there *was* a difference between artifacts produced by the precompiled compiler v.s. one build from source - this is likely explained by introduced environmental differences cuased by the compilation of the compiler itself e.g. the contents/versions of modules in the Go package cache which we have seen as having an impact on other produced binaries.
|
||||
- **Go Compiler Acquisition** - our Docker container was compiling the Go compiler from source, while Repliqate was downloading a pre-compiled version. During debugging we changed the Dockerfile to also download the pre-compiled version in order to eliminate the difference as a potential reproducibility issue. Our tests indicated that there *was* a difference between artifacts produced by the precompiled compiler v.s. one built from source - this is likely explained by introduced environmental differences caused by the compilation of the compiler itself e.g. the contents/versions of modules in the Go package cache which we have seen as having an impact on other produced binaries.
|
||||
|
||||
## Repliqate Scripts
|
||||
|
||||
With those issues now fixed, Cwtch Android bindings are **officially reproducible**. The first version that officially met this requirement was 1.10.5, and you can find the repliqate script under [cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script) in the [cwtch repliqate scripts repository](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/):
|
||||
With those issues now fixed, Cwtch Android bindings are **officially reproducible!** The first version that officially met this requirement was 1.10.5, and you can find the Repliqate script under [cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/src/branch/main/cwtch-bindings-v1.10.5/libcwtch.v1.10.5-android.script) in the [Cwtch Repliqate scripts repository](https://git.openprivacy.ca/cwtch.im/repliqate-scripts/).
|
||||
|
||||
This is another big milestone towards our ultimate goal of full reproducibility for Cwtch releases.
|
||||
|
||||
|
|
|
|||
|
|
@ -60,3 +60,8 @@
|
|||
td > img {
|
||||
vertical-align: sub;
|
||||
}
|
||||
|
||||
figcaption {
|
||||
font-size:10fpx;
|
||||
text-align: center;
|
||||
}
|
||||
Loading…
Reference in New Issue