therecipe/go mod support (was: Non-Deterministic Qt related build failures) #196
Labels
No Label
1app
2 apps
accessibility
android
before-beta
bug
duplicate
enhancement
first-contact
fixed
help wanted
infrastructure
invalid
low-priority
must-fix
needs testing
nice-to-have
question
tor
user-feedback
wontfix
No Milestone
3 Participants
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: cwtch.im/ui#196
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Am trying to build the latest HEAD, however compilation fails. Device is Debian based x86.
Have ensured the qt bindings are the latest version, and built, and have fetched the other dependencies using
go get ./...
.Compiler output says that it is an issue with cwtch file applets.go, however cwtch itself compiles correctly into a container.
The output from running
qtdeploy build desktop
:Initial Fix:
SetStatusCallback was added to libricochet-go about a week ago (
6517665498
) so it looks like you might need to fetch that again.Longer term fix: the ui ~almost~ supports go modules and if you do this:
It should fetch all the actual dependencies (in many cases, because of development, the latest head is not the release that is being depended upon) and allow you to build everything.
Wasn't able to get the module route working, though its not something I'm familiar with.
Ran commands with
gocode/src/cwtch.im/ui
as my working directory.However, building
ui
failed with:build cwtch.im/ui: cannot load github.com/therecipe/qt: open /home/wheest/gocode/src/cwtch.im/ui/vendor/github.com/therecipe/qt: no such file or directory
Maybe my working directory should be
gocode/src
? I tried this, however I think I created a state in my Go setup. Removed all createdgo.mod
files, but a restart was needed.Checked out the speicifc commit of libricochet-go, and moved past the error in OP.
However, building then throws the following error:
I imagine it's because I'm working with the qt bindings HEAD. Having now read a bit more about what go modules are, I can see why it's useful in this case.
I think that the project root should be where the
go modules
stuff is run, though am not sure. Reran the module commands with everything fresh, but got the same output as before.GO_PATH should point to
gocode/
(the directory should have src, bin and pkg in it)Ah! This is good, because I've been getting the same error on my dev environment and we've been struggling to reproduce.
In my case about 1 in every 5-6 build attempts succeeds (the other 4-5 display the error you posted), we believe that there is some non-determinism in the therecipe build process and it is likely building something in the wrong order. If you try running it a few times to see if it succeeds then we can narrow down where this error is coming from.
I've found that you can also run
qtdeploy -fast -debug build desktop
and successfully compile (because the steps overlap so -fast has everything it needs to build the rest of the go code)Aye, can confirm that running
qtdeploy build desktop
about 4 times worked. Non-determinism!Me earlier, being a daftie:
I had to run
git clean -d -f
, and then open a new shell to stop getting the build error:build cwtch.im/ui: cannot load github.com/therecipe/qt: open /home/wheest/gocode/src/cwtch.im/ui/vendor/github.com/therecipe/qt: no such file or directory
Now that it's build, running the desktop app gets stuck in a log message loop, without the GUI actually launching. This is probably a separate issue however. (none of these onion addresses are private ?)
Aye, but I mean when running the go modules commands:
This is done with
PWD=[...]gocode/src/cwtch.im/ui
?qtdeploy -fast -debug build desktop
doesn't work on my system, throwing this permission error:go build runtime/cgo: open /usr/local/go/pkg/linux_amd64/runtime/cgo.a: permission denied
how much RAM do you have on this computer? there is a suspicion that our compilation process may spike RAM at some point causing this failure.
I have 16GB and haven't seen it yet...
Ditto on 16GB of RAM.
Running this morning, I can now get builds running successfully every time.
Resetting the build using
git clean -f -d
again, but it could be that isn't sufficient for how this build system creates stuff.Tried artificially filling up my available memory using
stress
:stress --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.95;}' < /proc/meminfo)k --vm-keep -m 1
But builds are still successful (though the console never launches the UI).
Actually, just pulled the new HEAD, and the random Qt related build error occurred. Maybe there is some build caching outside of the source tree?
Memory never goes more than 50% during build.
However, all my cores got 100% utilisation, and building failed 10 times.
Tried killing non-essential processes that are resource hungry, and build worked after 3 attempts.
Changed the title of this to reflect the Qt build errors.
Hmm, interesting. That would seem to support the idea that something isn't getting built properly because of memory contention.
Apologies if I was ambiguous. Memory utilisation seemed to be independent of potential for build success.
I think it was CPU utilisation, as the processes I killed to improve chances of successful builds were CPU hungry ones.
Admittedly, I only eyeballed
htop
, so perhaps there was a super-quick spike.Regarding the UI launch error, I think it was an independent issue, see how I resolved that in #200.
So it looks like it's coming from using go modules. we haven't quite cracked using therecipe and go modules yet. my reccomendation is
that seems to solve the irregularity in build fails for now. we are investigating this upstream
https://github.com/therecipe/qt/issues/932
this is non ideal and a work in progress
An update on this. Looks like the newest version of therecipe/qt has fixed the bugs that were causing build irregularities (see the discussion here https://github.com/therecipe/qt/issues/932)
I've not seen the issue in a few successive builds now, whereas I was seeing it almost constantly before.
Looks like this has been fixed. Closing.