Faster Docker build directly on host #10

nutomic · 2020-01-02T23:18:55Z

nutomic commented

2020-01-02 23:18:55 +00:00

I couldnt deal with the long build times to run federation tests, so I had the idea of doing it this way instead. Unfortunately its not working, the resulting Docker image always crashes with this error:

standard_init_linux.go:211: exec user process caused "no such file or directory"

Any clue what the problem could be?

I couldnt deal with the long build times to run federation tests, so I had the idea of doing it this way instead. Unfortunately its not working, the resulting Docker image always crashes with this error: ``` standard_init_linux.go:211: exec user process caused "no such file or directory" ``` Any clue what the problem could be?

dessalines commented

2020-01-02 23:37:25 +00:00

https://stackoverflow.com/questions/51508150/standard-init-linux-go190-exec-user-process-caused-no-such-file-or-directory

Alpine doesn't have bash, so you have to do #!/bin/sh for the script.

https://stackoverflow.com/questions/51508150/standard-init-linux-go190-exec-user-process-caused-no-such-file-or-directory Alpine doesn't have bash, so you have to do `#!/bin/sh` for the script.

nutomic commented

2020-01-03 01:40:31 +00:00

But its not running bash inside the container, just the Lemmy binary that is copied in. Basically the same as the existing Docker image (unless I am missing something).

Edit: To be clear, the run-federation-test.bash (and the actual build) run outside of Docker.

But its not running bash inside the container, just the Lemmy binary that is copied in. Basically the same as the existing Docker image (unless I am missing something). Edit: To be clear, the `run-federation-test.bash` (and the actual build) run outside of Docker.

dessalines commented

2020-01-04 17:52:58 +00:00

Hrm, I'm not sure. I've seen that error with docker before tho, docker doesn't play well with scripts as entrypoints from what I could see.

nutomic commented

2020-01-07 16:08:15 +00:00

Okay I found it, it was this problem: https://stackoverflow.com/questions/49098753/unable-to-run-a-docker-image-with-a-rust-executable

I fixed it by using the musl-builder image instead of Alpine. Installing libc might also fix the problem, but it doesnt matter much as its only for debugging.

Unfortunately its still not working entirely, the console doesnt show any errors now but the website says only No such file or directory (os error 2). No idea what file it means, config is loaded and the ui folder also looks correct.

Okay I found it, it was this problem: https://stackoverflow.com/questions/49098753/unable-to-run-a-docker-image-with-a-rust-executable I fixed it by using the musl-builder image instead of Alpine. Installing libc might also fix the problem, but it doesnt matter much as its only for debugging. Unfortunately its still not working entirely, the console doesnt show any errors now but the website says only `No such file or directory (os error 2)`. No idea what file it means, config is loaded and the ui folder also looks correct.

👍 1

nutomic commented

2020-01-09 16:20:16 +00:00

What the hell, I didnt change anything but its working now. One remaining problem I have is that the yarn build is run every time, is it possible to skip that if none of the ui files changed?

Other than that, the PR should be ready to merge.

Edit: Maybe this was fixed by me wiping my Docker volumes, so maybe we need to use seperate volumes here. Just a guess though.

What the hell, I didnt change anything but its working now. One remaining problem I have is that the yarn build is run every time, is it possible to skip that if none of the ui files changed? Other than that, the PR should be ready to merge. Edit: Maybe this was fixed by me wiping my Docker volumes, so maybe we need to use seperate volumes here. Just a guess though.

dessalines commented

2020-01-10 19:31:09 +00:00

I'll start building this on my machine, I know it'll take a while lol.

dessalines commented

2020-01-10 19:55:19 +00:00

Yup works good! One thing that's a little annoying, is that we need to clear storage when going from one instance to another, but I've never looked up how to scope browser storage to a specific port number, since they're mainly based on domain name (and when testing, both use localhost obvi).

An easier fix, which we might have to do eventually anyway, is add some /etc/hosts manipulation to this bash script.

Yup works good! One thing that's a little annoying, is that we need to clear storage when going from one instance to another, but I've never looked up how to scope browser storage to a specific port number, since they're mainly based on domain name (and when testing, both use localhost obvi). An easier fix, which we might have to do eventually anyway, is add some /etc/hosts manipulation to this bash script.

nutomic closed this pull request

2020-01-10 19:56:40 +00:00

nutomic commented

2020-01-10 22:43:26 +00:00

What about the thing I mentioned, not running the yarn build if nothing changed?

dessalines commented

2020-01-11 05:34:01 +00:00

Its not too big of a deal IMO, since the yarn fuse build takes an average of < 10 seconds. Docker is just smart and knows that if nothing in the folder has changed, don't even run the command, just used the cached version. And rust seems to also be smart by looking in the target folder too.