This is not 100% fix as I can't reproduce tests hanging in local
environment, but the idea is the following:
1. `reaper.Start()` started reaper loop in a goroutine which starts with
subscribing to `SIGCHLD`.
2. `reaper.Start()` just spawned goroutine never waiting for it.
3. if after `reaper.Start()` reaper goroutine never runs, but process is
created in the test and it terminates, `SIGCHLD` will be ignored and
reaper will never wake up to reap the child.
4. process test hangs as it waits for reaper to reap the child and
return its exit status.
Sample failures:
```
=== RUN TestProcessSuite/runReaper=true/TestRunLogs
2019/10/15 14:17:41 state Running: Process Process(["/bin/sh" "-c" "echo -n \"Test 1\nTest 2\n\""]) started with PID 11802
coverage: 60.0% of statements
panic: test timed out after 10m0s
```
```
=== RUN TestCmdSuite/runReaper=true/TestRun
true
coverage: 71.4% of statements
panic: test timed out after 10m0s
```
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This removes the github.com/pkg/errors package in favor of the official
error wrapping in go 1.13.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
The gofumpt linter is a stricter drop-in replacement for gofmt. The
rules are ones that I strongly agree with and I think it would be better
if we added this linter instead of nit picking every PR.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
In UNIX, any zombies without parent process get re-parented to process
with PID 1 (usually running init), and PID 1 process should take care of
them (usually simply clean them up). Cleaning up zombies is important,
as they still take kerner resources, and having enormous amount of
zombie processes signifcantly degrades system performance.
For Talos, PID 1 process is machined, and machined itself forks to run
other processes in process runner and `pkg/cmd` one-time commands. Naive
solution of running `wait()` loop doesn't work as it might race with
`Process.Wait()` and clean up zombie which wasn't re-parented which
leads to process execution false failure.
After considering other solutions, we decided to go with the simple
approach: machined runs global zombie process reaper which publishes
information about reaped zombies. Any call to `Process.Wait()` (or
`Command.Wait()` which calls it) should be replaced with listening to
reaper's channel for notifications to catch info about the process which
was created in this call.
There are several changes in this PR:
1. Reaper implementation itself, started from machined.
2. Process runner and `pkg/cmd` can either use regular `Command.Wait()`
or use reaper notifications depending on reaper status (running/not
running). This allows using this code outside of machined.
3. Small bug fixes with process log which was affecting the tests.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
