Repository navigation
fix(djpeg): reap the child process when decoding fails - #198
Merged
Merged
Conversation
run() returned as soon as bmp.Decode failed, skipping cmd.Wait(), so every file djpeg could not decode left a zombie behind - one per attempt, and Get()'s two-pass resize calls run() twice. In a container these accumulate until the cgroup pids limit is reached, after which every fork/exec inside the container fails. That includes ffmpeg, so video thumbnails silently stop being generated while photos keep working from cached or embedded-EXIF thumbnails - which makes it present as 'some videos have no thumbnail' rather than as a process leak. Observed on a 167k-file library with ~147 undecodable .jpg files: 1,547 zombie djpeg processes, pids pinned at the 1578 default, 3,745 recorded hits on the limit. cmd.Wait() is now called on every path. Error precedence is unchanged: a decode error is still reported ahead of a non-zero exit, which is what callers saw before. Adds a regression test that counts this process's zombie children across 20 failed decodes. It fails on the previous code with 'leaked 20 zombie(s)'. Fixes SmilyOrg#197
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused fix correctly addresses the process leak and includes suitable regression coverage.
0 open findings
What changed in this PR
Fixes zombie djpeg processes caused by decode failures, preventing eventual PID exhaustion.
Changes:
- Unconditionally waits for
djpegafter decoding while preserving error precedence. - Adds Linux regression tests for failure and success paths.
| File | Description |
|---|---|
internal/io/djpeg/djpeg.go |
Reaps the child process on every decode path. |
internal/io/djpeg/djpeg_test.go |
Verifies child reaping through /proc. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Owner
|
@PaulEdwardsAP thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #197.
The bug
Djpeg.run()only callscmd.Wait()on the success path. Whenbmp.Decode(stdout)fails it returns immediately, so the child is never reaped:So every file djpeg cannot decode leaks one zombie per attempt — and
Get()'s two-pass resize callsrun()twice, so up to two per file per index pass.exec.CommandContextkills the process when the context is cancelled, but killing is not reaping; aWait()is still required.Why it bites harder than it looks
In a container the zombies accumulate until the cgroup pids limit is reached, after which every
fork/execinside the container fails. That includes ffmpeg, so video thumbnails silently stop being generated while photos keep working (cached or embedded-EXIF thumbnails). It therefore presents as "some videos have no thumbnail", which is a long way from a process leak.Observed on a 167k-file library containing ~147 undecodable
.jpgfiles: 1,547 zombiedjpegprocesses,pids.currentpinned at the 1578 default, andpids.eventsrecording 3,745 hits on the limit.The change
cmd.Wait()is now called on every path.Error precedence is deliberately unchanged — a decode error is still returned ahead of a non-zero exit status, so callers see exactly what they saw before. Only the reaping is new.
Test
TestRunReapsChildOnDecodeFailurecounts this process's zombie children across 20 failed decodes of a non-JPEG file. On the previous code it fails with:and passes with the fix.
TestRunReapsChildOnSuccesscovers the success path and encodes its own sample JPEG, so neither test needs fixtures. Both skip when djpeg is absent or off Linux (reaping is only observable via/proc).Not affected
internal/io/ffmpeg/ffmpeg.gousescmd.Output(), which waits internally — I checked, and it has no equivalent problem. (The issue speculated it might; it does not.)