Repository navigation
Store local cache outputs by output ID instead of action ID - #26
Merged
Merged
Conversation
The local cache wrote each output to a file named after its action ID, with the metadata (output ID and size) in a separate file next to it. Two unserialized writers of one action ID with different outputs could interleave their renames so the metadata of one pointed at the data of the other. When the outputs had the same size, Go's size check could not catch it and the build used the wrong bytes. Outputs are now stored under their output ID, the SHA-256 of their content, and the metadata under the action ID acts as the pointer, as in Go's own disk cache. The data is written before the metadata, so whichever metadata wins references a complete file holding exactly its output. check() now also stats the output file and treats a missing or wrongly sized file as a miss. Entries written by earlier versions therefore fall through to the backend and are rewritten, without bumping fileFormatVersion, which would also invalidate the remote cache. handlePut no longer returns the path of an existing entry holding a different output ID: Go records the returned path as the file for the output ID it sent.
Earlier versions store data under the action ID and point to it from ".meta" files. When versions share one cache directory, or after a downgrade, an earlier version reading metadata written by this layout returned a data file that was missing or held a different output. Metadata now lives in ".meta2" files, so each layout only reads its own entries and earlier versions keep working on theirs.
richardartoul
added a commit
that referenced
this pull request
Oct 27, 2026
FSLockGroup waited at most one second for a lock and then failed the request. The holder is usually doing a backend GET, which regularly takes longer, so concurrent go processes requesting one action ID failed their builds with "failed to acquire lock: context deadline exceeded" (#23). Since #25 and #26, local cache writes are atomic and isolated, so the lock only deduplicates work. When it can't be acquired, the request now runs without it: a GET typically finds the entry the holder just cached, and a PUT writes it again. Other lock errors fall back the same way. Fallbacks are logged and counted in the exit stats. DoWithLock wraps acquisition failures in locking.ErrLockNotAcquired so the server can tell them apart from errors returned by the function. The wait is configurable with -lock-timeout / LOCK_TIMEOUT, and the default goes from 1s to 5s, since waiting a little longer usually ends in a local hit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #25.
localcachestored each output in a file named after its action ID, with its metadata (output ID and size) in a separate.metafile. #25 made each file's write atomic, but data and metadata are still two separate renames. Two unserialized writers of one action ID with different outputs (for exampleLOCK_TYPE=memoryacross processes with a non-reproducible action) can interleave like this:When the sizes differ, Go's size check turns this into a miss. When they match, the build silently uses the wrong bytes. This matters more once #23 turns a lock timeout into "proceed without the lock".
Change
<subdir>/v2<outputID>-d). Go computes the output ID as the SHA-256 of the body, so concurrent writers of one output file write identical bytes. The.metafile under the action ID is now the pointer to that file, as in Go's own disk cache.write()also skips rewriting an output file that already exists with the right size..meta2suffix so they never mix with earlier versions'.metafiles (see Compatibility).check()now stats the output file and treats a missing or wrongly sized file as a miss (one extrastatper lookup).handlePutused to return the existing entry's path whenever the action ID was already cached.cmd/gorecords the returnedDiskPathas the file for the output ID it sent (noteOutputFile(out, res.DiskPath)incmd/go/internal/cache/prog.go), so for a different output ID that mapped the new output ID to the old bytes. It now short-circuits only when the output IDs match.writeWithMetadatarejects an empty output ID instead of building a path from it.Compatibility
fileFormatVersionis not bumped, because it is also part of the backend key and bumping it would invalidate every remote cache. Instead, metadata now lives in.meta2files, so this layout and earlier versions never read each other's local entries. Versions sharing one cache directory, or a downgrade, each keep using their own entries. Earlier versions'.metafiles and action-ID data files are left untouched.Testing
TestLocalCacheConcurrentWritesOfOneActionIDWithDifferentOutputs: 16 writers put one action ID with distinct, same-size outputs while 4 readers keep checking that every hit's metadata matches the bytes in its file. Onmainit fails 5 of 5 runs, both from readers and in the final entry. Here it passes 30 of 30 runs with-race.TestLocalCacheIgnoresEntriesOfEarlierVersions: an entry in the old layout is a miss, and writing the entry again leaves the old.metaand data files unchanged (fails with the.metasuffix).TestLocalCacheIgnoresEntriesWithoutOutputFile: metadata whose output file is missing is a miss.TestPut_ExistingEntryWithDifferentOutput: a second PUT of one action ID with a different output returns aDiskPaththat holds the new output (fails onmain).go vet .andgo test -race . ./pkg/...pass. The cloud-backend integration tests were not run locally.