Package download failure not categorized as spurious

[RalfJung]

This run had the following two regressions:

[INFO] [stderr] error: failed to download `aho-corasick v1.1.3`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

[INFO] [stderr] error: failed to download `itoa v1.0.10`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

Those should both be categorized as spurious.

[Skgland]

I think #788 should fix this and rust-lang/rustwide#98 should reduce false positives
Nope, this error doesn't appear to be in the prepare phase so those won't help.

Weird I would have expected that to already fail during prepare maybe there was a TOCTOU race condition.

[RalfJung]

Not sure how often this happens, but https://crater-reports.s3.amazonaws.com/pr-147185/index.html also had one of these:

[INFO] [stderr] error: failed to download `anyhow v1.0.64`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   unable to get packages from source
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   config.json not found in registry

[Skgland]

Thinking more about this, the only registry used by crater/rustwide is crates.io and that should always have a config.json¹² (it's not even a file that would be regularly updated so TOCTOU doesn't make sense after all). Makes me think there might be a network error underneath.

Footnotes

1. https://index.crates.io/config.json ↩

2. https://github.com/rust-lang/crates.io-index/blob/master/config.json ↩

[RalfJung]

Yeah I assume this is a network error.

[Skgland]

Looking at the logs I don't think this can be a network error as the command is run withe the --frozen flag (implying --offline) so no network requests should be made (also doesn't crater run without network access anyways during the test phase?).

So looking at the code at https://github.com/rust-lang/cargo/blob/79df5f4d52c8e8e30be353be20d4572c8653fa69/src/cargo/sources/registry/http_remote.rs#L505-L519 I would expect this to be one of those two error conditions resulting in the error we see being emitted at https://github.com/rust-lang/cargo/blob/79df5f4d52c8e8e30be353be20d4572c8653fa69/src/cargo/sources/registry/http_remote.rs#L439.

Could it be that between the prepare phase and the actual test a disk space clean-up ran and removed the cache?

[Skgland]

I think https://crater-reports.s3.amazonaws.com/pr-156114/try%235f6abd89da253a562ad85d235b9342a3ac336c28/reg/tael-0.2.0/log.txt also falls under this, but cargo has updated its error message

[INFO] [stderr] error: no matching package named `clap` found
[INFO] [stderr] location searched: crates.io index
[INFO] [stderr] required by package `tael v0.2.0 (/opt/rustwide/workdir)`
[INFO] [stderr] note: offline mode (via `--frozen`) can sometimes cause surprising resolution failures
[INFO] [stderr] help: if this error is too confusing you may wish to retry without `--frozen`

Also https://crater-reports.s3.amazonaws.com/pr-156114/try%235f6abd89da253a562ad85d235b9342a3ac336c28/reg/tauri-plugin-keygen-rs2-0.11.0/log.txt

[INFO] [stderr] error: no matching package named `keygen-rs` found
[INFO] [stderr] location searched: crates.io index
[INFO] [stderr] required by package `tauri-plugin-keygen-rs2 v0.11.0 (/opt/rustwide/workdir)`
[INFO] [stderr] note: offline mode (via `--frozen`) can sometimes cause surprising resolution failures
[INFO] [stderr] help: if this error is too confusing you may wish to retry without `--frozen`