Skip to content

fix(scoring): resolve indexers once per batch, not once per result in parallel - #863

Open
m4bard wants to merge 1 commit into
Listenarrs:canaryfrom
m4bard:fix/bug17-dbcontext-concurrency
Open

fix(scoring): resolve indexers once per batch, not once per result in parallel#863
m4bard wants to merge 1 commit into
Listenarrs:canaryfrom
m4bard:fix/bug17-dbcontext-concurrency

Conversation

@m4bard

@m4bard m4bard commented Aug 21, 2026

Copy link
Copy Markdown

Summary

ScoreSearchResults fans out with Task.WhenAll and each task queried the same scoped IIndexerRepository, so a batch of N results issued N concurrent queries against one scoped DbContext. This resolves each distinct indexer once before the fan-out and passes the results in.

Full write-up in #862.

Changes

Fixed

  • QualityProfileService.ScoreSearchResults resolves the batch's distinct indexers sequentially up front and hands them to the scorer.
  • SearchResultScorer takes an optional pre-resolved lookup and prefers it over querying.

The single-result ScoreSearchResult is unchanged and still queries; there is nothing to batch there, and it stays on the interface as it was.

Why it matters more than a caught exception suggests

The scorer already catches the failure and logs at Debug, so nothing fails. What it loses is indexerRetention, which stays 0, and the branch that sets isNzb from the indexer's type. Age and retention checks then score that result against assumptions that do not match its indexer. The batch comes back ordered plausibly and quietly wrong, and at the default log level there is nothing to see.

Three callers reach it: the quality profile controller, download submission, and the six-hour automatic search sweep.

Fewer queries, not just safer ones

A batch commonly carries many results across a handful of indexers. This goes from one query per result to one per distinct indexer, so it removes redundant work as well as the overlap.

IndexerSearchWorkflow already fetches its indexers before its own fan-out rather than inside it, so this makes the two agree rather than introducing an approach.

Testing

Asserting on the EF exception would mean racing it, which makes for a flaky test and a weak claim. The new test counts the overlap directly: a stub IIndexerRepository records the highest number of calls it ever had in flight at once, with a small delay so the overlap is observable rather than a scheduling accident.

It reports 12 concurrent without the change and 1 with it, over twelve results across three indexers. It also pins the call count at 3, one per distinct indexer, so a future change that restores per-result querying fails even if it somehow avoids overlapping.

Full suite: 3,030 passed, 0 failed, 125 skipped, against a 3,029 baseline on 03958c15.

Notes

The context's lifetime is the load-bearing fact here, so I verified it rather than inferring it from the registration. Reading the ServiceDescriptor back out of the built collection gives ListenArrDbContext => Scoped, with the repositories scoped as well. AddDbContextFactory(..., ServiceLifetime.Singleton) at PersistenceRegistrationExtensions.cs:25 registers the factory as a singleton and the context as scoped.

I have not reproduced wrong scores end to end against a running instance, and the issue says so. A collision needs the queries to actually overlap in time, which is machine-dependent. What is deterministic is that the calls are issued concurrently against a context that permits one at a time.

… parallel

ScoreSearchResults fans out with Task.WhenAll, and each task built a
SearchResultScorer over the same scoped IIndexerRepository and queried it for the
result's indexer. IIndexerRepository is scoped and the ListenArrDbContext behind
it is scoped, so a batch of N results issued N concurrent queries against one
context. EF rejects a second operation started on a context while another is in
flight.

The symptom is not a failed request. The scorer catches the exception and logs at
Debug, leaving indexerRetention at 0 and skipping the Usenet detection that sets
isNzb. So age and retention checks silently score against the wrong assumptions
for whichever results lost the race, and the batch comes back plausibly ordered
and quietly wrong.

Resolve each distinct indexer once, sequentially, before fanning out, and pass the
results into the scorer. Three callers reach this: the quality profile controller,
download submission, and the six-hour automatic search sweep.

This is also fewer queries rather than merely safer ones. A batch commonly carries
many results across a handful of indexers, so it goes from one query per result to
one per distinct indexer. IndexerSearchWorkflow already fetches its indexers before
its own fan-out, so this makes the two agree.

The single-result ScoreSearchResult keeps its old behaviour and still queries,
since there is nothing to batch there.

Asserting on the EF exception would mean racing it, so the test counts the overlap
directly: a stub repository records the highest number of calls in flight at once.
It reports 12 without the change and 1 with it, and also pins the query count at
one per distinct indexer rather than one per result.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant