Make the OpenLibrary fallback reachable - #861
Open
krejko wants to merge 3 commits into
Open
Conversation
AsinCandidateCollector passed the composed query straight through as the
OpenLibrary title:
var books = await _openLibraryService.SearchBooksAsync(query, null, 5);
The composed query still carries its field prefixes, so the value was
"AUTHOR:Abelson TITLE:Structure and Interpretation of Computer Programs".
OpenLibrary's normalizer tokenizes on [a-z0-9-]+, which drops the colons and
leaves "AUTHOR" and "TITLE" behind as literal search terms:
q=AUTHOR:Abelson TITLE:Structure and Interpretation... -> numFound 2
q=AUTHOR Abelson TITLE Structure and Interpretation... -> numFound 0
q=Abelson Structure and Interpretation... -> numFound 2
With no documents returned, OpenLibraryDerivedResults stayed empty and the
existing fallback that merges OpenLibrary candidates when no enriched metadata
was found could never fire. Books with no Audible edition silently returned no
results at all.
Pass the fields the caller already parsed - SearchService logs them one line
before the call - and use the author too, which was previously discarded as
null. Callers that only have a composed string get the prefixes stripped
instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The candidate loop skipped any document whose title equalled the search term:
if (!string.IsNullOrEmpty(book.Title) && !string.Equals(book.Title, query, ...))
An exact title match is the strongest candidate available, so this discarded the
best result for every book whose catalogue title matches what was searched for.
Searching "Radicalized" by Cory Doctorow returns six OpenLibrary documents, the
first two titled exactly "Radicalized"; all of them were dropped and the search
returned nothing. "The Garden of Rama" kept only a four-book Rama omnibus,
because the omnibus title differs from the query while the actual book's does
not.
Keep every document with a title and let the existing scoring and dedupe decide.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The query was assembled by joining tokens with "+" and then escaping the whole
string. Uri.EscapeDataString encodes "+" as %2B, so OpenLibrary received literal
plus characters between the words rather than separators, and matched almost
nothing:
q=radicalized%2Bcory%2Bdoctorow -> numFound 0
q=radicalized cory doctorow -> numFound 6
q=the%2Bgarden%2Bof%2Brama%2Barthur%2Bc%2Bclarke -> numFound 1
q=the garden of rama arthur c clarke -> numFound 5
The single Rama hit was a four-book omnibus whose title happened to survive the
corrupted query, which is why the fallback looked like it was working
occasionally rather than failing outright.
Join with spaces and undo the plus-joining that NormalizeForOpenLibrary applies
to its own tokens.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SearchServicemerges OpenLibrary-derived candidates when no enriched metadata was found, so books with no Audible edition still produce results:That branch can never be taken. Three independent defects upstream of it leave
OpenLibraryDerivedResultsempty, so the fallback silently does nothing and such books return zero results.The three defects
1. The composed query is used as the OpenLibrary title.
NormalizeForOpenLibrarytokenises on[a-z0-9\-]+, dropping the colons and leavingAUTHOR/TITLEas literal search terms. The author is also discarded asnull, despite the caller having parsed it.2. Exact title matches are discarded.
An exact title match is the strongest candidate available. Searching
Radicalizedby Cory Doctorow returns six documents, the first two titled exactlyRadicalized— all dropped.The Garden of Ramakept only a four-book omnibus, precisely because the omnibus title differs from the query while the actual book's does not.3. Query terms are joined with
+, then URL-escaped.OpenLibrary receives literal plus characters between words. Measured against the live API:
radicalized%2Bcory%2Bdoctorowradicalized cory doctorowthe%2Bgarden%2Bof%2Brama%2Barthur%2Bc%2Bclarkethe garden of rama arthur c clarkeThis is the dominant one — it corrupts every OpenLibrary query the app makes.
Changes
Fixed
SearchBooksAsync(SearchServicelogs both one line before the call); stripFIELD:prefixes for callers that only have a composed string.+.Added
AsinCandidateCollector_OpenLibraryQueryTests— parsed fields preferred, prefixes stripped when absent, plain queries untouched, and no call when OpenLibrary is disabled.Testing
34/34 pass, including the existing parser suite.
Verified end-to-end against a running instance. Log lines that never appeared before now do:
Real searches, before → after:
Radicalized/ Cory Doctorow'Radicalized'firstThe Garden of Rama/ Arthur C. Clarke'Rama Series, Collection Set of 4 Books…''The Garden of Rama'Structure and Interpretation…/ AbelsonA control case still correctly returns nothing:
Chronicles of Narnia Introis an intro track rather than a book, and OpenLibrary has no such title (numFound=0); dropping "Intro" returns 234.Notes
Each defect masked the next, so fixing any one alone leaves the symptom unchanged — I confirmed each independently against the live OpenLibrary API before and after.
Behaviour is unchanged when
EnableOpenLibrarySearchis off, and no Audible/Audnexus path is touched; this only affects queries that would otherwise have returned nothing.🤖 Generated with Claude Code