Skip to content

feat(dir): dht only remote discovery - #1967

Open
tkircsi wants to merge 25 commits into
mainfrom
feat/dht-only-remote-discovery
Open

tkircsi wants to merge 25 commits into
mainfrom
feat/dht-only-remote-discovery

Conversation

@tkircsi

@tkircsi tkircsi commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Closes #1972

Signed-off-by: Tibor Kircsi <tkircsi@cisco.com>
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e47f92c7-27d2-4700-9237-51fdb80882be

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size/XL Denotes a PR that changes 2000+ lines label Aug 4, 2026
Signed-off-by: Tibor Kircsi <tkircsi@cisco.com>
@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

@tkircsi
tkircsi marked this pull request as ready for review August 5, 2026 07:34
@tkircsi
tkircsi requested a review from a team as a code owner August 5, 2026 07:34
@csirmazbendeguz

Copy link
Copy Markdown
Contributor

In the issue you mentioned that to migrate to v2, users need to delete the routing datastore directory.
Can we do this in a migration automatically?

@tkircsi

tkircsi commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

@csirmazbendeguz Thanks! I investigated it and yes, we can. We just need to create the datastore in subdirectory where the subdir's name is the ProtocolPrefix so v2. I'll add this to this branch soon.

@tkircsi

tkircsi commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @paralta for review!

discoveryKey in server/routing/search_remote.go picks the single deepest label out of a multi-query Search, runs one FindProvidersAsync on it, and contacts only the peers that lookup returns.

That holds for an AND query — a matching record carries every queried label, so any one of them finds the holder. But Search is OR-with-threshold, and min_match_score defaults to 1. A peer holding records that match only a different, non-selected query never advertised the chosen label, so it is never discovered, never asked, and its matches are silently missing: no error, no truncation flag.

Discovery is the only stage still assuming AND. The proto, rpc.matchingCIDs ("runs each query separately and unions the results"), rpc.QueryRecordsRequest ("Queries are OR'd") and scoreMatch all implement OR.

Worth separating from the best-effort caveat already documented in #1972. That one is Kademlia reaching whoever it can inside a budget; this is a reachable peer that discovery structurally cannot see, however long you wait.

Fix. Look up every distinct query label, run the lookups concurrently, union the peer sets. Nothing downstream changes — queryPeer already forwards the full query set. Three details: share one SearchDiscoveryTimeout across the lookups rather than per-lookup, widen the seen peer dedupe to span them, and bound the concurrency. The single-label shortcut stays valid when minMatchScore == len(queries) and could be kept behind that condition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dir/server area/dir size/XL Denotes a PR that changes 2000+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Discovery in v2: replacing GossipSub with DHT-native search

2 participants