Skip to content

fix(rss): 跳过自动扫描的离线 archive 查询 + archive 流式可取消 + URL 噪音系列名 - #96

Merged
ModerRAS merged 1 commit into
masterfrom
fix/rss-scan-stall
Oct 21, 2026
Merged

ModerRAS merged 1 commit into
masterfrom
fix/rss-scan-stall

Conversation

@ModerRAS

Copy link
Copy Markdown
Owner

修复 ANi RSS 全量扫描在真机上卡死的问题,并顺手修掉同一次联调里暴露的 URL 噪音系列名。

根因(真机证据钉死,非推测)

  • ANi 全量扫描冻在 55 文件/黑色五葉草约 15 分钟,取消近 10 分钟才生效,结束后 anime/episode/index 全 0 行
  • 设备 MiruLog 埋点:bangumi.archive.search 单次 704993ms(11.7 分钟),query 仅 6 字符;986MB subject.jsonlines 当日刚下载完,前一日无 archive 时同款扫描 105 秒跑完
  • 机制:BangumiArchiveSubjectSearch 每次查询全量 parse 1GB 文件成 40 万对象 + 全排序,在 50MB 堆的 TV 盒子上 GC 泥潭;且 searchByAlias 按候选逐个调,每个候选都重复一次;且整个循环无取消点

改了什么

  1. 自动扫描跳过 archive:MetadataScraper.searchAnime/searchByAlias 新增 includeArchive = true 参数(默认保持原行为,手动搜索不受影响);ScanCoordinator enrichment 传 false。扫描恒有网(刚拉完 feed),在线搜索 0.5s 级,archive 只留给手动搜索/离线场景
  2. archive 本体加固:流式单遍 + 有界 top-k(不再物化全量 list、不再全排序)+ 每 4096 行 ensureActive(取消/超时立刻生效);findById 同样流式早退
  3. URL 噪音系列名:分类器对远程 URL 只用文件名做 PATH 证据,跳过 host/date/百分号段(之前 PATH_BERT 700 碾压干净标题 FILE_BERT 510,造出 A1%80% 这类名字并送进本就慢的查询)

验证

  • assembleDebug + test + lint 全绿
  • 新单测:includeArchive=false 跳过 archive 只走在线;100 万行文件 200ms 取消必抛;分类器 URL 用例 ×2;扫描单测断言 enrichment 全传 false
  • 现有 archive 排序/匹配单测全过(top-k + 季节降权语义等价)
  • checklist 已追加本 PR 条目 + 新 failure class(Offline-index-at-query-time)

未覆盖

  • 真机重扫验证:需要重装 debug 包会登出用户账号 + 丢配置,等用户授权再做(或等 CI 包)
  • archive 持久索引(一次性建索引、查询毫秒级)是正经解法,单独立项,不在本 PR

@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0ac91f0d-9252-4605-ae45-aabcab7927aa
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ModerRAS
ModerRAS merged commit 5ee1b6f into master Oct 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant