fix: regression hunt over the analysis path (watcher, batches, vision, model memory, dates, Organize confidence) by iLevyTate · Pull Request #110 · iLevyTate/StratoSortStack

iLevyTate · 2026-11-10T16:25:32Z

Regression hunt over the analysis path after #108 and #109, plus one bug reported from a screenshot (Organize showed "1% confidence" where the details panel showed 90%). 18 commits, one per group of findings; each fix has a test that fails on the unpatched source.

Fixed

Organize

A cluster-boosted match had a confidence above 1, which the card read as a percentage: 1.24 became 1%. calculateConfidence now has a ceiling of 1.
unquoteDescription had a literal s where \s was meant and cut the letter s off both ends of folder descriptions.

Smart-folder watcher

The watcher's own read of a file (NTFS last-access time, reported by chokidar 6 as a change) queued the file again during its analysis.
An embedding retry replaced a queued reanalysis or change event; lost its file when the queued item was then skipped; embedded a result for a file that had changed since; and recorded start stats that hid a real edit.
Queue, in-flight map and analysis lock use one key, so "Reanalyze all" cannot start a file beside itself.
A recording gets three hours, not ten minutes. A file deleted or renamed during analysis gets no history entry, notification or learning feedback.
A transcript the text model gave no answer for is recorded as provisional and retried.

Batches and Discover

Stop during the path check was answered "unknown batch". The run is registered before the check.
A batch is cancelled when its renderer is destroyed, crashes or reloads.
A throttled progress report is dropped when its batch has ended or was cancelled; a batch that ran to its end sends its closing report.
The history keeps the size and mtime from before the analysis.
The silent 5-minute no-progress check always pre-empted the 10-minute one that showed a notice. One check now, with the notice.
On the per-file path a slow file fails alone.

Vision

A user Stop no longer counts against the vision breaker; a timeout still does. A renderer invoke timeout counts again.
An OCR result standing in for the vision model is never cached.
Bytes that are not PNG, JPEG, GIF or BMP after a failed preprocessing step are not sent to the vision runtime. failed to load image is not transient. The runtime's stderr excerpt travels as visionRuntimeHint and is no longer appended to the error message.
An empty image batch does not enter vision batch mode.
In a batch, an image waits up to 45 s for a text request to release the text model instead of being refused the runtime.

Model memory

A text model that is still loading counts as holding the GPU (ModelMemoryManager.isLoadedOrLoading) for the embedding CPU option and the batch respawn check.
A CPU-only embedding load in a vision batch is declared beside the text model when that has replaced the runtime, and no longer evicts it.
An oversized text load stops waiting once nothing else holds memory.
Whisper runs on the CPU while the text model is still loading onto a GPU that cannot hold both.

Dates

Years 0000–0099 and years of five or six digits are rejected; 2024-05-05Z keeps its day; a numeric date from the model is a year or nothing.
Search results and cluster time ranges read a date-only document date as a local calendar day. The tests for this are checked under TZ=UTC with the call sites reverted.

Found and left

An image that arrives while a text load is still evicting the vision runtime, before the text model is reserved, can still respawn the runtime. Narrow window; the fix is to reserve before evicting, which changes the load path.
After a text load beside a CPU embedding model fails, the embedding model stays on the CPU until the next text load. Performance only.
An oversized text load evicts CPU-only residents that free no VRAM, and is refused while a small resident is pinned. Needs placement tracking in ModelMemoryManager; older than this change set.
The load-size estimate and the budget measure different memory pools (warning noise, first load under-reserved). Design change.
A provisional media entry stays unembedded after both watcher retries.
The 1 s "retrying once" delay still runs for input errors in VisionService, and VisionService itself does not refuse unknown image signatures (the caller now does).

Verification

Full Jest locally on Windows: 935 suites, 13587 passed, 4 skipped.
typecheck, typecheck:js, verify:checkjs-ratchet, verify:env-docs, Prettier and ESLint on the changed files: clean.
Not verified: no run against a real GPU with models loaded; the memory and batch fixes are covered by unit tests with a stubbed memory manager (the batch wait in particular), not by a live vision batch. No packaged build was made.

Refs #100

… folder descriptions keep their s The watcher's read of a file updates its NTFS last-access time, which chokidar 6 reports as a change for a file that was there at startup. The file was queued again while its analysis was still running. A change event for a file being analyzed is now dropped when its mtime and size are the ones the analysis started with. An image batch whose items the history already has no longer enters vision batch mode (text and embedding unloaded, vision runtime started, text loaded again) to analyze nothing. unquoteDescription's character class held a literal s where \s was meant, so Organize's target-folder cards cut the letter s off both ends of every description.

…h shows a document's day A written year of 0000-0099 was read as 1900-1999 and passed the plausibility window; a year of five or six digits passed on its first four. "2024-05-05Z" moved back a day west of Greenwich. Search results and cluster time ranges parsed a date-only document date as UTC midnight and showed the day before. Two tests depended on the zone they ran in.

…et three hours; a vanished file is not recorded - An embedding retry no longer replaces a queued reanalysis or change event for the same file (the request was lost and the old result embedded again). - Queue, in-flight map and the analysis lock use one key, so an item pushed without a dedupKey (Reanalyze all) is not started beside itself and a handed-back vision batch merges with the item queued meanwhile. - A recording gets the three hours a Discover run allows, not ten minutes. - A file deleted or renamed during its analysis gets no history entry, notification or learning feedback.

…ds with its renderer, and sends no progress after it ended - The run is registered before the path check, so a Stop that arrives during the check finds it. The check reads the disk for every file; a Stop in that time was answered 'unknown batch' and never sent again. - A batch is cancelled when the renderer that asked for it is destroyed, loses its process or loads the page again. It ran on with no owner before: the watcher waited behind it and a new Analyze ran the same files beside it. - A progress report held back by the throttle is dropped when its batch was cancelled or has ended by the time the window closes, and the renderer does not write a batch report to the store while no analysis is running. - The history keeps the size and date a file had before its analysis, not those of a version saved while the model worked.

…answer - An OCR result that stands in for the vision model is never cached, whatever made the vision model unavailable (open breaker, model that cannot take images, a failed request), so the next analysis asks the vision model again. - A Stop while the vision runtime works on a request no longer counts against the vision breaker. Five of them opened it and every image got OCR only for five minutes. withAbortableTimeout now marks the signal it aborts when the time ran out, and the request tells the two apart; a timeout still counts. - A failure the text model returns for an audio or video transcript (engine error, model not installed, unparseable answer) is recorded as provisional, as for documents, instead of as a finished transcription with confidence 0. A memory budget refusal returned that way is rethrown.

Two checks watched the same clock: one after 5 minutes that reset the run and said nothing, one after 10 that would have told the user. The first always won and disarmed the second, so the progress bar went away with no word. The 5-minute check now shows the notice and is the only one; the global watchdog keeps the hard cap. The test for an abort during the batch-busy wait aborted before the wait was entered and passed whatever the wait did. It now aborts while the wait runs, and fails when the abort listener or the timer cleanup is removed.

… GPU Two checks asked only whether the text model was in memory, while the budget already counted one that was being loaded. - An embedding load that started during the text load was offered no CPU option: it waited for the text model to go idle and then evicted it. - In a vision batch, an image that arrived during the text load respawned the vision runtime on top of it. ModelMemoryManager.isLoadedOrLoading answers for both states and does not count as using the model.

The cluster boost multiplies a suggestion's score by 1.3 for ranking, and calculateConfidence returned the higher of confidence and score without a ceiling. Organize reads a confidence above 1 as a percentage, so 1.24 became 0.0124 and the card showed 1% for the strongest matches.

…olds memory The wait for room ran to its deadline when the model's own estimate was larger than the budget, though nothing else was left to release. Optional work may use a model that is the only one loaded, and is refused while the model is being disposed. A load in progress is counted at its refined size.

…e a loading text model, slow per-file runs) - An invoke the renderer timed out counts against the vision breaker again; only a caller that gave up is left out. - A numeric date from the model is a year or nothing, never milliseconds. - Whisper runs on the CPU while the text model is still being loaded onto a GPU that cannot hold both. - On the per-file path a slow file fails alone instead of ending the run. - The no-progress notice no longer states a duration it cannot know. - JSON repair and prose extraction timeouts read once, not twice.

… report The report was held by the throttle and dropped once the run was unregistered, so the search and embedding counts that refresh on it did not. A cancelled run still sends nothing.

The history row keeps `provisional` and the reason but not the isFallback flags, and the transcript keeps its real extraction method, so the stored row failed isFallbackAnalysis and the watcher never retried it; being provisional it was never embedded either. A provisional row is now a fallback, and for this case the watcher asks whether the text model is installed rather than Whisper, which is already there.

…ale content An embedding retry carries the result of a finished pass and only embeds it. - It gave way to an item already queued for the file; when that item was then skipped as already analyzed, nothing embedded the file. The item that stays now carries the result and embeds it in that case. - It embedded its result even when the file had changed since the pass. It now compares the file with what its pass saw and drops out. - It recorded start stats as if it read the file, so a real edit during it was dropped as the watcher's own read echo. - A batch hand-back that met a queued retry kept the retry (stripped of its result) instead of the waiting item, losing the event type and mtime. Also corrects the descriptions of ANALYSIS_LOCK and GLOBAL_ANALYSIS and the watcher comments that still named a single ten-minute timeout.

…sion model Preprocessing failed on an SVG, a TIFF or a damaged WebP and the raw bytes went to the vision runtime labelled image/png. The runtime rejected them, the call was retried, and each rejection counted toward the breaker that turns vision off for the images after it. Bytes that are not PNG, JPEG, GIF or BMP by signature now skip the vision call and Tesseract; the file is analyzed from its text if it has any, else from its name, with a reason that names the file and not the model. 'failed to load image' no longer counts as a transient failure. The stderr excerpt of the vision runtime travels on the error as visionRuntimeHint and is no longer appended to its message, where it reached the UI and decided how unrelated errors were classified.

…f the text model A text load inside a vision batch evicts the runtime. The next image tried to free the text model, found it pinned by the request that loaded it, and was refused the runtime: it lost its vision analysis for a text request that ends seconds later. The image now waits up to 45 s for that request, stops waiting when it is cancelled or the app shuts down, and tries once more. The refusal stands when the text model is still in use after that.

…the text model that replaced the runtime The batch's CPU-only embedding load was declared beside the vision runtime for as long as the batch held its pin, also after a text load had evicted the runtime. With no runtime to sit beside, the load evicted the idle text model, or waited for a busy one, for a model that takes no VRAM. It is now declared beside the text model in that window and stays the batch's CPU model, unloaded when the batch ends.

The two tests for the stored-day fix compared against local midnight, which is what the old UTC parse also gives where the zone is UTC, as on CI. Each now has a case where the calendar-day helper returns an instant no parse of the stored value gives; with either call site reverted they fail under TZ=UTC.

iLevyTate added 18 commits November 10, 2026 08:01

fix(analysis): a batch that ran to its end sends its closing progress…

6bb723f

… report The report was held by the throttle and dropped once the run was unregistered, so the search and embedding counts that refresh on it did not. A cancelled run still sends nothing.

docs(changelog): record the regression-hunt fixes

6040398

iLevyTate merged commit 6f33454 into main Nov 10, 2026
10 checks passed

iLevyTate mentioned this pull request Nov 10, 2026

Known problems in v5.1.2 and earlier, fixed in the next release #100

Open

iLevyTate deleted the fix/regression-hunt branch November 10, 2026 16:47

Navigation Menu

Repository navigation

fix: regression hunt over the analysis path (watcher, batches, vision, model memory, dates, Organize confidence) - #110

fix: regression hunt over the analysis path (watcher, batches, vision, model memory, dates, Organize confidence)#110
iLevyTate merged 18 commits into
mainfrom
fix/regression-hunt

iLevyTate commented Nov 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

iLevyTate commented Nov 10, 2026

Fixed

Found and left

Verification

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant