Init reindex progress - #4
Merged
Merged
Conversation
init and reindex ran silently for seconds (about 19s on this repo, 91% of it embedding), and the only output was Hugging Face's own warning and weight-loading bar. A first-run model download printed nothing at all. - Add a Progress observer threaded through index_repository, sync_repo and the embedding backlog. It defaults to NullProgress, so MCP search-triggered syncs stay silent and stdout remains the JSON-RPC transport. - Add ConsoleProgress (rich): phases paint only after ~300ms, only the active phase animates, and each resolves to a static line. Narration goes to stderr; results stay on stdout, unchanged. A cold model download shows elapsed time, never a byte total or estimate. - Add --quiet / --progress to init and reindex; narration is otherwise on only for an interactive stderr. NO_COLOR is honoured. - Probe the local HF cache and load a cached model with local_files_only=True (about 1.1s faster per run), retrying without it if the cache is incomplete. - Mute huggingface_hub and transformers warnings and progress bars after import. - Archive the OpenSpec change and merge its deltas into the main specs.
A local path (or any non-repo-ID name) made huggingface_hub raise HFValidationError, a ValueError nothing caught, so init, reindex and search-triggered syncs ended in a traceback. _load_tokenizer has had this since M3; probe_model_cache added a second crash site. - probe_model_cache treats an invalid name as uncached instead of raising. - _load_tokenizer raises the new InvalidModelName (an EmbeddingError), which the CLI and MCP tools already render as one line. It names the setting and says local paths are not supported, rather than advising network access.
The tokenizer is fetched during the files phase, before _load_model mutes anything, so a first run on a terminal drew the hub's download bar over the live progress line.
Under GITHUB_ACTIONS, Typer forces rich's terminal mode, which styles "--quiet" as "-" + "-quiet" with escape codes between them, so the substring check failed in CI but passed locally.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
indexter init and reindex used to run silently for seconds (about 19s on this repo, most of it embedding), and the only output was Hugging Face's own warnings and progress bars. This PR adds progress narration on stderr for both commands, with one line per phase that settles into a static summary. It only appears when stderr is an interactive terminal, and --quiet or --progress override that. Results on stdout are unchanged, and MCP-triggered syncs stay silent so stdout remains the JSON-RPC channel. A cached model now loads without going online, which saves about 1.1s per run, and it falls back to downloading if the cache is incomplete. Hugging Face and transformers warnings and progress bars are muted, including the tokenizer download on a first run. An embedding_model that isn't a Hugging Face model ID, such as a local path, now gives a one-line error instead of a traceback. The PR also removes a leftover planning file and archives the OpenSpec change.