Skip to content

refactor: simplify strategy UI and optimize Auto-K pipeline for 1,000–5,000+ notes - #41

Merged
Harsh16gupta merged 1 commit into
masterfrom
refactor/webview-strategy-ui
Aug 10, 2026
Merged

refactor: simplify strategy UI and optimize Auto-K pipeline for 1,000–5,000+ notes#41
Harsh16gupta merged 1 commit into
masterfrom
refactor/webview-strategy-ui

Conversation

@Harsh16gupta

@Harsh16gupta Harsh16gupta commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

This PR cleans up the webview UI and optimizes the clustering pipeline to scale smoothly for large note collections (500 to 5,000+ notes).

Key changes:

  • UI cleanup: Removed raw silhouette scores from the webview and defaulted the view to K-Means. Added a short description card under the dropdown explaining HDBSCAN vs K-Means.
  • Hides empty Uncategorized card: Automatically hides the empty Uncategorized card when K-Means is selected since K-Means categorizes 100% of notes.
  • Dynamic Auto-K scaling: Auto-K range now scales dynamically for larger vaults (up to 2,000–5,000+ notes) so clusters stay balanced (~25–35 notes per category) without creating mega-clusters.
  • UMAP & TF-IDF optimizations: Fixed intermediate UMAP to 5D, capped neighbors at 15, and cached TF-IDF n-grams for faster topic extraction.
  • Version bump: Updated version to 0.1.6.

@Harsh16gupta
Harsh16gupta force-pushed the refactor/webview-strategy-ui branch from 2583d25 to 6837e40 Compare August 8, 2026 21:42
@Harsh16gupta Harsh16gupta self-assigned this Aug 8, 2026
@Harsh16gupta
Harsh16gupta force-pushed the refactor/webview-strategy-ui branch from 6837e40 to b515f00 Compare August 8, 2026 21:50
@Harsh16gupta
Harsh16gupta force-pushed the refactor/webview-strategy-ui branch from b515f00 to 312efd3 Compare August 9, 2026 20:45
@Harsh16gupta Harsh16gupta changed the title refactor: simplify strategy UI and optimize clustering pipeline refactor: simplify strategy UI and optimize Auto-K pipeline for 1,000–5,000+ notes Aug 9, 2026
@Harsh16gupta

Copy link
Copy Markdown
Collaborator Author

The results which I get form the k-means algo are perfect; I like the clusters created, and how it scales to large note corpus
problem for now is that the hdbscan algo on a large corpus has a lot of notes that fall under the uncategorised category, so the next thing is to make it scale better. I will cater the hdbscan problem in the comming pr

for now we can merge this pr

@Harsh16gupta
Harsh16gupta merged commit 3de3dec into master Aug 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant