Skip to content

fix(core): check the lockfiles on watch events instead of every 20 ms - #37113

Open
sdjayna wants to merge 1 commit into
nrwl:masterfrom
sdjayna:sj/daemon-outdated-check-events
Open

sdjayna wants to merge 1 commit into
nrwl:masterfrom
sdjayna:sj/daemon-outdated-check-events

Conversation

@sdjayna

@sdjayna sdjayna commented Sep 18, 2026

Copy link
Copy Markdown

The daemon's outdated check moves from a 20 ms setInterval onto the workspace watch and the client connection, and the interval goes. An idle daemon stops re-reading and hashing the lockfiles 34 to 50 times a second; restart behaviour is unchanged.

idle daemon, 5 minutes, daemon process only Mac (M4 Max) before after Windows 11 laptop before after
CPU, share of one core 5.6% 0.15% 10.55% 0.53%
reads / s 217 0
MB / s read 50.6 0
file creates / s (WPR FileIO) 279 0

Workspace: 1,188 projects, yarn.lock 1.69 MB. A lockfile rewrite on the patched daemon logged lock file hash changed, LOCK_FILES_CHANGED, Restarting daemon... within a second of the write; an unrelated write logged nothing.

Current Behavior

server.ts starts a setInterval(..., 20) after writing the process file. Every tick: getDaemonProcessIdSync() (read of server-process.json), isNxVersionMismatch() (resolve and read the installed nx/package.json), then lockFileHashChanged(): five existsSync, hashFile() of each lockfile present, hashArray. With a 1.69 MB yarn.lock that is 2.34 GB read in 41 s on an idle Windows laptop, every open passing through the filter drivers. On a corporate image with three of them, about 505 µs per create between them, the loop costs about 140 ms of filter-driver time per second of idle before the daemon's own CPU.

Expected Behavior

  • Lockfiles on the watch. recordLockFileHash() runs once in startServer, before the watch is set up, so any change the watch reports is a real change. routeAppliedChanges in project-graph-incremental-recomputation.ts, which already receives every applied batch and already calls restartDaemonIfIgnoreFilesChanged, now calls restartDaemonIfLockFilesChanged(paths) beside it. Only a batch naming package-lock.json, yarn.lock, pnpm-lock.yaml, bun.lockb or bun.lock at the root reads the lockfiles. A restart goes through handleServerProcessTerminationWithRestart with the same log lines.
  • Dropped events. The native watch re-walks the workspace on a rescan and hands the diff out as an ordinary batch, so a lockfile write lost in the drop still reaches the check. The process file is outside the walk, so stopDaemonIfReplaced (event-driven since feat(core): add includeIgnored filesets and let the workspace context own the daemon's watch #37025) now also runs on the rescan marker.
  • Connection. stopDaemonIfOutdated() runs once per client connection: process file, installed version, lockfile hash. It returns early and the shutdown destroys the new socket with the others; the client reconnects to whichever daemon comes next, as today when the interval fired mid-request. This is the safety net for a change the watch never reported.
  • The interval, daemonIsOutdated(), lockFileHashChanged() and existingLockHash are removed from server.ts. No timer replaces them.

Implementation Notes

  • Files: restart-checks.ts (+120), restart-checks.spec.ts (+152), server.ts (-91), project-graph-incremental-recomputation.ts (+9).
  • Tests. restart-checks.spec.ts: 14 pass, including: a root lockfile change restarts with LOCK_FILES_CHANGED; paths that do not name a root lockfile (including pkg/yarn.lock) read nothing; an unchanged rewrite reads once and does not restart; stopDaemonIfOutdated for nothing changed, lockfile changed unreported, another process owning the process file, installed version differing; stopDaemonIfReplaced on rescan with and without ownership. vitest run src/daemon: 19 files, 282 passed. tsc --noEmit and eslint clean on the four files.
  • Measurement. Five-minute idle samples of the daemon process only, before and after, on both machines; on Windows also an 81.5 s Windows Performance Recorder FileIO trace: node.exe 22,770 creates and 17,712 reads before, no events after; WdFilter delay across all processes 1.19 s before, 0.24 s after.

Related Issue(s)

Fixes #37112

#37025 made the process-file check event-driven; this removes the poll that ran beside it. #37086 and #37088 concern what a rescan can hide from the outputs tracker; this change relies on the watch's re-walk already on master, not on those PRs.

@sdjayna
sdjayna requested a review from a team as a code owner September 18, 2026 13:03
@netlify

netlify Bot commented Sep 18, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit b435eb5

@netlify

netlify Bot commented Sep 18, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-dev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit b435eb5

@AgentEnder

Copy link
Copy Markdown
Member

Thanks for the PR. Approving the workflow run so CI can actually build it, and assigning it for review.

Triage notes (automated triage, reviewed before posting)

The head commit here has no approved workflow run, so no GitHub Actions check runs exist against it. On a fork pull request the run waits for a maintainer to approve it, and until that happens the green tick comes from third-party checks only, so it says nothing about the build. Approved as part of triage.

Closes #37112. Branch merges cleanly.

Routed to the daemon owner rather than the scope rotation, since the change is in the daemon watch path.

@nx-cloud

nx-cloud Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit 29cf282

Command Status Duration Result
nx affected --targets=lint,oxlint,test,build,e2... ⛔ Cancelled 1h 38m 51s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 4s View ↗
nx-cloud record -- pnpm nx-cloud conformance:check ✅ Succeeded 1m 2s View ↗
nx build workspace-plugin ✅ Succeeded <1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 20s View ↗
nx-cloud record -- nx format:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2026-09-20 07:08:21 UTC

The daemon's outdated check ran on a 20 ms interval. Every tick read the
process file, resolved the installed nx version and hashed every lockfile
at the workspace root. With a 1.7 MB yarn.lock an idle daemon read the
file 34 to 50 times a second: 8 to 12% of a core and 57 MB/s on a
Windows laptop, 5 to 6% of a core on an M4 Mac, for a daemon doing nothing.

The watch already names every changed path. The lockfile hash is now
recorded once, before the watch starts, and compared only when a batch
names a lockfile at the root; a change the kernel dropped arrives as an
ordinary batch after the re-walk. The process-file check already ran on
the watch event for that file, and now also runs when the watch reports
dropped events, since that event may have been among them. A client
connection runs all three checks once before the daemon serves it, so a
change the watch never reported is still caught at the next request.
The interval is gone.
@sdjayna
sdjayna force-pushed the sj/daemon-outdated-check-events branch from 29cf282 to b435eb5 Compare September 21, 2026 20:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: core core nx functionality type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Daemon hashes every lockfile at the workspace root every 20 ms while idle

3 participants