Skip to content

perf: deploy and retrieve flow efficiency improvements @W-24222996@ - #1836

Merged
soridalac merged 10 commits into
mainfrom
wr/sdrPerformanceImprovements
Sep 29, 2026
Merged

soridalac merged 10 commits into
mainfrom
wr/sdrPerformanceImprovements

Conversation

@WillieRuemmele

@WillieRuemmele WillieRuemmele commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

@W-24222996@

Summary

Collection of targeted performance improvements across SDR's deploy and retrieve flows:

  • Avoid materializing lazy collections: Replace getSourceComponents().toArray() with spread syntax in buildFileResponsesFromComponentSet
  • Hoist regex literals to module scope: Prevent regex recompilation on every function call in path.ts, metadataResolver.ts, and baseSourceAdapter.ts
  • Use composite key for FileResponse dedup: Replace JSON.stringify() with a string key concatenation in getFileResponses
  • Use Set for O(1) lookups: Replace linear Array.find() with Set.has() in warnIfUnmatchedServerResult
  • Remove redundant statSync guard: Replace statSync + readdirSync with try/catch around readdirSync in NodeFSTreeContainer.readDirectory
  • Index ZipTreeContainer directory children: Build a Map<dir, children[]> at construction time instead of O(n) scanning all zip entries on every readDirectory call

Benchmark results

Benchmark Before After Improvement
eda-componentSetCreate 370ms 159ms 57% faster
eda-sourceToMdapi 2438ms 1976ms 19% faster
eda-sourceToZip 2608ms 2105ms 19% faster
eda-mdapiToSource 3284ms 2841ms 13% faster
lotsOfClasses-componentSetCreate 515ms 374ms 27% faster
lotsOfClasses-sourceToMdapi 6125ms 4213ms 31% faster
lotsOfClasses-sourceToZip 7838ms 5916ms 25% faster
lotsOfClasses-mdapiToSource 6150ms 4221ms 31% faster
lotsOfClassesOneDir-componentSetCreate 861ms 493ms 43% faster
lotsOfClassesOneDir-sourceToMdapi 10950ms 7578ms 31% faster
lotsOfClassesOneDir-sourceToZip 14269ms 11612ms 19% faster
lotsOfClassesOneDir-mdapiToSource 11216ms 7839ms 30% faster

Test plan

  • All unit tests pass
  • All snapshot tests pass
  • Build passes with no lint errors
  • Performance benchmarks show improvement across all suites

Use ComponentSet.size getter instead of getSourceComponents().toArray().length.
Avoids iterating all source components and allocating a full array just
to get a count used for polling frequency calculation.
Spread the lazy collection directly instead of calling .toArray() which
allocates an intermediate array. The flatMap only needs iteration, not
random access.
Move RegExp allocations out of hot-path functions (parseMetadataXml,
getSuggestionsForUnresolvedTypes, parseAsFolderMetadataXml, and
baseSourceAdapter resolution) into module-level constants. These
functions are called thousands of times during resolution — once per
file per resolution attempt.
Replace JSON.stringify-based deduplication in getFileResponses with a
composite string key built from the response fields. Avoids serializing
every FileResponse object to JSON just to use as a Map key.
Replace O(n*m) linear scan with a Set of type#fullName keys for O(1)
lookups. Also extract manifest file names into a module-level Set.
For large deploys with hundreds of messages and file responses, this
eliminates hundreds of thousands of comparisons.
…ctory

Replace the upfront statSync().isDirectory() guard with a try/catch on
readdirSync. This eliminates one syscall per readDirectory call — and
since callers typically check isDirectory first, it was a double stat
on every directory in the resolution tree.
The method resolves API version through a multi-step fallback chain
(project config, ConfigAggregator, REST call). Cache the result to
avoid redundant lookups on repeated calls.
readDirectory previously scanned all zip entries on every call. Build
a Map<dir, children[]> in the constructor for O(1) lookups.
@github-actions

Copy link
Copy Markdown

This PR lacks any commits of the 'fix' or 'feat' type, and therefore will not trigger a release. To silence all further warnings, react to this warning comment (or any other) with the 👀 emoji.

NOTE: If your repo uses squash commits, make sure to add the appropriate conventional commit prefix (e.g., 'fix:', 'feat:', 'chore:', etc) in the squash commit message.

cs.size counts all map entries including non-SourceComponent metadata,
while getSourceComponents() filters to only resolved source-backed
components. The count must reflect actual source components for
accurate polling frequency calculation.
sourceApiVersion and apiVersion are public mutable properties with no
setter hooks, so a cached result can go stale if they are changed
after the first call. The method is only called a handful of times per
operation, so the fallback chain cost is negligible.
@WillieRuemmele WillieRuemmele changed the title perf: deploy and retrieve flow efficiency improvements perf: deploy and retrieve flow efficiency improvements @W-24222996@ Sep 17, 2026
@soridalac
soridalac merged commit c086d40 into main Sep 29, 2026
52 of 55 checks passed
@soridalac
soridalac deleted the wr/sdrPerformanceImprovements branch September 29, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants