Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/macOS.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,4 @@ jobs:
-c ${{ matrix.configuration }} \
--enable-experimental-prebuilts \
--build-path .build-test-${{ matrix.configuration }} \
--filter '\.(SymbolTestsCoreDumpSnapshotTests|SymbolTestsCoreInterfaceSnapshotTests|SymbolTestsCoreCoverageInvariantTests|STCoreE2ETests|STCoreTests|GenericSpecializationTests|MultiPayloadEnumTests|MetadataReaderDemanglingTests|DependencyLoadNameTests|DependencyClosureTests|FileDependencyLocatorTests|SwiftInterfaceBuilderDependenciesTests|ManglingPrefixTests|MethodDescriptorTests|MethodOverrideDescriptorTests|MethodDefaultOverrideDescriptorTests|ProtocolRequirementTests|ResilientWitnessTests)(/|$)'
--filter '\.(SymbolTestsCoreDumpSnapshotTests|SymbolTestsCoreInterfaceSnapshotTests|SymbolTestsCoreCoverageInvariantTests|STCoreE2ETests|STCoreTests|GenericSpecializationTests|MultiPayloadEnumTests|MetadataReaderDemanglingTests|DependencyLoadNameTests|DependencyClosureTests|FileDependencyLocatorTests|SwiftInterfaceBuilderDependenciesTests|ManglingPrefixTests|MethodDescriptorTests|MethodOverrideDescriptorTests|MethodDefaultOverrideDescriptorTests|ProtocolRequirementTests|ResilientWitnessTests|LargeStackTaskExecutionTests|BoundedConcurrentMapTests|SwiftEvolutionInterfaceBuilderTests|EvolutionCommandValidationTests|DiffCommandValidationTests)(/|$)'
19 changes: 16 additions & 3 deletions AGENTS.md

Large diffs are not rendered by default.

36 changes: 36 additions & 0 deletions Changelogs/0.19.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# 0.19.0

A minor release on top of `0.18.0`. Two related changes, both output-neutral: the library's async entry points now run on swift-demangling's large-stack task executor, and multi-version preparation (`diff` / `evolution`) is parallel. Evolution proposal `large-stack-executor-and-cross-version-parallelism`; implementation note `Documentations/Internal/LargeStackTaskExecutorAdoption.md`.

## What changed

1. **Async entry points run on a 16 MB task executor.** The demangler decides per call whether to hop to its 8 MB pool by probing the calling thread's remaining stack; Swift Concurrency's cooperative threads carry 512 KB, so an async print loop paid one thread round trip per printed symbol. `MachOSymbols.LargeStackTaskExecution.run` sets swift-demangling 0.6.3's `LargeStackTaskExecutor` as the task executor preference around every library entry — indexer `prepare()`, `SwiftInterfaceBuilder.prepare()` / `printRoot()`, `SwiftDiffableInterfaceBuilder.prepare()`, the evolution builder's `prepare` / render entries, the diff renderer, the printer's four per-definition entries and the six `Dumpable.dump(using:in:)` conformers — so the whole pipeline demangles, prints and remangles inline. Hosts change nothing. Requires macOS 15 / iOS 18 / tvOS 18 / watchOS 11 / visionOS 2 at runtime; below that (or off Darwin) the body runs exactly as before. `LargeStackTaskExecution.isEnabled` (seeded by `MACHO_SWIFT_SECTION_LARGE_STACK_EXECUTOR=0`) turns it off process-wide.
2. **Cross-version preparation is parallel.** `AnySwiftEvolutionInterfaceBuilder.prepare(maximumConcurrentPreparations:)` (and the pack-generic façade) indexes the versions concurrently, defaulting to the processor count; `1` is the former oldest-first serial order. `swift-section diff` and `swift-section evolution` index their inputs the same way and gained `--jobs N`. The result is byte-identical to serial preparation; only event delivery interleaves. Intra-version parallelism is out of scope (MachOKit's file reads share one `FileHandle`).
3. **`Collection.concurrentMap(maximumConcurrency:_:)`** (`Utilities`): a windowed task-group map — source-ordered results, first failure rethrown, pending elements never started; cancelling the calling task stops submission and fails the call with `CancellationError`.
4. **Event delivery is serialized process-wide.** `SwiftIndexEvents.Dispatcher` takes one recursive lock around handler invocation, so a host handler shared by concurrently prepared versions is never entered concurrently (`Handler` still has no `Sendable` requirement).
5. **Labeled console diagnostics.** `ConsoleEventHandler(label:)` prints `[label]` after the timestamp; `diff` tags its sides `old` / `new`, `evolution` tags each version with its axis label (`AnySwiftEvolutionInterfaceBuilder.init` gained `eventHandlersPerVersion:`), snapshot inputs use their provenance label or file name. stderr only.
6. `MACHO_SWIFT_SECTION_LARGE_STACK_EXECUTOR` accepts `0` / `false` / `no` / `off` (case-insensitive) as off; anything else, or unset, is on.

## Measured

Release binaries, host dyld cache (macOS 26.5.2, 10-core Apple Silicon), two runs each; outputs byte-identical across every configuration.

| Configuration | SwiftUICore dump | SwiftUICore interface | SwiftUI dump | SwiftUI interface |
|---|---|---|---|---|
| 0.18.0 (swift-demangling 0.6.0) | 48.8 s / 48.6 s | 56.4 s / 57.0 s | 79.4 s / 79.9 s | 87.5 s / 89.4 s |
| 0.19.0, executor off | 48.6 s / 48.7 s | 56.4 s / 55.7 s | 78.0 s / 79.7 s | 88.4 s / 89.7 s |
| 0.19.0, executor on (default) | **40.6 s / 40.4 s** | **47.0 s / 47.1 s** | **61.3 s / 60.2 s** | **71.2 s / 70.9 s** |

`swift-section evolution` over three archived SwiftUI caches (macOS 15.5 / 26.5.2 / 27.0 beta 6): `--interface` 306.7 s → 151.9 s, lineage report 282.4 s → 139.4 s (executor off + `--jobs 1` → executor on + default parallelism), identical output.

## Compatibility

- Additive at the source level: no existing signature changed; `prepare()` keeps working through the new parameter's default.
- **Dependency floor**: swift-demangling `0.6.3 ..< 0.7.0` (was `0.6.0 ..< 0.7.0`). 0.6.1 re-ranked the demangler's pool QoS per hop and slowed `dump` / `interface` 3–4×; 0.6.2 fixed that; 0.6.3 adds the executor. Other pins unchanged.
- Rendered output is byte-identical (dump and interface, verified with the rendering A/B harness, executor on and off).
- Snapshot `formatVersion` unchanged.

## Requirements

- Swift 6.2+
- Xcode 26.0+
2 changes: 1 addition & 1 deletion Documentations/Evolutions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@
| [0017](0017-macho-dependencies-module.md) | 依赖闭包下沉为 MachODependencies 模块:两套依赖加载合一 | Implemented |
| [0016](0016-exported-only-interface.md) | Interface 只打印导出声明(`--exported-only`):提案 0008 标注的过滤形态,打印期按描述符 / 派生符号 / 扩展目标裁决 | Implemented |
| [0018](0018-self-contained-abi-layer.md) | ABI 层自包含:MachOSwiftSection 不再依赖符号索引——描述符只暴露实现地址,符号查询上移 SwiftInspection,值类型下沉 MachOResolving | Implemented |
| [draft](draft-large-stack-executor-and-cross-version-parallelism.md) | 大栈任务执行器接入与跨版本并行准备:打印路径零线程跳转(执行器本体在 swift-demangling 0014),diff / evolution 多版本并行 | Draft |
| [draft](draft-large-stack-executor-and-cross-version-parallelism.md) | 大栈任务执行器接入与跨版本并行准备:打印路径零线程跳转(执行器本体在 swift-demangling 0014),diff / evolution 多版本并行 | In Progress |
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Draft - 大栈任务执行器接入与跨版本并行准备

- **状态**: Draft
- **状态**: In Progress
- **作者**: JH
- **创建日期**: 2026-09-03
- **最后更新**: 2026-09-03
- **所属愿景**: 无
- **关联提案**: swift-demangling 提案 0014「大栈 TaskExecutor」(上游前置,执行器本体在那边;本提案只做接入);[0018-self-contained-abi-layer](0018-self-contained-abi-layer.md)(同一轮调研产物,互不依赖)
- **实现分支 / PR**: 待定
- **配套文档**: 待定 —— 落地时登记实现说明的链接
- **实现分支 / PR**: `feature/large-stack-executor-and-cross-version-parallelism`,[PR #122](https://github.com/MxIris-Reverse-Engineering/MachOSwiftSection/pull/122)(堆叠在 ABI 提案的 PR #121 之上,base 随其合并切到 `next`)
- **配套文档**: [LargeStackTaskExecutorAdoption.md](../Internal/LargeStackTaskExecutorAdoption.md)(实现说明);任务报告 [2026-09-03-large-stack-executor-and-cross-version-parallelism.md](../Internal/TaskReports/2026-09-03-large-stack-executor-and-cross-version-parallelism.md)

## 摘要

Expand Down Expand Up @@ -188,3 +188,11 @@ AGENTS.md(`SwiftInterface` / `SwiftIndexing` 条目补执行器一句;测试
| 2026-09-03 | 库入口自装偏好;macOS 15 以下静默回退;默认并行上限取核数 | 第二轮 |
| 2026-09-03 | 上游提案由本人在 sibling 仓库起草;上游发版号 0.6.3 | 第三轮与收尾确认;用户告知 0.6.2 已实测恢复 |
| 2026-09-03 | 上游 0.6.3 已发版,接口与四处修订对齐 | swift-demangling 会话通知:tag `8f32e30`、实现 `eaf7e76`;`StackSafeExecutor.taskExecutor` / `LargeStackTaskExecutor`;优先级映射改用 `JobPriority` 原始值、只用稳态额度、双级回退、16 MB 实测深度。状态仍为 Draft,等用户置 Accepted 后再抬 pin 与开工 |
| 2026-09-03 | Accepted → In Progress | 用户指示「基于上一个 PR 实现 async 提案」,视为批准;分支自 ABI 提案的分支切出,第一步抬 swift-demangling pin 到 0.6.3 |
| 2026-09-03 | 落地偏差:`isEnabled` 初值读环境变量 | 提案只有静态开关;加 `MACHO_SWIFT_SECTION_LARGE_STACK_EXECUTOR=0` 是为了让渲染 A/B 与计时用同一个二进制比较开 / 关,宿主不必重编 |
| 2026-09-03 | 落地偏差:并行用通用 `concurrentMap(maximumConcurrency:)` 而非 `async let` | `DiffCommand` 与 evolution 的 lineage 输入同样需要窗口化,一个 `Utilities` 帮手三处复用,`--jobs 1` 与默认走同一条代码路径 |
| 2026-09-03 | 落地偏差:CLI 的 `dump` 循环不再额外包一层 | 六个 `Dumpable.dump` 已各自包裹,CLI 逐类型进出执行器约一万跳、零点几秒,远小于原来逐符号跳转 |
| 2026-09-03 | 核对:库内零处非结构化 `Task {}` | 唯一的 `withTaskGroup` 在 `TypeIndexing.TypeDatabase`,结构化、继承偏好;落地步骤 2 无需改动 |
| 2026-09-03 | 验证:全量 1637 测试通过;计时 −16% ~ −23%(单版本)、2.0×(三版本 evolution);四种配置输出逐字节一致 | 数据见实现说明「实测数据」;0.6.0 → 0.6.3 仅抬 pin 持平,证明 0.6.1 的回归未带入 |
| 2026-09-04 | Review 修复批次(PR #122 review,15 条:真缺陷 5、误报 1、取舍 9) | 修:`concurrentMap` 取消语义(`addTaskUnlessCancelled` + 抛 `CancellationError`)、执行器测试同时挡 `isEnabled`、环境变量接受 `0/false/no/off`、并行等价测试先跑并行、三方 barrier 钉窗口宽度;用户裁定:F2 用 Dispatcher 进程级递归锁串行化投递(不改 `Handler` API)、F6 保持核数、F7 加 `ConsoleEventHandler(label:)` 与 `eventHandlersPerVersion`;F9 重复门判为不可消除(A32);其余登记 A25–A33。清单见 `Roadmaps/2026-09-04-pr122-review-findings.md` |
| 2026-09-03 | 收尾判断:写实现说明;「大栈执行器」入术语表 | 实现说明记录探测机制为何免改调用点、入口清单与嵌套免费、回退与开关、并行安全性与不做版本内并行的原因、计时表;术语在 AGENTS.md / 提案 / 实现说明 / 账本多处出现,登记 `Documentations/Glossary.md` |
7 changes: 7 additions & 0 deletions Documentations/Glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,13 @@ diff / evolution 两条对比渲染路共享结构遍历核心(`InterfaceUnion
- **主要出现在**:`Sources/SwiftInterface/InterfaceUnionWalker.swift`(协议与遍历器)、`SwiftDiffableInterfaceRenderer.swift`(`DiffUnionStrategy`)、`SwiftEvolutionInterfaceRenderer.swift`(evolution 策略)
- **延伸阅读**:[提案 0014](Evolutions/0014-unify-interface-renderers.md)

### large-stack executor(大栈执行器,`LargeStackTaskExecution`)

swift-demangling 0.6.3 起提供的 `TaskExecutor`(`StackSafeExecutor.taskExecutor`,线程栈 16 MB,`@_spi(Internals)`)。demangler 每次 demangle / print / remangle 都按**调用线程的剩余栈**决定要不要跳到它的 8 MB 线程池——协作线程只有 512 KB,探针永远不过,async 打印循环因此每打印一个符号付一次线程往返;task 跑在大栈执行器的线程上时探针每个入口都通过,全程原地执行、零跳转。本库通过 `MachOSymbols.LargeStackTaskExecution.run` 在库入口(索引器 prepare、interface builder、printer 逐定义入口、diff / evolution、dump)自装偏好,宿主零改动;macOS 15 / iOS 18 以下静默回退为原样执行。与「跳转池」(demangler 自己的 8 MB `LargeStackThreadPool`,同步 `withLargeStack` 批次用)是两个池:执行器的 job 是整段 task,会占线程上百秒,不能挤占同步跳转的额度。

- **主要出现在**:`Sources/MachOSymbols/LargeStackTaskExecution.swift`、各 async 入口的 `LargeStackTaskExecution.run { … }`
- **延伸阅读**:[LargeStackTaskExecutorAdoption.md](Internal/LargeStackTaskExecutorAdoption.md)、提案 [draft-large-stack-executor-and-cross-version-parallelism](Evolutions/draft-large-stack-executor-and-cross-version-parallelism.md)、上游 swift-demangling `Documentations/StackSafety.md` 第八节

### late-name 路径(`lateDemangledNode(forName:)`)

sweep 覆盖范围之外的名字走的旁路:demangle 后 intern 进 `Storage` 自持的一个可追加 side store,名字 → 裁决字典保证一个名字只 demangle 一次(拒绝也缓存为 `nil` 裁决、不再重试)。与主表冻结不可变的性质相对。
Expand Down
Loading