Skip to content

Investigation handoff: PaddleOCR crash in the Mac Intel CI - #1811

Closed
pa-devbot wants to merge 4 commits into
mainfrom
mac-paddleocr-crash-handoff
Closed

pa-devbot wants to merge 4 commits into
mainfrom
mac-paddleocr-crash-handoff

Conversation

@pa-devbot

Copy link
Copy Markdown
Collaborator

Draft handoff for the PaddleOCR crash in C++ CI Serial Programs Mac Intel.

The full investigation lives in MacPaddleOCRCrash_Handoff.md (crash/
control run table, log pattern, what was ruled out, code map, ranked
hypotheses, and concrete next steps including two ready-to-paste CI
snippets for capturing a stack trace).

Summary:

  • macos-15-intel segfaults (exit 139) during the first PaddleOCR ONNX
    session creation
    of the test run, ~0.1s after the
    "Set potential model cache path in session options" stdout line
    (SerialPrograms/Source/ML/Models/ML_ONNXRuntimeHelpers.cpp:28).
    CPU-only sessions (CI default USE_GPU = false); no CoreML involved.
  • Probabilistic: 3 crash observations so far (Sockets: add the Qt ServerSocket and the loopback test harness #1787 Mac Intel attempts 1
    and 2; Sockets: add the POSIX ServerSocket #1793 Mac Intel), all with three
    Test_SandwichRecipeDetector tests starting within ~2ms and the socket
    test Sockets (Qt) - Rapid reconnect still running; one run had the
    same overlap and survived. No crash on main, on Mac arm64, or on Sockets: add the WinSocket ServerSocket #1794's
    Mac Intel.
  • Ruled out: the socket PRs' code (crash is in the PaddleOCR/ONNX init
    window; socket tests all passed before it; byte-identical socket code
    passed Mac CI earlier), the GPU/CoreML path, model-cache writes (none on
    CPU), and a main regression.
  • Also documented while reading the code: a latent cross-language model
    cache race
    in ML_ONNXRuntimeHelpers.cpp (all PaddleOCR rec models
    share one cache dir + HASH.txt; clean_up_old_model_cache() does
    remove_all on hash mismatch with no synchronization, and
    ensure_paddle_ocr_instance() only serializes per language group).
    Only reachable when the cache is actually written (GPU path / user
    machines), but worth a small standalone fix.

No code changes in this PR - documentation only, for handing off to
another session.

Run IDs and gh commands for every log referenced are in the document.

Investigation findings and next steps for the segfault in
' C++ CI Serial Programs Mac Intel' during the first PaddleOCR ONNX
session creation. The crashing code is unrelated to the TCP socket
work; see the document for the evidence, code map and hypotheses.
@pa-devbot

Copy link
Copy Markdown
Collaborator Author

Closing: the investigation in #1812 found the Mac Intel exit-139 was not in PaddleOCR/ONNX Runtime at all — it was the stack-use-after-return in the Qt server socket's close_connection() completion path, already fixed on main by the shared-ownership change from #1787. This handoff doc is superseded by the findings in #1812.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant