Current state
pm_optimize writes the network as one JSON line to the Julia subprocess's stdin; Main.jl
prints the result JSON to stdout, interleaved with debug logs, printlns and Gurobi
banners. Python identifies the result line by out.rstrip().startswith('{"name"').
Why this is fragile
- Result detection depends on unguaranteed JSON key order.
JSON.json(::Dict) serializes
in hash/iteration order — "name" being the first key is coincidence, not contract. If the
result line ever starts with another key (different Julia/JSON.jl version, changed key set),
the startswith branch misses it: from_powermodels is never called, no results are
written to the eDisGo object, the raw JSON is echoed to stdout, and the run still logs
"Julia process was successful." — silent loss of results on a successful solve.
- Logs and payload share one channel, separated only by prefix heuristics (
"Set parameter",
"Academic").
- The input contract ("exactly one JSON line on stdin", read via
readline) is implicit and
unenforced; large inputs are written in full before stdout is drained (OS pipe-buffer
deadlock risk for very large networks).
Proposal
Exchange input and result as files (paths passed as arguments, e.g. into the existing
results directory or a temp dir): Python writes <run>_input.json, Julia writes
<run>_result.json; stdout remains purely a log channel. This removes the key-order trap, the
log/payload mixing and the pipe-buffer risk in one step.
Interim hardening (if stdout must stay temporarily): wrap the result in an unambiguous sentinel
(e.g. ###RESULT###{…}) instead of keying on {"name".
Current state
pm_optimizewrites the network as one JSON line to the Julia subprocess's stdin;Main.jlprints the result JSON to stdout, interleaved with debug logs,
printlns and Gurobibanners. Python identifies the result line by
out.rstrip().startswith('{"name"').Why this is fragile
JSON.json(::Dict)serializesin hash/iteration order —
"name"being the first key is coincidence, not contract. If theresult line ever starts with another key (different Julia/JSON.jl version, changed key set),
the
startswithbranch misses it:from_powermodelsis never called, no results arewritten to the eDisGo object, the raw JSON is echoed to stdout, and the run still logs
"Julia process was successful." — silent loss of results on a successful solve.
"Set parameter","Academic").readline) is implicit andunenforced; large inputs are written in full before stdout is drained (OS pipe-buffer
deadlock risk for very large networks).
Proposal
Exchange input and result as files (paths passed as arguments, e.g. into the existing
results directory or a temp dir): Python writes
<run>_input.json, Julia writes<run>_result.json; stdout remains purely a log channel. This removes the key-order trap, thelog/payload mixing and the pipe-buffer risk in one step.
Interim hardening (if stdout must stay temporarily): wrap the result in an unambiguous sentinel
(e.g.
###RESULT###{…}) instead of keying on{"name".