Memory, thread and lifecycle safety in the P/Invoke layer

[henrikottesorensen]

Fixes eight memory-safety defects in the P/Invoke layer, makes the native calls thread safe, and corrects two API contracts that were silently wrong. Every fix has a test that failed first for the right reason; the ones that could not be tested in-process were reproduced out-of-process and the evidence is in the commit messages.

⚠️ Breaking changes

Two, both requiring a one-line change in consuming code:

1. LibLouis no longer implements IDisposable. Replace LibLouis.Instance.Dispose() with LibLouis.Shutdown(), or — more likely — delete the call. Surfaces as CS1674.
2. TranslatedString.OutputPosition / InputPosition are sized to the strings they index and are no longer the arrays passed in. Slicing by Output.Length still works.

A consumer audit found exactly one affected call site (Translator.cs:218), and a fix for it is queued.

What was wrong

Four of these did not merely misbehave — they killed the process, which is why they had gone unnoticed rather than being reported as bugs.

Defect	What actually happened
typeform write-back overran the caller's array	liblouis writes one entry per output cell into an input-sized array. The existing SingleMode test triggered it: 15 characters in, 20 cells out, 10 bytes past the end of a pinned managed array. It passed because the corrupted memory happened not to matter.
lou_indexTables array not NULL-terminated	liblouis walks until it reads a null pointer. A string[] marshals to exactly Length pointers, so it read past the end and handed garbage to _lou_logMessage as a char*. Stack sample: lou_indexTables → _lou_logMessage. Hung the process.
lou_hyphenate output buffer marshalled as ref string	Passed a pointer-to-pointer; liblouis wrote its results over the marshalling stub's stack. Hyphenate could never have worked. Left the test host unkillable in uninterruptible exit.
Returned strings were freed	lou_version, lou_getDataPath and lou_setDataPath return pointers into liblouis's static storage. Setting DataPath aborted the process: ___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED.
Log callback delegate not rooted	Nothing held the delegate; the interop stub keeps it alive only for the registration call. After a collection: "A callback was made on a garbage collected delegate."
Lengths counted in UTF-16 code units	Every binary is built --enable-ucs4, so a widechar is a whole character. CharactersToDots("𝄞𝄞") walked 4 widechars through a 3-widechar buffer and returned 4 cells for 2 characters.
lou_free ran outside the lock	Freed the shared table chains while other threads were translating. Eight threads plus one Dispose produced no mapping for dot pattern in display table — liblouis reading a freed display table.
Version and Logging bypassed the lock	Logging.SetCallback and the LogLevel setter wrote liblouis's callback pointer and log level with no synchronisation at all.

Plus two contract corrections:

• Positions were reported in widechars but consumed as string indices — correct for BMP text, silently misaligned on the first emoji. The audit found this feeding line-breaking and hyphenation in production with no BMP validation.
• Disposal meant nothing. disposedValue was set and never read, so the singleton kept working after Dispose() by lazily recompiling the tables it had just freed. The only lasting effect was silently discarding every compiled table in the process.

One thing the audit disproved

The inlen argument passes input.Length + 1, which looks like an off-by-one that counts the NUL terminator as translatable input. Reading liblouis shows it is not: the length is clamped at the first NUL (lou_translateString.c:1191) and then overwritten with the count actually consumed (:1354) before any position mapping. It was safe — but resting on two undocumented internals, so it now passes the documented count and there are tests pinning why.

Testing

61 tests, up from 4. They document the native contracts in prose, so the next person does not have to re-derive them from the C.

Independently corroborated: a separate harness ran the upstream Danish YAML specs through this branch — 2,100 forward and 1,609 back-translation cases, zero mismatches — confirming the length and typeform changes did not perturb output.

Tests now run serially. liblouis has process-global state and is not thread safe, but xUnit parallelises across test classes; the existing suite passed only because all four tests lived in one class.

Also included

TranslatedString.OutputDots78 surfaces the per-cell dot 7/8 information liblouis reports through the typeform parameter, which the overrun fix would otherwise have discarded. It is safe to expose because the danger was where liblouis wrote it, not the data itself.