Repository navigation
Built-in engines: faster LZMA / LZMA2 decoding (1.2–1.5×) - #1527
Merged
Merged
Conversation
Rewrite the decoder's inner loop; output, errors and limits are unchanged. - Decode into the window up to the end of the caller's buffer or of the window, then copy the block out, instead of writing every byte twice. - Literal runs and match runs are decoded by two methods that keep the range coder, input position and state in locals for the whole run. - All probabilities live in one array reached through bounded refs. - Literal and tree bits are decoded with masks instead of branches; literals read both children's probabilities before the bit is known. - While 64 or more input bytes are buffered (a symbol reads at most 48) bytes are read without checks; the rest of a chunk or stream is read by a checked copy of the same code. Stream input is topped up. - Matches copy 8 bytes at a time when they don't overlap closely. Same machine, median of 3, CPU MB/s old -> new (7-Zip 26.02 -mmt=1): text .xz 24.6 -> 36.8 (77), mixed binary .xz 35.4 -> 42.1 (82), Tesseract NSIS payload 41.3 -> 52.1 (93). Tests: LzmaDecoderTests (all lc/lp/pb, wrapping dictionaries, odd read and input sizes, truncation, 7-Zip LZMA/LZMA2 property variants). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KnifeLemon
added
the
built-in engines
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Built-in engines: faster LZMA / LZMA2 decoding, used by the 7z, xz / LZMA and NSIS readers. Same output, errors, limits and cancellation granularity.
What changed (
Archives/Lzma/LzmaDecoder.cs,RangeDecoder.cs)LiteralRun,MatchRun) so range, code, input position and state stay in registers for the whole run; with one big method the JIT spilled the range coder to the stack (checked in the disassembly).TopUp.DecodeBitis unchanged.Numbers (CPU time, median of 3, MB/s; the PC was shared, ±15 %; old/new alternated in one process are more reliable than the absolute values)
t -mmt=1-mx9, 36 MB)-mx9, 273 MB)-mx5)1.2–1.5× faster; from 2.3–3.1× slower than 7-Zip to 1.8–2.1×. Binary-heavy data gains less (frequent literal/match switches, far matches in a 32 MB window); several other layouts (all inline, match-only calls, cmov-style, branchy literals) were the same or slower.
Tests
LzmaDecoderTests(10): 7 lc/lp/pb combinations, small dictionaries that wrap the window many times, streams fed 7 bytes at a time, random read sizes, truncated streams, LZMA/LZMA2 property variants made by 7-Zip.dotnet format --verify-no-changes --severity warnclean; no OS/architecture assumptions (Linux/macOS/ARM64 checked by CI).Left
Getting to 2× needs cheaper literal↔match switches and fewer mispredicted decision bits.
Archives/Crc32.cs(byte-table CRC) is the next biggest cost in 7z reading.🤖 Generated with Claude Code