Repository navigation
Prepare v0.2.0 - #11
Draft
3leapsdave wants to merge 33 commits into
Draft
3leapsdave wants to merge 33 commits into
3leapsdave wants to merge 33 commits into
Conversation
Rust: - quick-xml 0.42, zip 8 (deflate decode backend only), scraper 0.27, time >= 0.3.47, rsfulmen =0.2.0 - jsonschema 0.57 without default features; test validators build an offline registry - base64 0.23 (dev) - Track Cargo.lock; cargo invocations in Makefile, CI and scripts use --locked - Pin toolchain 1.94.1 in rust-toolchain.toml; MSRV stays 1.88.0 with a CI job that builds and tests on it TypeScript bindings: - napi 3, napi-derive 3, @napi-rs/cli 3, TypeScript 6 - tsconfig moves to Node16 module resolution with explicit node types - Track package-lock.json Gates: - make check now includes cargo-audit; cargo-deny runs every check - deny.toml: sources deny, graph targets for shipped platforms, bans on OpenSSL/native-tls and quick-xml < 0.41 - Direct clippy runs alongside goneat v0.6.0; sfetch pinned to v0.4.11 and bootstrap installer arguments corrected - CI actions moved to current majors; runner image pinned by digest; cbindgen, cargo-deny and cargo-audit versions pinned Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Archive part reads now cap the bytes actually decompressed instead of relying on the size declared in the archive header, and buffer pre-allocation no longer follows the declared size. Archives carry a total decompression budget (4x the per-part limit) shared by every part read from them; exceeding either limit is a ResourceLimit error. open_archive now returns an OoxmlArchive wrapper that holds the budget. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
- docprims-ooxml: malformed, truncated and non-archive inputs across all three formats; entry-count limit; attribute-count scaling; deep nesting; out-of-range shared strings; entity decoding; non-ASCII text and block byte ranges; understated entry sizes - TypeScript: v0 goldens through the native addon; Uint8Array and offset views; error-code mapping; loader error without an addon - Go: v0 goldens through cgo; error-code mapping including zero-length input; per-thread error messages under concurrent calls Fixtures are generated in-test. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Remove `ExtractOptions::timeout_ms` and the CLI `--timeout-ms` flag, which were accepted but never applied. Update the architecture threat table to list the limits that are enforced. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Decimal and hexadecimal character references in OOXML text are now resolved instead of dropped, matching the XML text extractor. References that do not name a Unicode scalar value are still dropped. This changes extracted text for documents that contain numeric character references. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
XML and OOXML extraction now share one rule for numeric character references: a reference resolves only if it names an XML 1.0 Char (tab, line feed, carriage return, U+0020-U+D7FF, U+E000-U+FFFD, U+10000-U+10FFFF). Other references, including NUL and other control characters, are dropped. The rule lives in docprims_core::xml. This changes extracted text for documents that contain such references. Markdown and HTML handling is unchanged and pinned by tests. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Every extractor now drops characters outside the XML 1.0 Char set from its text before block byte ranges are computed, so literal control bytes and HTML or Markdown references to them no longer reach the output, and block ranges still slice the document text exactly. The legacy ExtractedText API applies the same filter. This changes extracted text for documents that contain such characters. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Extracted text now also excludes U+007F-U+009F, which XML 1.0 allows but discourages. The exclusion is a separate predicate (docprims_core::xml::is_output_char), so is_xml_char still matches XML 1.0 exactly. HTML references in that range that map to Windows-1252 characters are kept as those characters. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Add a `docprims` crate as the entry point for the library. It re-exports the extract/v0 types and provides extraction by file or bytes with format dispatch by extension (`Format`), behind per-format features (`markdown`, `html`, `xml`, `docx`, `xlsx`, `pptx`, grouped as `text` and `ooxml`). The command-line tool moves into this crate behind a `cli` feature; the binary name and output are unchanged. `docprims-cli` is removed. Workspace targets build with all features, and `make lean-lib-check` verifies library builds pull no CLI dependencies and no dependencies of disabled formats. `docprims-ooxml` now builds with any subset of its format features. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
- C and TypeScript bindings extract through `docprims::extract_file` and `docprims::extract_bytes`; results and error codes are unchanged - Remove `ExtractOptions` from docprims-core; make `docprims_ooxml::common` private and hide `docprims_core::xml` from docs - Document that format dispatch is by declared extension or explicit `Format`, never by content, with tests - `UnsupportedFormat` names the feature to enable - Document literal zero limits in Rust and TypeScript and zero-means- default in Go - Subcrate docs point to `docprims` as the supported entry point; README covers the library, features and `cargo install docprims --features cli` - Add `make doc-check` (docs.rs-style build, warnings denied) and a minimal-feature test run to `make lean-lib-check` - Split long runner image lines in workflows Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Re-export `DocprimsQualityStatus` and `DocprimsContainerKind`, the types of public fields in the extract/v0 output, so they can be named without depending on docprims-core. `DocprimsQualityStatus` gains `PartialEq` and `Eq`. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
When the output byte limit leaves no room for any of a block's text, stop without emitting an empty block or a trailing separator. Previously a limit of 0, or truncation at a multi-byte character, produced a block with empty text. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
`DocprimsError`, `DocprimsQualityStatus` and `DocprimsContainerKind` can gain variants, so matches on them outside docprims-core need a wildcard arm. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
The TypeScript compiler binary is an optional dependency of the typescript package, so workflows install Node dependencies with `npm install` instead of `npm install --omit=optional`. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Also record the non_exhaustive public enums. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
napi-rs 3 writes .node files next to the addon's Cargo.toml unless --output-dir is given. Build straight into prebuilds/ and fail if the expected file is missing. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
The workspace defaults to publish = false. docprims, docprims-core, docprims-text and docprims-ooxml opt in and carry docs.rs, README and license metadata; docprims-ffi and docprims-ts-napi stay unpublished. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Set the version to 0.2.0 across the workspace, npm manifest and lock files, and move the unreleased notes under 0.2.0. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Adapted from the storageprims release tooling. - config/release/publishable-crates.txt is the ordered crates.io list; release-crates.py checks it against Cargo metadata, and docprims-ffi and docprims-ts-napi must stay unpublished - check-packages.sh packages and verifies the listed crates without publishing; release-crates-dry-run.sh and release-crates-verify.sh cover the ordered publish from a guarded tag - version-sync.py writes VERSION to Cargo.toml, Cargo.lock and the npm manifests; check-version.sh checks every one of those sites - release-guard-tag-version.sh binds the release tag to VERSION and, in strict mode, to origin/main - check-release-notes.sh requires docs/releases/vX.Y.Z.md to match its RELEASE_NOTES.md section - make release-preflight runs these before tagging; prepush runs the tooling tests; help is generated from target comments - RELEASE_CHECKLIST.md gains the crates.io steps Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Adapted from the storageprims release tooling. - Release tags are signed by a subkey of the committed public pin, bound to VERSION and origin/main, and checked against the tag ruleset; public fingerprint anchors are generated and verified with Decernor - The release asset set is defined by config/release/cli-platforms.txt; CLI archives must hold exactly the binary and both licenses as top-level regular files - make release runs the serialized signing ceremony against the exact unsigned draft; CI never holds signing keys - The release workflow can run manually to build and validate the full asset set without releasing; on a tag push it verifies the signed tag before opening the draft, and adds a CycloneDX SBOM - RELEASE_CHECKLIST.md and docs/cicd.md describe the flow Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Every workflow pins actions to a commit SHA, names its Rust toolchain, declares read-only contents permission at the top level and sets a concurrency group. ci gains a publish-check job that packages the crates and runs the release tooling tests with a checksum-pinned Decernor. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
The committed package.json no longer lists the platform packages as optionalDependencies, so the lock file is complete and every workflow installs with npm ci. The publish workflow writes optionalDependencies for exactly the platform packages it publishes, at the root version, with scripts/set-platform-packages.js. check-version.sh rejects optionalDependencies in the committed manifests. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
The dependency license list mirrors deny.toml. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
extract/v0 is classified Evolving. The page states the output guarantees every result meets and that extracted text carries the sensitivity of its input document. The architecture overview carries the same character-set guarantee and now shows parser dispatch by extension or explicit format. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
set-platform-packages.js maps each napi target to its platform with napi's parseTriple and requires the package directory to match exactly, each package to carry its addon, name and version. --check verifies without writing. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
The prebuild and publish workflows verify platform packages with set-platform-packages.js instead of their own platform lists. Generated by Claude Opus 5.5 via Claude Code under supervision of @3leapsdave Co-Authored-By: Claude Opus 5.5 <noreply@3leaps.net> Role: devlead Committer-of-Record: @3leapsdave
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepares the v0.2.0 release.
docprimscrate as the library entry point; the CLI and the C and TypeScript bindings now extract through it.🤖 Generated with Claude Code under supervision of @3leapsdave
Role: devlead