Collect every release artifact under one convention - #433
Closed
holodorum wants to merge 5 commits into
Closed
Conversation
Included is an untested draft version of release steps that might be useful when shipping a new release of the rust artifacts to crates.io. This should be validated and corrected with Adolfo's help before carrying on.
Simplifying and improving the instructions while testing them.
The prior example was no longer accurate since requiring `options` instead of a boolean. This new example demonstrates some indentation options and also a bit more about the json->kson pipeline.
`store-native-artifacts` stored `kson-lib/build/bin/nativeKson/releaseShared`
and `releaseStatic`. Nothing writes those paths: kson-lib declares only `jvm()`
and `js()` targets, so there is no Kotlin/Native build to produce them, and all
three `build-*` jobs have been storing nothing. Meanwhile the native library
that does exist -- in `kson-lib/build/kotlin/compileGraalVmNativeImage` -- and
the CLI binary that `check` already builds on all three platforms were both
thrown away with the build directory.
Add `./gradlew packageReleaseArtifacts`, which stages this platform's share of
a release into `build/release-artifacts`:
kson-lib-shared-<arch>-<os>.tar.gz the native-image output directory, verbatim
kson-cli-<arch>-<os>.tar.gz the CLI binary
SHA256SUMS checksums for both
The `<arch>-<os>` tokens are not ours to pick. `lib-rust/kson-sys/build.rs`
downloads `kson-lib-shared-{arch}-{os}.tar.gz` from the `kson-lib-X.Y.Z` release
on kson-org/kson-binaries and unpacks it straight into its `OUT_DIR`, so the
kson-lib archive carries that directory's contents at the archive root --
unpacking it leaves exactly what `build_kson_from_source` copies there -- and
the tokens live in `BinaryArtifactPaths` beside a note naming whose contract
they are. The CLI archive follows the same convention so there is one to learn
rather than two.
CI packages inside `gradle-core-tasks`, before `verifyCleanCheckout` so every
build proves packaging leaves the checkout clean, and all six jobs now store
through one `store-release-artifacts` command into `release-artifacts/<platform>`.
Branch filters and caching are untouched.
Also fixes `:tooling:cli:buildNativeImage` declaring `kson` as its output on
Windows, where native-image writes `kson.exe`: packaging needs the real name,
and the task could never be up to date there.
`docs/release_process.md` gains a table of which job produces what and where to
download it, and fills in the collection half of the lib-rust and tooling/cli
sections. Publishing the CLI stays a todo: no distribution channel has been
chosen for it.
The lib-rust process this builds on (kson-org#423) opened with a workaround for exactly
the gap closed here: download three Python wheels and unzip `jni_simplified.h`
and the platform library out of each. Its Step 1 becomes three downloads of
archives CI now builds ready-made, and the facts around it are corrected with
it -- the libraries come from the `build-*` jobs rather than
`test-python-sdist-*`, they land in `release-artifacts/<platform>`, and an
archive holds the native-image output directory verbatim (four files on macOS)
rather than the two a wheel could supply. That document's publish ordering,
`KSON_LIB_VERSION` check, crates.io steps and their warnings are kept as they
are.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xh2RGwbRjLofpWRJK9cT5z
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
store-native-artifactsstoredkson-lib/build/bin/nativeKson/releaseSharedandreleaseStatic.Nothing writes those paths — kson-lib declares only
jvm()andjs()targets — so all threebuild-*jobs have been storing nothing, while the native library that does exist (inkson-lib/build/kotlin/compileGraalVmNativeImage) and the CLI binary thatcheckalready builds onall three platforms were both thrown away with the build directory.
A new
./gradlew packageReleaseArtifactsstages this platform's share of a release intobuild/release-artifacts:kson-lib-shared-<arch>-<os>.tar.gzkson-cli-<arch>-<os>.tar.gzSHA256SUMSThe
<arch>-<os>tokens are not ours to pick.lib-rust/kson-sys/build.rsdownloadskson-lib-shared-{arch}-{os}.tar.gzfrom thekson-lib-X.Y.Zrelease onkson-org/kson-binariesand unpacks it straight into its
OUT_DIR, so the kson-lib archive carries that directory'scontents at the archive root — unpacking it leaves exactly what
build_kson_from_sourcecopiesthere. The tokens live in
BinaryArtifactPathsbeside a note naming whose contract they are, withtests pinning the spellings against
build.rs. The CLI archive follows the same convention so thereis one to learn rather than two.
CI packages inside
gradle-core-tasks, beforeverifyCleanCheckout, so every build provespackaging leaves the checkout clean; all six artifact-producing jobs now store through one
store-release-artifactscommand intorelease-artifacts/<platform>. Branch filters and caching areuntouched.
Also fixes
:tooling:cli:buildNativeImagedeclaringksonas its output on Windows, wherenative-imagewriteskson.exe: packaging needs the real name, and the task could never be up todate there.
What this means for #423
#423's Step 1 is a workaround for exactly the gap this closes — it has the release manager download
three Python wheels and unzip
jni_simplified.hplus the platform library out of each. Thosearchives now come out of CI ready-made, so this PR rewrites that step and corrects the facts around
it:
forloop is gone. TheKSON_LIB_VERSIONcheck (and its "every downstreamcargo build404s" warning), the localKSON_PREBUILT_BIN_DIRsanity build, and thegh release createinvocation all stay.build-linux-amd64/build-macos-arm64/build-windows-amd64intorelease-artifacts/<platform>, not fromtest-python-sdist-*intopython-*. Every name was checked against.circleci/config.ksononthis branch.
jni_simplified.hplus thelibrary. The CI archive is the native-image output directory verbatim — on macOS that is
graal_isolate.h,graal_isolate_dynamic.h,jni_simplified.handlibkson.dylib.build-python-and-testto a greenbuild-all, which is the workflow whose jobs build these libraries.#423's publish ordering, prerequisites,
cargo publish --dry-runsteps, the "published crates canonly be yanked, never changed" warning and the crates.io verification are kept as they are.
Deliberate gaps
kson-org/kson-binariesstays a documented manualgh release create.todo in the doc, because nothing has been chosen.
build.rsmaps tokens forarm64/amd64acrosslinux/macos/windows, but CI produces onlyamd64-linux,arm64-macosandamd64-windows.The other three —
arm64-linux,amd64-macosandarm64-windows— 404 on download, andconsumers there must set
KSON_ROOT_SOURCE_DIRorKSON_PREBUILT_BIN_DIRthemselves (the docsays so). Adding jobs for them is a separate change.
Verified locally (macOS arm64)
cd buildSrc && ./gradlew check— green, including 9 new tests (Sha256SumsTaskTest,BinaryArtifactPathsTest)./gradlew check— green on this base (146 tasks), so replaying ontorust-releasebrokenothing
./gradlew transpileCircleCiConfigTask— regeneratesconfig.ymlwith no diff, so it iscommitted in sync with
config.kson./gradlew packageReleaseArtifacts— produceskson-lib-shared-arm64-macos.tar.gz(four files,library mode
0755),kson-cli-arm64-macos.tar.gz(kson, mode0755) andSHA256SUMS;shasum -a 256 -c SHA256SUMSpasses./gradlew verifyCleanCheckout— clean, both before and after packagingkson-lib-shared-arm64-macos.tar.gzandbuilding with
KSON_PREBUILT_BIN_DIRpointed at it compileskson-sysandkson-rs, so thearchive really is a drop-in for what
build.rsexpectsLinux and Windows packaging is unverified locally — that is what the
build-*jobs on this branchare for.