packaging: make tracked-source sdist and wheel builds reproducible

[rickisba]

Motivation

• Fix release-chain break where building a wheel from a sdist (created from tracked source) failed with error: package directory 'data/CacheRoute_dataset' does not exist, caused by explicit setuptools package entries that pointed at repository data/static/native directories instead of tracked Python packages.
• Provide auditable, deterministic tracked-source build paths (tracked export -> wheel, tracked export -> sdist -> wheel) and strengthen package-map governance so wheel tests no longer rely on untracked or ignored files.

Description

• Update pyproject.toml to remove explicit package declarations that represent repository/runtime data, static assets, native sources or calibration data rather than tracked Python packages (examples removed: data.CacheRoute_dataset*, kdn_server.KV_database/text_database*, UI.*.static/templates as package entries, instance.TTFT_predictor.data, proxy.metrics.data, instance.resource_agent.src), while preserving reviewed package-data rules and keeping explicit discovery in place.
• Strengthen governance tests in test/test_repository_governance.py by adding path-mapping helpers and test_explicit_packages_have_reviewed_tracked_python_sources() that assert every [tool.setuptools].packages entry maps to tracked Python source under the repository; also replace directory-walk root discovery with tracked-file-based discovery.
• Replace the raw-worktree wheel test with a deterministic tracked-source flow in test/test_wheel_install.py: export the exact tracked revision via git archive HEAD, assert the export matches git ls-tree tracked files, build a direct tracked-source wheel, create an sdist from the tracked export, unpack the sdist in a clean tree, and build the final wheel from that unpacked sdist (no .git metadata). Add fixtures to exercise both direct and sdist-derived wheel build paths and assert final wheel contents and isolated-install module origins.
• Update maintained developer-handbook docs to document the distinct validation layers (active checkout, tracked export, sdist, sdist-derived wheel, isolated install) and to clarify that package discovery remains explicit and only directories with reviewed tracked Python source should be declared as packages (doc/developer-handbook/development-and-validation.md and doc/developer-handbook/package-and-module-map.md).
• Files changed: pyproject.toml, test/test_repository_governance.py, test/test_wheel_install.py, doc/developer-handbook/package-and-module-map.md, doc/developer-handbook/development-and-validation.md.
• Pre-fix failure reproduced: building a wheel from a clean sdist unpack failed with error: package directory 'data/CacheRoute_dataset' does not exist because pyproject.toml still declared that package while the sdist (created from tracked files) did not include non-Python dataset directories.

Final head SHA: a96938de1131f7fe8db7108ac7ada2a0e8eebc58 (branch codex/187-reproducible-packaging).
Base/main SHA used for merge-base checks: 5b9278cb7c9bbc82df734bbe00c8cf6ca4b6571a.

Testing

• Built and audited the tracked-source release chain: exported tracked files via git archive HEAD, produced an sdist (cacheroute-0.1.9.tar.gz) and built a wheel from the unpacked sdist; recorded artifacts: sdist cacheroute-0.1.9.tar.gz (503,430 bytes, SHA-256 e8b2293b9a265237fdbe5acd9bd567d5f65235595c41f5aa08240912592e06ce) and final wheel cacheroute-0.1.9-py3-none-any.whl (539,002 bytes, SHA-256 6c3290bde4e6936bd6a5586428dab81bf36cc15d2b08dda9de6ba9764b874030).
• Automated checks and test runs (executed from the final committed head) passed: python3 -m compileall -q src test scheduler proxy instance kdn_server (OK), pytest suites: test/test_repository_governance.py (16 passed), test/test_namespace_layout.py (10 passed), test/test_source_checkout_imports.py (2 passed, warnings), test/test_wheel_install.py -m "not network" (4 passed, 1 network test deselected), test/test_documentation_governance.py (12 passed), test/test_contract_foundation.py (5 passed), test/test_contract_service_migration.py (11 passed). The sdist->wheel build-from-unpacked-sdist and isolated-install assertions passed in the release-artifact tests.
• Environment-blocked items and notes: running python3 -m build --sdist --no-isolation in this environment was ENVIRONMENT-BLOCKED (no build module installed by default) and publishing the branch / creating a GitHub PR was ENVIRONMENT-BLOCKED due to HTTP/GitHub auth/network restrictions; these were not auto-installed to preserve environment safety. The network-marked full dependency wheel test (CACHEROUTE_RUN_NETWORK_TESTS=1) was intentionally not run here (deselected) and should be executed by CI or a maintainer with the required network/wheelhouse available.

Closes #187
Refs #137
Refs #142
Refs #180
Refs #184
Refs #185
Refs #186

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89f2a2c701

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Skip release-chain checks in Git-free sdists

When this test is run from the unpacked sdist, which includes test/test_wheel_install.py but intentionally has no .git directory, this command exits with fatal: not a git repository before any wheel assertion runs. I reproduced that failure from the sdist generated by this commit; either mark this checkout-only test as skipped when Git metadata is absent or structure the fixture so the shipped test suite can validate an unpacked source distribution without resolving HEAD.

Useful? React with 👍 / 👎.