JOSS submission: tracking issue for the four desk-reject gates (target late October 2026)

[shivamlalakiya]

Umbrella issue for the JOSS submission. Nothing is implemented here; every box links to the issue
or PR that carries the work. Target submission: late October 2026, and the reason it is not
sooner is gate 2 below.

Sources, read against the live pages rather than recalled:
submitting ·
review criteria ·
reviewer checklist ·
the Jan 2026 AI policy.

The four desk-reject gates

Gate	JOSS wording	Status
Public history	"public for more than six months prior to submission"	PASS. Public 2026-02-21; cleared 2026-08-21
Iterative development	"sustained development over time (preferably months or years) rather than rapid, recent code generation"	AT RISK. See below
Research impact	"must be evidence that the software is being used for research"	FAIL today. #124
Open-source practice	"lacking issues, pull requests, releases, tests, CI, documentation, or contribution guidelines"	PASS, with one soft spot: #126

Gate 2 is why the date is October, and it constrains how we merge

171 commits on main: 21 in February, 32 in March, zero in April, May and June, 28 in July,
90 in August. That is 53% of the project's history in one month, with a three-month hole in the
middle.

Consequences, and they are unintuitive:

• Merging is not free any more. Every merge this month pushes August's share up, on the exact
  criterion JOSS desk-rejects for.
• Take the open PRs a few days apart, ideally in September. Never another twenty-PR evening.
• Commit at a low steady rate through September and October. The Validate on real donor data: replicate the leakage experiment on KDD Cup 1998 (the tractable JOSS impact gate) #124 work naturally produces that,
  which is convenient: the thing that closes gate 3 also closes gate 2.
• No whole-tree churn. chore: remove every em dash from the repository's prose #91 (remove every em dash, 92 files) already spent that budget once.
  Another mechanical sweep before submission inflates the recent-commit share for zero review
  benefit.

Check it any Monday:

for m in 02 03 04 05 06 07 08 09 10; do
  printf "2026-%s: %s\n" "$m" \
    "$(git rev-list --count origin/main --since=2026-$m-01 --until=2026-$m-31T23:59)"
done

One thing to be conscious of rather than to fix: "rather than rapid, recent code generation" and
the now-mandatory AI usage disclosure point at the same profile, and an AI-assisted history with
half its commits in one month is that profile. The mitigation is that the disclosure landed in #103
is thorough, first-person, and specific about what was not generated. Do not shrink it.

Checklist

Phase A, the research-impact artifact. The only true blocker, and it needs nobody's permission.

• Validate on real donor data: replicate the leakage experiment on KDD Cup 1998 (the tractable JOSS impact gate) #124 KDD Cup 1998 fetcher, real-data leakage replication, docs page, Zenodo deposit
• A preprint that uses the library, framed as the finding rather than the software. Working
  title: Whole-history feature aggregation inflates donor-propensity backtests more than
  splitter choice does. SocArXiv or OSF Preprints needs no endorsement and mints a DOI; arXiv
  stat.AP needs an endorsement if you have never posted there. This is what converts
  "credible near-term significance" into "realized impact"
• Rewrite paper.md's Statement of need and Research impact statement on the real number
• Find one named institutional user (the actual JOSS gate) #83 the institutional user. Parallel, not a prerequisite. Strengthens the paper if it
  lands during review

Phase B, what a reviewer actually checks.

• Cut and tag v0.7.0, publish to PyPI, and archive to Zenodo (JOSS needs a release matching the paper) #125 cut and tag v0.7.0, publish to PyPI, archive to Zenodo. The version DOI goes in the
  submission form
• ci: publish the benchmark table as a reproducible artifact (nothing runs scripts/benchmark_models.py) #7 publish the benchmark table as a reproducible CI artifact. Every AUC number in the paper,
  the README and the benchmarks page is a performance claim, and the checklist asks whether
  performance claims were confirmed
• "No network calls" claim is broader than the code: encounter_path and CLI data path accept remote URIs #114 / fix: reject network-scheme paths at the read boundary #123 the no-network claim, resolved in commit 0612dce on fix: reject network-scheme paths at the read boundary #123. The claim is now two
  enforceable halves, and _NETWORK_ALLOWED in tests/test_no_network.py is the documented
  exemption point Validate on real donor data: replicate the leakage experiment on KDD Cup 1998 (the tractable JOSS impact gate) #124's fetcher adds one line to. Still needs merging
• CI: make CONTRIBUTORS.md/CHANGELOG.md updates a merge-blocking check #113 merge-blocking CONTRIBUTORS/CHANGELOG check. CONTRIBUTORS.md is missing @shubhrai23 (merged #64, #65, #66) #112 fixed the symptom; this is the cause,
  and the checklist has an authorship-and-credit item
• paper.md's merged-PR/contributor counts must be re-verified against the repo at JOSS submission time #116 re-verify the paper's merged-PR and contributor counts on the day of submission. They
  have already moved from 13 to 17
• Add a real-data example to the docs once Validate on real donor data: replicate the leakage experiment on KDD Cup 1998 (the tractable JOSS impact gate) #124 lands. The checklist wants examples "ideally to
  solve real-world analysis problems" and the quickstart is synthetic
• Leave the twelve good first issue items open. An open tracker is evidence of practice.
  An empty one reads worse

Phase C, submission mechanics.

• Zero external issues and zero Discussions: post and pin the who's-using-this thread #126 post and pin the who's-using-this Discussion, and ask the five external contributors
  whether they hit anything worth filing
• Decide the track and say why in the submission notes. The AMC surface argues health/medical;
  the propensity modelling and the fact that the State of the field section is entirely about
  ecosystem ML packages argues social/behavioural, which is the recommendation
• Name three non-conflicted suggested reviewers. Candidates sit around the packages the paper
  cites (feature-engine, scikit-lego, mlxtend, pymc-marketing, MAPIE, crepes, scikit-uplift)
  plus volunteers at https://reviewers.joss.theoj.org/
• Sign up as a JOSS reviewer at that same URL. Cheapest goodwill available, and reviewing is how
  you meet reviewers
• Disclose the upstream contribution history (MAPIE, pymc-marketing, sktime, statsforecast) in
  the submission notes. Not a conflict under JOSS's definition, and disclosing beats a reviewer
  discovering it
• Submit at https://joss.theoj.org/papers/new

Explicitly not on the critical path

• Find one named institutional user (the actual JOSS gate) #83, the institutional user. Valuable, slow, and no longer required.
• pyOpenSci presubmission #337. Zero comments since 2026-08-02, and their published scope is
  Geospatial and Education. The partnership would only save a second review. Do not wait on it.
• JMLR MLOSS. Its submission requirements name "evidence of an active user community" as a
  required cover-letter element, which is strictly harder than anything JOSS asks, and its
  criteria want coverage "near 100%" against our 92% gate. After a JOSS acceptance, not before.
• 1.0.0. JOSS needs a tagged archived release, not a major one. See Cut and tag v0.7.0, publish to PyPI, and archive to Zenodo (JOSS needs a release matching the paper) #125.

Why the preparation is worth two extra months

Everything in openjournals/joss-reviews is permanently public, including a desk rejection, and
a desk reject carries a resubmit-in-six-months-or-more penalty. That asymmetry is the whole argument
for not submitting next week.

If it desk-rejects anyway

• SoftwareX (Elsevier), ~3000 words, no six-month or impact gate. APC ~$1000, waivers exist.
• JORS only if the rejection is specifically on scope. Otherwise it is a £824 duplicate.
• The preprint plus the Zenodo release, which Phase A produces regardless.
• And the better paper may not be the software paper. "Whole-history feature aggregation inflates
  donor-propensity backtests by ~0.13 AUC while splitter choice is worth ~0.01, and a widely used
  synthetic-data pattern lets a model beat its own Bayes ceiling" is a real contribution to a field
  that mostly does not test for this. JAMIA Open, an AMIA symposium paper, or NVSQ, with the library
  as the supplementary artifact. Phase A produces that paper's data as a byproduct, which is the
  strongest argument for doing Phase A whatever JOSS decides.