Skip to content

feat: forward certificateSettings so a course.json can pick its certificate template - #34

Merged
rryoung98 merged 10 commits into
mainfrom
feat/course-certificate-settings
Nov 12, 2026
Merged

rryoung98 merged 10 commits into
mainfrom
feat/course-certificate-settings

Conversation

@rryoung98

@rryoung98 rryoung98 commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Summary of changes

Stacked on #33 (base is feat/course-duration-weeks, not main); merge #33 first.

The same round-trip that dropped durationWeeks also drops certificateSettings: the validate step rewrites the payload through the Course model, and pydantic ignores fields the model does not declare. So an author could not enable certificates or choose a certificate design from course.json; only the qBook admin UI could, after the deploy.

  • Course gains an optional certificateSettings with enabled (strict bool), criteria ({type: completion|points, value >= 0}) and templateId (qbraid, quera or qct), mirroring the qBraid API's validator so a typo fails at validation with a field-level message instead of a 400 at deploy time. Both models forbid unknown keys, so templateID fails with an error naming the key instead of being dropped. criteria.value is strict and finite: "80", true, inf and NaN are refused, not coerced.
  • Course.to_payload() replaces the bare model_dump at the write site. It omits certificateSettings when absent (and criteria when absent inside it), because the API's schema is optional(object) and refuses an explicit null. durationWeeks keeps its documented null.
  • Tests: forwarded verbatim; absent → key omitted; nested absent criteria omitted; QCT, thailand, missing enabled, "yes", grade, negative value, string value, boolean value, inf, NaN all fail validation; an unknown key (templateID, criteria.valu) fails with an extra_forbidden error whose location names that key.

Example course.json addition:

"certificateSettings": {
  "enabled": true,
  "criteria": { "type": "completion", "value": 80 },
  "templateId": "qct"
}

On the API side, POST /learn/articles/course/ingest and PUT /learn/articles/course/:customId already accept this shape; qBraid/qbraid-api#1136 makes the create path persist templateId and preserves a stored template when a redeploy omits it.

Validation run: pytest test (198 passed: 188 unit, 10 e2e), make format-check clean.

Partner designs (quera, qct) need a grant on the course's organization, set by qBraid staff; the deploy domain grants nothing. Without the grant the API answers 403 CERTIFICATE_TEMPLATE_NOT_ALLOWED and the action fails showing that response. A new course with no templateId gets quera only on a quera.com course whose organization holds the quera grant, otherwise qbraid. The README says this.

Coordinated PRs

Deploy a course naming qct only once #1136 and #40 are live on the target environment; before that the API's enum rejects it with a 400 naming the allowed values.

🤖 Generated with Claude Code

…ficate template

The validate step rewrites the deploy payload through the Course model, and
pydantic drops undeclared fields, so certificateSettings in course.json never
reached the API. An author could not enable certificates or choose a design
at deploy time; only the qBook admin UI could.

Course gains an optional certificateSettings (enabled, criteria, templateId)
mirroring qbraid-api's validator: templateId is one of qbraid, quera, qct;
enabled is a strict boolean. Course.to_payload omits the object when absent
(and criteria when absent inside it), because the API refuses an explicit
null there, unlike durationWeeks.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 22, 2026 11:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

CertificateCriteria.value currently allows unintended coercions (e.g., booleans / numeric strings) that can let invalid course.json pass validation and fail later at deploy time.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

This PR extends the Course validation/serialization pipeline so course.json can declare certificateSettings (including certificate enablement criteria and an optional certificate template), and ensures the deploy payload preserves these fields instead of dropping them during the Pydantic model round-trip.

Changes:

  • Added CertificateSettings/CertificateCriteria models and a Course.certificateSettings field to validate certificate configuration early.
  • Introduced Course.to_payload() and switched the validate step to write course_data.json using it, omitting certificateSettings (and nested criteria) when absent to avoid sending explicit null.
  • Added unit tests covering forwarding behavior, omission behavior, and several invalid-shape/invalid-enum validation failures.
File Description
src/​scripts/​validate_course.py Adds certificate models and to_payload() to correctly forward/omit certificateSettings in the deploy payload.
test/​unit/​test_validate_course.py Adds tests ensuring certificateSettings is forwarded and invalid inputs fail validation.
CHANGELOG.md Documents the new forwarding behavior for certificateSettings.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +53 to +59
class CertificateCriteria(BaseModel):
"""When a learner earns the certificate: a completion percentage or a
points total, both of which the qBraid API accepts as `value >= 0`."""

type: Literal["completion", "points"]
value: float = Field(..., ge=0)

@rryoung98

Copy link
Copy Markdown
Member Author

Exercised live through the Action from a qBraid-tutorial workflow (uses: qBraid/upload-course-action@feat/course-certificate-settings, update mode) into a preview API on qbraid-api#1136, reading the course back by id after each run:

  • run 35757248347: course.json names templateId: "quera", completion 5 → stored {enabled: true, criteria: {completion, 5}, templateId: quera}.
  • run 35757373137: certificateSettings removed from course.json → stored values unchanged (the key is omitted from the payload, and the API's merge keeps the stored template).
  • run 35757548917: templateId: "qct", completion 0 → moved back.

durationWeeks (#33) rode through all three redeploys.

@rryoung98

Copy link
Copy Markdown
Member Author

Exercised live: the Action deployed a new course into a preview of qbraid-api #1093/#1136/#1156, first in create mode and then in update mode.

Checked live, working

  • Create mode with certificateSettings: {enabled, criteria, templateId: "qct"} stored all three.

Findings

  1. A course.json that sets certificateSettings without criteria erases the criteria an admin stored. Reproduced live: points >= 5 was set through the settings route, then an update-mode deploy with {enabled: true, templateId: "qct"} left the stored settings as {enabled: true, templateId: "qct"}, so issuance falls back to completion 100. The API merge keeps only templateId (qbraid-api #1136, mergeCertificateSettings). Either the API carries criteria over too, or the README should say that omitting criteria resets it.
  2. CertificateCriteria.value accepts "inf" and 1e999. json.dump then writes a bare Infinity, and the API rejects the body with 400, which is the late failure validation is meant to prevent. Field(..., ge=0, strict=True, allow_inf_nan=False) fixes it. (Reproduced with pydantic locally.)

rryoung98 and others added 2 commits September 26, 2026 12:40
An author had no way to learn the valid templateId values short of a failed
validation. Lists the fields, the three designs, and what an omitted field
does on redeploy, including that an omitted criteria resets to 100%.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
quera and qct are reserved for their partner organizations by the qBraid API,
and a deploy naming one the course's org cannot use fails with 403
CERTIFICATE_TEMPLATE_NOT_ALLOWED. Documents that, and that a course keeps a
design it already has.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@rryoung98
rryoung98 changed the base branch from feat/course-duration-weeks to main September 28, 2026 20:52
rryoung98 and others added 2 commits September 30, 2026 19:33
CertificateSettings and CertificateCriteria used pydantic's default
extra="ignore", so a typo such as templateID passed validation and the
course deployed with the default design. Both models now forbid extra
keys, and the error location names the unknown key. criteria.value is
strict and finite: "80", true, inf and NaN are refused, not coerced.

The README and changelog said quera was open to any course deployed to
quera.com. The qBraid API grants nothing by deploy domain: a partner
design needs a grant on the organization, set by qBraid staff, and the
API otherwise answers 403 CERTIFICATE_TEMPLATE_NOT_ALLOWED. A new course
with no templateId gets quera only on quera.com with that grant.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The spell checker flagged the deliberate misspelling.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@rryoung98

Copy link
Copy Markdown
Member Author

@avanbussum Ready for review. Changes from the internal pass: certificateSettings and criteria now reject unknown keys (a typo like templateID used to be dropped silently), and criteria.value is strict (9d038c1). The README's quera rules are corrected: a partner design needs an org grant.

@rryoung98
rryoung98 requested a review from avanbussum October 1, 2026 01:10
TheGupta2012 and others added 4 commits November 12, 2026 17:31
…ings

Reverting the write site to model_dump wrote certificateSettings: null,
which the qBraid API refuses, and every test still passed. The
durationWeeks tests now read to_payload too, the payload actually sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README said an omitted criteria resets to 100% completion and told
authors to always send it. The API writes each sent field on its own
path, so an update keeps stored criteria; only a new course gets the
default. Following the old advice made every redeploy overwrite
criteria an admin set in qBook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
null for certificateSettings, criteria or templateId passed validation
and was dropped, so the course kept its stored value with no message.
The API refuses null for these keys; the validator now does too and
names the key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A completion value is a percentage, so 250 passed validation and the
course could never issue a certificate. Points totals keep no upper
bound. This is stricter than the API, like the unknown-key check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TheGupta2012

Copy link
Copy Markdown
Member

Review of #34, with fixes pushed

I reviewed the branch against the qBraid API's validators on staging and main (validators.ts, certificate-settings-update.ts, service.ts, learn.utils.ts). Then I pushed one commit per finding. The model accepts what the API accepts, and merging changes nothing for a course.json without the block.

Fixed

# Finding Commit
1 No test covered the write site. Reverting json.dump(course.to_payload(), f) to model_dump wrote certificateSettings: null, which the API refuses with a 400, and all 32 tests still passed. test_validate_valid now parses the written payload and asserts the key is absent. The durationWeeks tests now read to_payload(). aa139c5
2 The README said an omitted criteria resets to 100% completion and told authors to always send it. The API writes each sent field on its own path, so an update keeps stored criteria. Only a new course gets the default. Following the old advice made every redeploy overwrite criteria set in qBook. 8b05995
3 An explicit null for certificateSettings, criteria or templateId passed validation and was dropped, so the stored value stayed with no message. It now fails and names the key. b4367a9
4 A completion value above 100 passed, so the course could never issue a certificate. It now fails. points totals keep no upper bound. This is stricter than the API, like the unknown-key check. 9c0d2d1

Not changed

  • Unknown top-level keys are still ignored. "certificatSettings" deploys and sends nothing. Adding extra="forbid" to Course would fail any consumer's spec that carries an extra top-level key. Every consumer uses @main, and I only checked the Quantum-Club-Thailand specs.
  • The PR description is out of date. It says the PR is stacked on feat: forward durationWeeks from course.json to the deploy payload #33, which merged on 2026-09-25, so the base is now main. It also cites a criteria.valu test, which 69fb4ec renamed to threshold.

Checks

uv run pytest -q test                                  204 passed
uv run black --check src/scripts test                  26 files would be left unchanged
uv run isort --check-only src/scripts test             clean
uv run pylint src/scripts --fail-under=7.0             9.89/10 (the two warnings predate this PR)
  • Mutation checks:
    • Reverting the write site fails test_validate_valid.
    • Removing the null check fails the 3 new null tests.
  • All six Quantum-Club-Thailand specs, main versus this branch: each spec gives a byte-identical course_data.json with no certificateSettings key.
  • Each spec with a qct block added (qBraid/Quantum-Club-Thailand#47): all six validate. The payload carries the block as written, for example {'enabled': False, 'criteria': {'type': 'points', 'value': 32.0}, 'templateId': 'qct'}.

Not verified

  • The full 403 CERTIFICATE_TEMPLATE_NOT_ALLOWED path end to end. I only read the line that logs response.text.
  • The README claim that a design change affects only certificates not yet delivered.

CI sets GITHUB_OUTPUT, so validate() also writes outputs through the
same mocked open and the joined writes held two JSON documents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TheGupta2012

Copy link
Copy Markdown
Member

Follow-up on CI:

  • d72f7a9 fixes a test I added in aa139c5. CI sets GITHUB_OUTPUT, so validate() also writes outputs through the same mocked open, and the joined writes held two JSON documents. The test now reads the payload from json.dump. It passes with and without GITHUB_OUTPUT, and it still fails when the write site is reverted to model_dump.
  • Pytest (3.11) still fails, but not on a test. All 204 tests pass. The Codecov upload step then fails: This repository is deactivated. To resume uploading to it, please activate the repository in the Codecov UI. fail_ci_if_error: true (.github/workflows/test.yml:112) turns that into a job failure. Someone with Codecov admin access needs to reactivate the repository. Nothing in this PR changes that step.

@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@rryoung98
rryoung98 merged commit 3cd701c into main Nov 12, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants