Repository navigation
feat(policies): read all pages in the chainloop.discover builtin - #3781
Conversation
The chainloop.discover builtin (Rego and WASM) returned only the first page of references (20 by default), so a policy could not see older references of a digest. It now requests pages of 100 references, follows the pagination cursor, and merges the references of all pages into the same result shape. A cap of 10 pages bounds the requests of one call; when the cap stops the read, the result has "truncated": true. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: ec58f4bc-6b1c-47bc-aa15-a90ad6cfff3a
PR validation — ✅ 3 passing
AI Session Checks — 🟢 94% · ✅ 0 failing
|
| Status | Attribution | File | Lines |
|---|---|---|---|
| created | ai | pkg/policies/engine/builtins/discover_test.go |
+217 / -0 |
| modified | ai | pkg/policies/engine/builtins/discover.go |
+73 / -11 |
| modified | ai | pkg/policies/engine/builtins/rego.go |
+14 / -7 |
| modified | ai | pkg/policies/engine/builtins/wasm.go |
+2 / -2 |
Policies (4)
| Status | Policy | Material | Messages |
|---|---|---|---|
| ✅ Passed | ai-config-ai-agents-allowed |
ai-coding-session-ec58f4 |
- |
| ✅ Passed | ai-config-no-dangerous-commands |
ai-coding-session-ec58f4 |
- |
| ✅ Passed | ai-config-no-secrets |
ai-coding-session-ec58f4 |
- |
| ✅ Passed | ai-config-mcp-servers-allowed |
ai-coding-session-ec58f4 |
- |
Security Checks — ✅ 5 passing
✅ secret-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | secrets-detection |
- |
✅ sast-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | owasp-top10-2025 |
- |
| ✅ Passed | sast |
- |
| ✅ Passed | cwe-top25 |
- |
| ✅ Passed | cwe-top26-40-cusp |
- |
Scans not applied (3)
| Scan | Reason |
|---|---|
vulnerability-scan |
no manifest/lockfile changed |
github-actions-scan |
no workflow files changed |
iac-scan |
no IaC files changed |
Security context
✅ Nothing this change touches has a recorded security-fix history.
View in Chainloop ↗ · How this works ↗
Powered by Chainloop and Chainloop Trace
There was a problem hiding this comment.
All reported issues were addressed across 4 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
The discover builtin now stops when a control plane ignores the cursor and returns the same next cursor again, so it does not merge a page twice. A control plane without pagination support already returns all references in one response with no next cursor, and a test now covers that case. The Rego builtin returns null instead of panicking when there is no control plane connection, and has its own tests. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: ec58f4bc-6b1c-47bc-aa15-a90ad6cfff3a
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
When a control plane repeats its next cursor, the discover builtin stops with only the pages it could read. It now sets truncated in that case, so a policy does not take incomplete references for complete ones. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: ec58f4bc-6b1c-47bc-aa15-a90ad6cfff3a
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
The truncated field is set when the page cap is reached and when the control plane repeats its next cursor. The field and builtin comments now say so. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: ec58f4bc-6b1c-47bc-aa15-a90ad6cfff3a
The
chainloop.discoverpolicy builtin (Rego and WASM) returned only the first page of the paginatedDiscoverPrivateAPI, which holds the 20 newest references by default. A policy that looked for an older reference of a digest, such as a CI run behind several newer deploy or scan runs, could not find it and got no error.The builtin now reads all the pages:
next_cursoruntil there is no next page."truncated": true; otherwise"truncated": false.truncated.The builtin does not send the run's
project_name. That would filter out references from other projects, which policies such as provenance checks can depend on, so it would change what the builtin returns.This change is limited to the page-reading loop in
pkg/policies/engine/builtinsand does not overlap the organization scoping in #3777.Note:
pkg/policies/engine/rego/builtins/discover.goholds an older copy of the discover builtin that nothing in the repo registers. It still reads one page. This PR leaves it as is; it can be removed in a follow-up.Testing: table-driven unit tests with a fake referrer client cover a single page, several pages, the last page allowed by the cap, the cap that stops the read (
truncated), an error on a later page, a control plane without pagination, and a control plane that ignores the cursor. They also cover the JSON shape of the result and the output of the Rego builtin.AI disclosure: this contribution was made with the help of Claude Code.
Closes #3780
🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri using the eng-autopilot skill