-
Notifications
You must be signed in to change notification settings - Fork 14
Stop the test suite from calling live provider endpoints #174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,12 @@ | ||
| import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test' | ||
| import { | ||
| afterAll, | ||
| afterEach, | ||
| beforeEach, | ||
| describe, | ||
| expect, | ||
| mock, | ||
| test, | ||
| } from 'bun:test' | ||
| import { mkdtemp, readFile, rm } from 'node:fs/promises' | ||
| import { tmpdir } from 'node:os' | ||
| import { join } from 'node:path' | ||
|
|
@@ -16,9 +24,23 @@ import { | |
| setAccountEnabledPersistent, | ||
| } from '@cortexkit/anthropic-auth-core' | ||
| import { AnthropicAuthPlugin } from '../index' | ||
| import { DEFAULT_FETCH_MOCK, installDefaultFetchMock } from './test-fetch' | ||
| import { | ||
| createTimerTracking, | ||
| type PluginTimerOverrides, | ||
| } from './timer-tracking' | ||
|
|
||
| let tempDir: string | ||
| let accountPath: string | ||
| const tempDirs = new Set<string>() | ||
| const originalFetch = globalThis.fetch | ||
| const timerTracking = createTimerTracking() | ||
| const { | ||
| activeIntervals, | ||
| disabledPluginTimerOverrides, | ||
| trackedClearInterval, | ||
| trackedSetInterval, | ||
| } = timerTracking | ||
|
|
||
| const baseStorage = (): AccountStorage => ({ | ||
| version: 1, | ||
|
|
@@ -68,15 +90,48 @@ const baseStorage = (): AccountStorage => ({ | |
| }) | ||
|
|
||
| beforeEach(async () => { | ||
| installDefaultFetchMock() | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
|
||
| timerTracking.reset() | ||
| if (tempDir) { | ||
| await rm(tempDir, { recursive: true, force: true }).catch(() => {}) | ||
| } | ||
| tempDir = await mkdtemp(join(tmpdir(), 'anthropic-auth-acct-cmd-')) | ||
| tempDirs.add(tempDir) | ||
| accountPath = join(tempDir, 'anthropic-auth.json') | ||
| process.env.OPENCODE_ANTHROPIC_AUTH_FILE = accountPath | ||
| }) | ||
|
|
||
| afterEach(async () => { | ||
| delete process.env.OPENCODE_ANTHROPIC_AUTH_FILE | ||
| await rm(tempDir, { recursive: true, force: true }) | ||
| mock.restore() | ||
| try { | ||
| // Restore only the fixture's tagged mock; an untagged custom mock left | ||
| // installed must reach the preload's leak detector, not be masked here. | ||
| const currentFetch = globalThis.fetch as | ||
| | (typeof fetch & { [DEFAULT_FETCH_MOCK]?: true }) | ||
| | undefined | ||
| if (currentFetch?.[DEFAULT_FETCH_MOCK]) { | ||
| globalThis.fetch = originalFetch | ||
| } | ||
| delete process.env.OPENCODE_ANTHROPIC_AUTH_FILE | ||
| await Promise.all( | ||
| [...tempDirs].map((directory) => | ||
| rm(directory, { recursive: true, force: true }).catch(() => {}), | ||
| ), | ||
| ) | ||
| tempDirs.clear() | ||
| mock.restore() | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: The unconditional Prompt for AI agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Refuted by probe on this head: Bun 1.3.14's mock.restore() restores spyOn-created mocks only — a plain mock() ASSIGNED to globalThis.fetch survives it (observed: the assignment is still installed afterward), so an untagged assigned mock does reach the preload's detector; the comment's claim holds. The variant that WOULD be masked is spyOn(globalThis, 'fetch'), and no test in these files spyOns fetch (grepped). The unconditional mock.restore() stays: it is what resets call counts and non-fetch mocks between tests. |
||
| } finally { | ||
| // Assert last so a detected leak cannot abort the cleanup above. | ||
| expect(activeIntervals.size).toBe(0) | ||
| } | ||
| }) | ||
|
|
||
| afterAll(async () => { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: Prompt for AI agents |
||
| await Promise.all( | ||
| [...tempDirs, tempDir].map((directory) => | ||
| rm(directory, { recursive: true, force: true }).catch(() => {}), | ||
| ), | ||
| ) | ||
| tempDirs.clear() | ||
| }) | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
|
|
@@ -513,11 +568,22 @@ describe('account command INFO logs (via plugin)', () => { | |
| } | ||
| } | ||
|
|
||
| async function getPlugin() { | ||
| return (await AnthropicAuthPlugin({ | ||
| // @ts-expect-error: minimal mock for testing | ||
| client: createMockClient(), | ||
| })) as Promise<any> | ||
| async function getPlugin(timerOverrides?: PluginTimerOverrides) { | ||
| const defaultTimerOverrides = disabledPluginTimerOverrides() | ||
| const plugin = (await ( | ||
| AnthropicAuthPlugin as unknown as ( | ||
| ctx: Parameters<typeof AnthropicAuthPlugin>[0], | ||
| timers?: PluginTimerOverrides, | ||
| ) => ReturnType<typeof AnthropicAuthPlugin> | ||
| )( | ||
| { | ||
| // @ts-expect-error: minimal mock for testing | ||
| client: createMockClient(), | ||
| }, | ||
| { ...defaultTimerOverrides, ...timerOverrides }, | ||
| )) as any | ||
| await plugin.__fallbackRefreshReady | ||
| return plugin | ||
| } | ||
|
|
||
| async function executeCommand( | ||
|
|
@@ -636,4 +702,20 @@ describe('account command INFO logs (via plugin)', () => { | |
| capturedRecords.filter((r) => r.channel === 'commands'), | ||
| ).toHaveLength(0) | ||
| }) | ||
|
|
||
| test('does not retain a background interval unless the helper opts in', async () => { | ||
| await saveAccounts(baseStorage(), accountPath) | ||
| await getPlugin() | ||
| expect(timerTracking.disabledIntervalCalls).toBe(1) | ||
| expect(activeIntervals.size).toBe(0) | ||
|
|
||
| await timerTracking.withTrackedInterval(async () => { | ||
| await getPlugin({ | ||
| setInterval: trackedSetInterval, | ||
| clearInterval: trackedClearInterval, | ||
| }) | ||
| expect(activeIntervals.size).toBe(1) | ||
| }) | ||
| expect(activeIntervals.size).toBe(0) | ||
| }) | ||
| }) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: This change drops preservation of the persisted
mainQuotaCheckedAtand always reportsentry.checkedAt. Under a concurrent writer harvesting quota for the same main account (multi-process is supported here), the reloadedmainQuotaCheckedAt— whichmergeHeaderQuotaForPersistencemerges from — can be newer than this request'sentry.checkedAt, sopublishQuotaHeaderFeedpublishesobserved_at_msthat is older than the newest quota window data it carries. It also contradicts the PR description's stated goal of "preserve quota timestamps." Please confirm this is intentional; in the single-process case the two are identical, so the change only rewrites cross-process behavior.Prompt for AI agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Half right, and the half that's right is already fixed in a different PR — but not restoring the adoption, because that is a cross-account leak.
Why it was removed. The persisted top-level
mainQuotaCheckedAtcarries no account binding of its own, while the quota snapshot beside it does. The old code validated identity on the snapshot and then trusted the unbound timestamp, so a published feed entry for account A could carry account B's observation time. Reproduced deterministically: quota identityaccount-awith top-level timestamp2_000_000publishesobserved_at_ms=2_000_000for a request whose own observation was1_000_000. It surfaced first as an intermittent CI failure, which is how I found it — removing ~300ms of network latency from the suite made the interleaving reachable where it previously wasn't. The same unbound pair exists at a second consumer (getPersistedMainQuota→seedMainFromStorage), filed as #177.So restoring the adoption to fix a timestamp-ordering nit would reintroduce cross-account data in a published feed. Not a trade I'll make.
Your inconsistency is real, though, and I'd have told you it was already handled if I hadn't checked. My first instinct was that schema v3's per-field provenance map answers it —
observed_at_mscovers header-derived fields, and merged poll-owned fields carry their owncheckedAt. That is true, and it is not on this branch:81200aais schema v2 with zerofieldSources. The provenance work is #172, open and independent of this stack.So on this branch as it stands, a published entry can carry merged poll fields newer than the timestamp it reports, with nothing telling a reader that. Narrow, but there is a live consumer of this feed, so it isn't theoretical.
Disposition: the structural fix belongs in #172, where the per-field provenance already exists — I'm not duplicating it here. What this PR gets is a doc comment on
observed_at_msstating what it covers, so the contract is written down rather than reverse-engineered from the merge logic.On the PR description contradiction you flagged: fair, and it's fixed. The description was written before the leak was found and still described preserving persisted quota times. It now says what the code does.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Intentional. Reporting
entry.checkedAt(the request's own observation time) rather than the reloaded persistedmainQuotaCheckedAtis deliberate: the persisted top-level timestamp has no account binding, so under a concurrent cross-account writer, adopting it re-times this request's publishedobserved_at_msto a different account's observation — the cross-account leak filed as #177. So the single-process identity you note is the correct behaviour, and the cross-process case must NOT inherit the persisted timestamp.The narrow real gap is on this branch's v2 feed: a poll-owned field merged in can be newer than the header-derived
observed_at_ms, with no reader-visible provenance to distinguish them. The structural fix for that is per-field provenance (each field carries its own source +checkedAt), which is PR #172's schema v3 — not present on this branch. This branch documents the v2 contract (observed_at_msis header-observation time; merged poll fields retain their owncheckedAt) and leaves the mechanism to #172.