Add package manager command smoke tests - #1677
Conversation
|
Contributes to #1622 |
469835c to
023bf16
Compare
023bf16 to
5a9aaac
Compare
6128c10 to
51074af
Compare
51074af to
7b78eef
Compare
## Summary Introduces the reusable command-object layer for package management while leaving existing package-manager call sites unchanged. ## Scope - Adds the `PackageManagerCommand` base class and shared execution options. - Adds abstract templates for install, uninstall, list, version, available versions, and direct package names. - Adds concrete Pip/UV, Conda, and Poetry implementations. - Adds command factories, exports, and command-local execution helpers required by those implementations. - Preserves the existing executables, flags, environment targeting, prerelease defaults, and list-command timeouts. ## Non-goals This PR does not add command tests or migrate package managers to the new classes. Those changes are isolated in follow-up PRs #1677 and #1678. ## PR stack 1. This PR: command classes and implementations, based on `main`. 2. #1677: command smoke tests, based on this branch. 3. #1678: adoption and cleanup, based on #1677. ## Testing - `npm run lint` - `npm run compile-tests` - `npm run unittest` (1,448 passing, 4 pending)
7b78eef to
c229085
Compare
|
Note: These tests only verify that the commands are valid, a further PR will add integration tests with the package managers and round-trips such as: fetching available versions for a package, installing it, listing (direct) packages, uninstalling, etc... |
There was a problem hiding this comment.
Pull request overview
This PR adds lightweight “smoke” unit tests for the concrete package manager command classes introduced in #1621, ensuring each command’s execute() method can run end-to-end under minimal mocked conditions without throwing/rejecting.
Changes:
- Adds smoke unit tests for Pip/UV command classes, stubbing
runPython/runUVand providing minimal parseable outputs where needed. - Adds smoke unit tests for Conda command classes, stubbing
runCondaExecutableand providing minimal JSON/string outputs. - Adds smoke unit tests for Poetry command classes, stubbing
runPoetry/getPoetryVersionand providing minimal outputs for parsing commands.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/test/managers/builtin/commands.unit.test.ts | Smoke coverage for Pip/UV command classes with mocked execution helpers and minimal outputs. |
| src/test/managers/conda/commands.unit.test.ts | Smoke coverage for Conda command classes with mocked conda execution and minimal JSON/string outputs. |
| src/test/managers/poetry/commands.unit.test.ts | Smoke coverage for Poetry command classes with mocked runPoetry/getPoetryVersion and minimal outputs. |
Finding: The tests pass even when the commands do nothing or produce incorrect resultsSeverity: Medium Locations:
Every test uses this general assertion: await assert.doesNotReject(() => command.execute(...));In simple terms, this asks only:
It does not ask:
Why that mattersFor example, this test is named: test('PipInstallCommand executes without error', async () => {
const command = new PipInstallCommand(...);
await assert.doesNotReject(() =>
command.execute({ packages: [{ packageName: 'package' }] }),
);
});The real implementation should invoke something equivalent to: But all of the following broken implementations would still pass: async execute(): Promise<void> {
// Do nothing.
}async execute(): Promise<void> {
await runPython('python', ['-m', 'pip', 'uninstall', 'package']);
}async execute(): Promise<void> {
await runPython('python', ['incorrect-command']);
}Because Concrete regressions these tests would missPip and UVThe tests would not detect:
For example, The test does not assert any part of that command. CondaThe tests would not detect:
The environment path is particularly important: condaEnvironmentPath: 'environment'The test supplies it but never confirms it reaches the command runner. The implementation could silently install packages into Conda’s currently active environment and the test would still pass. PoetryThe tests would not detect:
Poetry operates against a project’s new PoetryAddCommand({
pythonExecutable: 'poetry',
cwd: 'project',
log: mockLog,
});never verifies that Recommended improvementAt minimum, each mutation command should verify the runner invocation. For example: await command.execute({
packages: [{ packageName: 'package' }],
});
assert.ok(
runPythonStub.calledOnceWithExactly(
'python',
['-m', 'pip', 'install', 'package'],
undefined,
mockLog,
undefined,
sinon.match.number,
),
);The exact assertion should match the helper’s contract, but the important point is to verify the executable, arguments, environment/cwd, cancellation token, and timeout. |
eleanorjboyd
left a comment
There was a problem hiding this comment.
The added parsing assertions are a useful improvement. I left one inline comment on the remaining command-construction coverage gap and the inaccurate integration-test claim; this is the main issue I would address before treating these as meaningful command tests.
| // execute() resolves. Their concrete command strings are exercised by integration tests. | ||
| suite('mutation commands execute without error', () => { |
There was a problem hiding this comment.
This integration-coverage claim is not currently true. packageManagement.integration.test.ts only selects venv environments for install/uninstall, so it never exercises the Conda or Poetry command classes; that path also dispatches to Pip or UV dynamically, so it does not deterministically cover either concrete command string. Because these unit tests do not inspect the runner invocation either, subcommand, flag, interpreter, environment-prefix, and cwd regressions remain uncovered. Please assert the runner arguments in these suites (including the analogous Conda/Poetry tests), or remove this claim until deterministic integration coverage exists.
fc8c0b6 to
2f904be
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
src/test/managers/builtin/commands.unit.test.ts:66
- PR description says these command tests intentionally do not assert parsed values, but this test asserts parsing/filtering behavior (prerelease filtering). Either update the PR description to match the actual coverage, or reduce these to smoke-only tests that just ensure execute() resolves.
test('PipAvailableVersionsCommand parses versions and filters prereleases', async () => {
runPythonStub.resolves(JSON.stringify({ versions: ['2.0.0rc1', '1.0.0'] }));
const command = new PipAvailableVersionsCommand({ pythonExecutable: 'python', log: mockLog });
const result = await command.execute({
packageName: 'package',
pythonVersion: '3.13.1',
includePrerelease: false,
});
assert.deepStrictEqual(result.map((v) => v.public), ['1.0.0']);
src/test/managers/poetry/commands.unit.test.ts:49
- PR description says these smoke tests "intentionally do not assert ... parsed values", but this test asserts parsing behavior (and the test title explicitly states it parses fields). Either update the PR description/scope accordingly, or adjust this to a pure smoke test (only assert execute() resolves).
test('PoetryShowCommand parses name, version and description', async () => {
runPoetryStub.resolves(['requests 2.31.0 Python HTTP for Humans.', ''].join('\n'));
const command = new PoetryShowCommand({ pythonExecutable: 'poetry', cwd: 'project', log: mockLog });
const result = await command.execute();
src/test/managers/conda/commands.unit.test.ts:57
- PR description states the new tests "verify only that each command executes without throwing or rejecting" and do not assert parsed values, but this test asserts parsed output (versions list). Consider either updating the PR description to reflect the added parser coverage, or converting this to a smoke-only doesNotReject test.
test('CondaAvailableVersionsCommand parses versions keyed by package name', async () => {
runCondaStub.resolves(JSON.stringify({ package: [{ version: '1.0.0' }, { version: '2.0.0' }] }));
const command = new CondaAvailableVersionsCommand({ pythonExecutable: 'conda', log: mockLog });
const result = await command.execute({ packageName: 'package', pythonVersion: '' });
assert.deepStrictEqual(result.map((v) => v.public), ['1.0.0', '2.0.0']);
I think you are right. I will try changing the order of these PRs. I think it would be better to have the proper integration first and then perform actual roundtrip integration tests; maybe even in the same PR. |
… doc Address PR review: strengthen command unit tests to verify output parsing behavior (package lists, direct names with normalization, version strings, available-version filtering) using representative runner output, instead of asserting the full argument vector (which merely restates buildCommand). Mutation commands remain smoke tests; their command strings are covered by integration tests. Also correct UvUninstallCommand doc comment: 'uv pip uninstall' has no -y/--yes flag and is non-interactive by default. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f16397e-0917-4efb-8d75-566c71ebf9ba
2f904be to
42b5e98
Compare
- pipPackageManager.manage(): rethrow CancellationError instead of swallowing it, so callers (e.g. venv creation's pkgInstallationCancelled) can distinguish cancel from failure; restores parity with main and the conda/poetry managers. - getDirectPackageNames: fix docstring to reflect uv uses 'uv pip tree --depth=0' (not 'uv pip list --not-required'). - parsePackageSpecs: omit undefined version property and clarify wording. - runPython: restore 'python:' prefix on stderr log output for consistency. - Remove overlapping command smoke tests; the reworked, parsed-value versions are owned by the tests PR (#1677). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f16397e-0917-4efb-8d75-566c71ebf9ba
Summary
Adds focused unit coverage for the concrete package manager command classes introduced in #1621, across Pip/UV, Conda, and Poetry. Also corrects the
UvUninstallCommanddoc comment.What the tests cover
show): stub the runner with representative output and assert the parsed result — package lists (with rows missing a version dropped), direct names with normalization,uv pip treeindentation filtering, version-string parsing, and available-version prerelease filtering.execute()resolves. They have no output to parse; their concrete command strings are exercised by integration tests and by the adoption PR Adopt package manager command classes #1686.These deliberately avoid asserting the exact argument vector, which would merely restate
buildCommand.Doc fix
UvUninstallCommand:uv pip uninstallhas no-y/--yesflag and is non-interactive by default — corrected the misleading comment.Relationship
maindirectly.main. Both currently add the command test files, so whichever merges first the other needs a trivial rebase on those files.Testing
npm run compile-testsnpm run unittest(58 command tests passing)