Skip to content

fix(cabal-install): cabal test runs the tests of the targets that have them - #12300

Open
zlonast wants to merge 5 commits into
haskell:masterfrom
zlonast:zlonast/without-tests
Open

zlonast wants to merge 5 commits into
haskell:masterfrom
zlonast:zlonast/without-tests

Conversation

@zlonast

@zlonast zlonast commented Aug 30, 2026 •

Copy link
Copy Markdown
Collaborator

fix: #11858

$ cabal test pkg-with-tests pkg-without-tests
No tests to run for the package pkg-without-tests-0.1.0.0
Running 1 test suites...
Test suite pkg-with-tests-test: PASS
1 of 1 test suites (1 of 1 test cases) passed.

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

@chreekat chreekat left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have pointed out a few typical mistakes made by agents of the current era. There are more to be found. I suggest taking a look and trying to improve its output.

Comment thread cabal-install/tests/IntegrationTests2.hs Outdated
Comment thread changelog.d/12300.md Outdated
Comment thread cabal-install/src/Distribution/Client/CmdTest.hs Outdated
Comment thread cabal-install/src/Distribution/Client/CmdTest.hs Outdated
@chreekat

Copy link
Copy Markdown
Collaborator

Sorry to rain on your parade with this review. I do appreciate that you tackled the issue I raised! The new test looks fine; if the impl was cleaned up I'm sure this would actually fix my problem.

@zlonast
zlonast force-pushed the zlonast/without-tests branch from 4397493 to c426385 Compare September 23, 2026 09:34
@zlonast

zlonast commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

@chreekat Thanks for the review; this time I decided to take a closer look. I hope you'll find time for another review :)

@zlonast
zlonast requested a review from chreekat September 23, 2026 09:44

@chreekat chreekat left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks again!

@philderbeast

Copy link
Copy Markdown
Collaborator

The cabal test --help comment on --test-fail-when-no-test-suites is very short. It could be worth explaining this flag's behaviour in more detail in the cabal test section of the user guide.

--test-fail-when-no-test-suites
# Exit with failure when no test suites are
found.

cabal/doc/cabal-commands.rst

Lines 1340 to 1344 in aeb7dbf

cabal test
^^^^^^^^^^
``cabal test [TARGETS] [FLAGS]`` tests test suites specified as targets
after ensuring they are up to date and building them, if necessary.

@zlonast

zlonast commented Sep 25, 2026

Copy link
Copy Markdown
Collaborator Author

@philderbeast Thanks, I've left a note.

@philderbeast

philderbeast commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

When reviewing, I added some cabal-testsuite package tests that I'd like you to include.

0001-Add-tests-for-running-all-tests-that-exist.patch

I'm not sure what the behaviour should be when the targets are p:tests q:tests instead of p q, so this is a separate patch that for now records the current failing behaviour.

0002-Failures-with-p-tests-and-q-tests.patch

If we do want the behaviour to be the same with :tests filters on the target selectors then I have explored that too.

0003-Behave-the-same-with-tests-targets.patch
0004-Update-the-test-expectations-and-output.patch
0005-Add-assertion-helpers-reorder-tests.patch

@philderbeast

Copy link
Copy Markdown
Collaborator

#12376 has merged so rebasing should enable that failing test to now pass.

zlonast and others added 5 commits September 26, 2026 04:30
Adds cabal-testsuite/PackageTests/NewBuild/CmdTest/NoTests covering
cabal v2-test with the p q, all, all:tests, q and q:tests targets, each
with and without --test-fail-when-no-test-suites.

Kind-filtered package targets that come up empty (q:tests when q has no
test suites, all:tests when no package has test suites) are now skipped
with a notice like plain package targets, instead of failing the
command.
@zlonast
zlonast force-pushed the zlonast/without-tests branch from 569295e to f955f3d Compare September 26, 2026 07:04
@zlonast

zlonast commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the tests, @philderbeast! I think :tests consistent skipping is what users expect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"cabal test without-tests with-tests" won't run tests

3 participants