Add test_without_assertions lint - #17626
Conversation
|
Thanks for the pull request, and welcome! You should hear from one of our reviewers after this PR gets at least 2 reviews from the community. Please see the contribution instructions for more information. |
47c2516 to
26a7bd7
Compare
|
This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
7e53400 to
d672949
Compare
There was a problem hiding this comment.
Community review
From the issue:
This can be a more general lint for tests which can never fail or always fail. Empty tests aren't particularly unique other than being the degenerate case.
I think you missed this part.
the lint I guess can start out with only linting for the degenerate case, but it should have the ambition to also lint for the provably true/false cases..
🤔
unless..
Actually I think type tests (where you ensure that for example a types size always is below a cache line for perf) cannot happen.
But that sounds a bit like an edge case
CC @Jarcho if that would be something where a "false positive" (is it one?) is good or bad?
| LL | / fn empty_with_comment() { | ||
| LL | | // This is still an empty body. | ||
| LL | | } | ||
| | |_^ |
There was a problem hiding this comment.
it would be very nice to get the #[test] attr into this span, but that sounds like some work, so this is also fine likely.
There was a problem hiding this comment.
Thanks, I agree that including #[test] in the span would make the diagnostic clearer. I looked into it, but it seems that #[test] has already been expanded by the time this late lint runs. I left the current function span as is for now. Please let me know if there is a simpler approach I have missed.
e93506a to
304373a
Compare
|
and the requested scope widening from the issue? @rustbot author |
|
Reminder, once the PR becomes ready for a review, use |
|
@CommanderStorm I checked the existing I am not yet sure which additional always-passing or always-falling test cases this lint should cover without overlapping with it, or flagging useful type/layout regression tests. Could you point me to the intended cases for this PR? I can then extend the lint accordingly. |
|
I mean a testcase without assertions or other without panic sites would be my idea. |
|
@CommanderStorm Thanks, that makes sense now. I’ll widen this to cover tests with no assertions or other visible panic sites, while treating ordinary calls as possible panic sites. For #[test]
#[should_panic]
fn empty_should_panic() {}It already fails because it returns normally instead of panicking. |
|
I think lining on this since empty is fair too. It is suspicious in any case |
Empty test functions execute no test code and cannot verify behaviour. Provide an opt-in restriction lint so projects can detect these no-op tests while allowing temporary placeholders where appropriate.
Detect test functions with no assertions or potential failure sites, including empty bodies. This catches test that can pass without exercising behaviour while preserving expected-panic tests and calls that may fail internally.
304373a to
0c06278
Compare
|
@CommanderStorm It looks like this newly added lint is causing some existing UI and ui-toml tests to fail. I’m not sure that adding Is that expected for a new lint in this category, or is there a different approach you would prefer? |
| sym::assert_macro | ||
| | sym::assert_eq_macro | ||
| | sym::assert_ne_macro | ||
| | sym::debug_assert_macro | ||
| | sym::debug_assert_eq_macro | ||
| | sym::debug_assert_ne_macro |
There was a problem hiding this comment.
We nowerdays also have assert_matches and it's debug variant. Could you add a case and testcase for this too?
| let is_failure_site = matches!(expr.kind, ExprKind::Call(..) | ExprKind::MethodCall(..)) | ||
| || root_macro_call_first_node(cx, expr).is_some_and(|macro_call| { |
There was a problem hiding this comment.
We need to explicitly state this false negative of not descending into the called functions in the known problems sections or fix it
| } | ||
| //~^^^ test_without_assertions | ||
|
|
||
| #[allow(clippy::unused_unit)] |
There was a problem hiding this comment.
Please use expect instead of allow, since it is a tighter fit
| #[test] | ||
| fn calls_helper() { | ||
| helper(); | ||
| } |
There was a problem hiding this comment.
Please annotate that this is a false positive and use black_box for another test where recursing into FN Def's is not possible and one test.
|
|
||
| fn helper() {} | ||
|
|
||
| fn main() {} |
There was a problem hiding this comment.
Our usual strategy (a bit inconsistent) in this case is to move everything into an mod main.
This way, all followups can be nicely placed into issue_* modules ^^
Could you do this?
|
I flew over the changes in assertions. Also run an lintcheck run to see if I missed a case that should actually lint in your opinion upon second view. |
Fixes #17488
Tests without assertions or other potential failure sites can pass without verifying behavior. This adds
test_without_assertions, a suspicious lint that detects these tests, including empty test bodies.changelog: [
test_without_assertions]: add a suspicious lint for tests without assertions or other potential failure sites.stderrfilecargo testpasses locallycargo dev update_lintscargo dev fmt