Skip to content

ci: let shellcheck see the extension-less scripts in bin/ - #24

Merged
gwarf merged 1 commit into
mainfrom
fix/megalinter-shellcheck-extensionless
Sep 1, 2026
Merged

ci: let shellcheck see the extension-less scripts in bin/#24
gwarf merged 1 commit into
mainfrom
fix/megalinter-shellcheck-extensionless

Conversation

@gwarf

@gwarf gwarf commented Sep 1, 2026

Copy link
Copy Markdown
Owner

MegaLinter has never linted the extension-less scripts in this repo — 18 of them,
including everything in home/private_dot_local/bin/. The 8 .sh files are
already covered and always have been; this PR is about the other half.

MegaLinter's BASH descriptor ships file_extensions: [".sh", ".bash", ".dash", ".ksh"] with no "" entry, and its filter_files() gates on the extension
before anything else:

if file_extension in file_extensions: pass
elif "*" in file_extensions: pass
elif file_names_regex_object.fullmatch(base_file_name): pass
else: continue

os.path.splitext("executable_pai-git-sync") yields an extension of "", which
is not in the set, and the BASH descriptor declares no file_names_regex. So the
file is dropped there, before the file_contains_regex shebang match runs — the
descriptor's ^#!/usr/bin/env bash patterns are dead code for these files.

Adding "" makes extension-less files candidates; the descriptor's own
file_contains_regex_extensions: [""] then scopes the shebang check to exactly
that case, so .sh-family files keep being trusted on their extension and
non-shell extension-less files are dropped. get_list() parses a value starting
with [ as JSON, so the empty-string element survives the trip through the
workflow env.

Scope

Deliberately BASH_SHELLCHECK only — Linter.py reads <name>_FILE_EXTENSIONS
where the name is the linter, not the descriptor, so BASH_SHFMT keeps its
current file set. That matters: shfmt would pull the same 18 files into
APPLY_FIXES: all, which runs shfmt -w and would open a reformat PR across
every script in bin/. Worth deciding on its own terms, not as a side effect.

Verified before opening

stage files
tracked 365
extension-less, so candidates after this change 85
dropped by the shebang filter 64
kept 21
minus deprecated/ (2) and the msmtpq exclusion (1) 18 newly linted

The 64 dropped are .gitignore, .chezmoiroot, .lycheeignore, the
shell-history files, the old i3 / polybar / dunst configs. None of the 21 carries
its shebang anywhere but line 1, so the re.MULTILINE match in
file_contains() costs nothing today.

All 18 are clean under shellcheck v0.11.0 — the version MegaLinter v10.0.0
pins (ARG BASH_SHELLCHECK_VERSION=v0.11.0), run in the same
koalaman/shellcheck image with MegaLinter's own --external-sources
arguments. Same version, same arguments, so this is the real check and not a
proxy for it.

Note that this PR's own CI does not exercise the change: VALIDATE_ALL_CODEBASE
is false on pull requests and the diff is one YAML file, so no BASH row appears
here. The BASH row on the push-to-main run goes from 8 files to 26 after merge.

MegaLinter has never linted a single shell script in this repo. Every one of
them is extension-less, and the BASH descriptor ships
file_extensions [".sh", ".bash", ".dash", ".ksh"] with no "" entry.

filter_files() gates on the extension first:

    if file_extension in file_extensions: pass
    elif "*" in file_extensions: pass
    elif file_names_regex_object.fullmatch(base_file_name): pass
    else: continue

os.path.splitext("executable_pai-git-sync") gives an extension of "", which is
not in the set, and BASH declares no file_names_regex — so the file is dropped
before the file_contains_regex shebang match ever runs. The shebang patterns in
the descriptor are dead code for these files.

Symptom: the MegaLinter report on every PR lists only REPOSITORY-level linters,
never a BASH row, even when the diff is a bash script. The existing
BASH_SHELLCHECK_FILTER_REGEX_EXCLUDE and the BASH_EXEC entry in DISABLE_LINTERS
both assume a linter that was not running.

Adding "" makes extension-less files candidates; the descriptor's own shebang
regexes then select the shell ones. Scoped to BASH_SHELLCHECK on purpose:
BASH_SHFMT would pull the same 18 files into APPLY_FIXES, which runs shfmt -w
and would open a reformat PR across all of them. That is a separate decision.

Verified before committing: 18 files become candidates, and all 18 are
shellcheck-clean at 0.9.0, so this turns the gate on without turning CI red.
MegaLinter ships a newer shellcheck, which may add findings the local run does
not see.
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Max errors Warnings Elapsed time
✅ ACTION actionlint 1 0 0 0.02s
✅ ACTION zizmor 1 0 0 0 1.23s
✅ REPOSITORY betterleaks yes no no 0.9s
✅ REPOSITORY checkov yes no no 20.44s
✅ REPOSITORY git_diff yes no no 0.02s
✅ REPOSITORY osv-scanner yes no no 0.39s
✅ REPOSITORY secretlint yes no no 1.99s
✅ REPOSITORY syft yes no no 1.51s
✅ REPOSITORY trivy-sbom yes no no 0.42s
✅ SPELL lychee 1 0 0 0.05s
✅ YAML prettier 1 0 0 0 0.43s
✅ YAML v8r 1 0 0 1.54s
✅ YAML yamllint 1 0 0 1.26s

See detailed reports in MegaLinter artifacts
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

@gwarf

gwarf commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

Two things worth adding before this is merged.

Does adding "" sweep in non-shell files?

It widens the candidate set and then narrows it again, by design. The BASH
descriptor carries a second key I did not mention in the description:

file_extensions: [".sh", ".bash", ".dash", ".ksh"]
file_contains_regex_extensions: [""]
file_contains_regex: ["^#!/usr/bin/env bash", "^#!/bin/bash", "^#!/bin/sh"]

file_contains_regex_extensions: [""] scopes the shebang check to exactly the
extension-less case — .sh-family files skip it and are trusted on their
extension. That key exists for no other purpose than this, and it has been
unreachable because nothing extension-less ever passed the extension gate.

Measured on this tree:

stage files
tracked 365
extension-less, so candidates after this change 85
dropped by the shebang filter 64
kept 21
minus deprecated/ (2) and the msmtpq exclusion (1) 18 linted

The 64 dropped are what you would expect: .gitignore, .chezmoiroot,
.lycheeignore, the shell-history files, the old i3 / polybar / dunst configs.

The residual risk is real but not present today. file_contains() reads the
whole file and matches with re.MULTILINE, so #!/bin/sh at the start of any
line matches, not just line 1 — an extension-less doc quoting a shell snippet
would be picked up. Right now none of the 21 has its shebang anywhere but line

  1. If one ever appears, it is one more entry in the
    BASH_SHELLCHECK_FILTER_REGEX_EXCLUDE that already exists.

This PR's own green does not test the fix

VALIDATE_ALL_CODEBASE is false on pull requests, and the diff here is one YAML
file, so the run linted ACTION, YAML, SPELL and REPOSITORY and still shows no
BASH row. BASH will first appear on the push-to-main run after merge, where
VALIDATE_ALL_CODEBASE is true.

So the check that matters was run locally instead, at a newer shellcheck than
the one MegaLinter ships, with MegaLinter's own arguments:

shellcheck 0.11.0, --color --external-sources, all 18 targets
exit 0, zero findings

Merging should leave main green rather than lighting up a backlog.

@gwarf

gwarf commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

_MREVIEW — Automated Review

Platform: github | Ref: #24 | CI: pass (2 checks)
Rules applied: global

Verdict: approved. Two minor factual points below. Neither blocks, and neither
asks for a code change — one is a correction to the description, the other closes
a risk the description left open.

Premise check

The mechanism holds, verified against MegaLinter at the pinned SHA
(15e5b45, v10.0.0), not against main:

  • megalinter/utils.py filter_files() — the extension gate (lines 303-312)
    runs before the file_contains_regex check (lines 316-333), so an
    extension-less file never reaches the shebang match. As described.
  • megalinter/descriptors/bash.megalinter-descriptor.ymlfile_extensions
    is [.sh .bash .dash .ksh] with no "", there is no file_names_regex, and
    file_contains_regex_extensions: [""] scopes the shebang check to exactly the
    extension-less case. As described.
  • megalinter/config.py get_list() (line 292) JSON-parses a value starting
    with [, so the empty-string element survives the workspace env round trip.
  • Linter.py:294 reads <name>_FILE_EXTENSIONS, and the linter's name is
    BASH_SHELLCHECK, so the override is linter-scoped and BASH_SHFMT keeps its
    current file set. The scope decision in the description is correctly
    implemented, not just correctly reasoned.
  • The documentation flavor does contain BASH_SHELLCHECK
    (descriptors/all_flavors.json), so nothing here risks a missing-linter
    failure.

Reimplementing filter_files() against the PR head tree reproduces exactly the
18 files claimed, with none of the 18 carrying its shebang anywhere but line 1.

Minor — the description's opening line overstates the gap

"MegaLinter has never linted a shell script in this repo" is not right. Eight
.sh files already pass the gate today, because .sh is not in
file_contains_regex_extensions and so skips the shebang check entirely. The
last push-to-main run (32361141513, 2026-08-20, VALIDATE_ALL_CODEBASE
true) shows it:

| BASH | shellcheck | .sh|.bash|.dash|.ksh | 8 | no  |
| BASH | shfmt      | .sh|.bash|.dash|.ksh | 8 | yes |

Both finished with 0 errors. What has never been linted is the 18
extension-less scripts, which is the real and correctly diagnosed gap — the
.sh half of the fleet was already covered. The "never a BASH row" symptom is
accurate for pull requests, where VALIDATE_ALL_CODEBASE is false and the diff
has to contain a shell file, but not for main. After merge that row goes from
8 files to 26.

Minor — the shellcheck version risk is already closed

The description says MegaLinter ships a newer shellcheck than the local 0.9.0
run, and the follow-up comment says the local 0.11.0 run was newer than what
MegaLinter ships. Neither is the case: v10.0.0 pins
ARG BASH_SHELLCHECK_VERSION=v0.11.0 in the BASH descriptor, and the run log
above confirms it at runtime — Using [shellcheck v0.11.0].

So the versions match exactly, which makes the local run the definitive check
rather than a weaker proxy for it. I re-ran it independently in the same image
MegaLinter uses, with MegaLinter's own arguments, over the 18 targets:

docker run koalaman/shellcheck:v0.11.0 --external-sources <18 files>
exit 0, zero findings

The post-merge push run should be green, and the residual uncertainty the
description flags can be dropped.


Posted by _MREVIEW v1.0 on 2026-09-01 CEST

@gwarf

gwarf commented Sep 1, 2026

Copy link
Copy Markdown
Owner Author

Both points confirmed independently, both were mine to get right, and the
description is updated.

The .sh half was already covered. 11 files carry a shell extension, 3 sit
under deprecated/ which is globally excluded, so 8 are linted today — exactly
the count in the run log. .sh is not in file_contains_regex_extensions, so
those files skip the shebang check and pass on their extension alone. My opening
line generalised from pull-request runs, where the diff has to contain a shell
file before a BASH row can appear, to all runs. It does not hold on main. The
gap is the 18 extension-less scripts, and after merge that row goes 8 → 26.

The version risk was never open. v10.0.0 pins ARG BASH_SHELLCHECK_VERSION=v0.11.0 with FROM koalaman/shellcheck:${BASH_SHELLCHECK_VERSION}, so the local run at 0.11.0 is
the same binary MegaLinter uses, not a newer one. I claimed "newer" without
checking the pin, twice — first calling the local 0.9.0 run weaker than
MegaLinter's, then calling the 0.11.0 run stronger. Neither was true and the
correction cuts the right way: same version, same arguments, so the local result
is definitive and the caveat is gone from the description.

Nothing here changes the diff — one env line, unchanged.

@gwarf
gwarf merged commit a6ad050 into main Sep 1, 2026
2 checks passed
@gwarf
gwarf deleted the fix/megalinter-shellcheck-extensionless branch September 1, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant