Difference when run with config file and without

[michasacuer]

So i have a bug where i provide a config file i have a lot of errors in my pipelines.

My Example config file for checkov:

{
    "tools": [
      {
        "tool": {
          "name": "Checkov",
          "version": "Latest"
        },
        "arguments": {
          "DownloadExternalModules": "false",
          "TargetDirectory": "$(Checkov.DefaultTargetDirectory)"
        }
      }
    ]
  }

and everything is fine but i am getting this error, even with false setting:

D:\a\_msdo\packages\nuget\Microsoft.Guardian.CheckovRedist_windows_amd64.3.2.144\tools\dist\checkov.exe --download-external-modules false --directory .\ --output-file-path D:\a\1\s\.gdn\.r\checkov\001\checkov.sarif
##[error]2024-07-08 11:42:17,665 [ThreadPoolEx] [WARNI]  Failed to download module git::https://ADOORG@dev.azure.com/ADOORG/PROJ/_git/kvmodule//src?ref=v0.3:None (for external modules, the --download-external-modules flag is required)

Task still yells that DownloadExternalModules is required even when it is provided and set to false. My pipeline is the simplest template as you can imagine:

parameters:
  TemplatesRepoName: ''

stages:
- stage: Microsoft_Defender
  displayName: Microsoft Defender for Cloud DevOps security
  condition: always()
  pool:
    vmImage: windows-latest
  jobs:
  - job: Microsoft_Defender_Scan
    displayName: Scan
    steps:
    - checkout: ${{ parameters.TemplatesRepoName }}
    - checkout: self
          
    - task: MicrosoftSecurityDevOps@1
      displayName: Microsoft Security DevOps
      inputs:
        config: configs/checkov.gdnconfig

From template above i got a lot of errors:

    Process:
      Convert:
        Converting any raw tool logs to Sarif format ...
        Completed converting raw tool logs to Sarif format.
      Import:
        No tool logs to process. 
    Break:
      Guardian is searching for results that meet the given criteria to break the build.
      Results Query Summary:
        Baselines: default
        Suppression Sets: default
        Policy: azuredevops
      Saved file D:\a\1\a\.gdn\msdo.sarif
      Found no breaking results.
      Active results: 0
      Skipped results: 0
        Baselined results: 0
        Suppressed results: 0
        Results excluded by tool filters: 0
        Results below minimum severity: 0
        Results classified as Pass: 0
        Results in flight: 0
##[error]Error running tool 1 of 1: checkov
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
##[error]BreakException: Guardian detected one or more breaking results.

When i do not provide config file, no issues, only error that DownloadExternalModules is required.

Why this task behaves differently when config file for checkov is provided?

[ptmetcalf]

I'm also seeing this same behavior. If I try to pass a config file even in the simplest form it causes it to fail and not upload any results.

{
  "tools": [
    {
      "tool": {
        "name": "Checkov",
        "version": "Latest"
      },
      "arguments": {
      }
    }
  ]
}

If it helps anyone else, I was able to use a checkov specific config file and it seems the scan picks up and uses that correctly. I just put a .checkov.yml file in my scan directory of my repo. This allowed me to set options as well as choose tests to skip.

directory: ./
output: sarif
soft-fail: true
framework: terraform
skip-check:
  - CKV_AZURE_190
  - CKV2_AZURE_1

You can see the details of the checkov specific config file on there site. Configuration using a config file

[DimaBir]

We're cleaning up the issue backlog in this repo, and this one has been inactive for a long time, so we're closing it.

If it's still affecting you, just reopen it or drop a comment — that's enough, and we'll pick it up from there. Nothing else needed from you.

Sorry we left this sitting.

[michasacuer]

We're cleaning up the issue backlog in this repo, and this one has been inactive for a long time, so we're closing it.

If it's still affecting you, just reopen it or drop a comment — that's enough, and we'll pick it up from there. Nothing else needed from you.

Sorry we left this sitting.

I just moved to another product. Why did you close this request? Do you assume it was fixed on its own?

[DimaBir]

Thanks for the report, and for the detail in it — the config, the emitted command line, and the log excerpt are what made this diagnosable.

Confirmed as a bug.

Root cause. The arguments block in a .gdnconfig replaces the MSDO CLI's default argument set for that tool rather than merging into it. Two defaults you hadn't specified were therefore dropped:

• OutputType: sarif — without it Checkov emits its default cli format, so nothing SARIF-shaped lands at the --output-file-path that MSDO subsequently reads. That produces No tool logs to process.
• Soft: true — without it Checkov's normal "failed checks found" exit 1 propagates instead of being handed to MSDO's own policy. That produces GuardianErrorExitCodeException: checkov completed with an Error exit code: 1.

This is also why the run behaved differently with no config file at all: that path keeps the full default set intact.

The WARNI ... the --download-external-modules flag is required line is Checkov's expected message when it encounters a git:: module source with downloads disabled, and is unrelated to the failure.

Workaround. Carry the dropped defaults explicitly:

{
  "tools": [
    {
      "tool": { "name": "Checkov", "version": "Latest" },
      "arguments": {
        "TargetDirectory": "$(Checkov.DefaultTargetDirectory)",
        "OutputType": "sarif",
        "Soft": true,
        "DownloadExternalModules": "false"
      }
    }
  ]
}

Note the key is Soft, not SoftFail — credit to newbloke82 in #130, who recovered the CLI's own default configuration via guardian configure -t checkov and confirmed the correct spelling. Running that command prints the full canonical default argument set for any tool, which is the reliable way to see what a hand-written arguments block would otherwise drop.

Fix. The merge happens inside the closed-source MSDO/Guardian CLI, so it is not addressable in this repository. It is being routed to that team as a single defect covering the four reports that share this root cause — #47, #97, #104 and #130 — since the same layer also mis-serializes list-valued arguments and mishandles mutually exclusive tool verbs.

Tracking here until it lands.