Skip to content

fix(workflows): reject a non-string 'integration'/'model' in command & prompt steps#3597

Open
Noor-ul-ain001 wants to merge 3 commits into
github:mainfrom
Noor-ul-ain001:fix/step-nonstring-integration-crash
Open

fix(workflows): reject a non-string 'integration'/'model' in command & prompt steps#3597
Noor-ul-ain001 wants to merge 3 commits into
github:mainfrom
Noor-ul-ain001:fix/step-nonstring-integration-crash

Conversation

@Noor-ul-ain001

Copy link
Copy Markdown
Contributor

Problem

A command or prompt step whose integration: (or model:) is a non-string crashes the run with a raw TypeError instead of a clean validation/step error.

In execute(), the resolved integration value is passed to get_integration(), which does INTEGRATION_REGISTRY.get(key) — using the value as a dict key. An unhashable value (list/dict) raises TypeError: unhashable type right there:

>>> from specify_cli.workflows.steps.command import CommandStep
>>> from specify_cli.workflows.base import StepContext
>>> CommandStep().execute({"id": "x", "command": "speckit.specify", "integration": ["claude"]}, StepContext())
TypeError: cannot use 'list' as a dict key (unhashable type: 'list')

Crucially this is not limited to unvalidated runs: neither the step's validate() nor validate_workflow checked the type of integration/model, so validate() returns [] for integration: [claude] and the crash survives validation. A non-string model similarly reaches build_exec_args() and would be fed into the CLI argv.

This is the same recurring shape as the existing command/prompt/input/options guards — a string-typed dispatch field that execute() passes downstream without a type check.

Fix

Both command and prompt steps:

  • validate() — reject a literal non-string integration/model, mirroring the existing sibling type checks. An explicit YAML-null (integration: → inherit the workflow default) and a "{{ ... }}" expression both stay valid.
  • execute() — guard the resolved value (after the or-fallback to the workflow default and expression evaluation) and return a clean FAILED StepResult rather than letting the run crash. This also catches an expression that resolves to a non-string and an unvalidated workflow-level default.

Tests

Added parametrized validate() rejection tests, "accepts None / expression" tests, and execute()-fails-loudly tests to both TestCommandStep and TestPromptStep. Test-the-test: with the fix stashed, the new tests fail — including the raw TypeError: unhashable type traceback at get_integration. Full tests/test_workflows.py shows no new failures (the 20 pre-existing failures are all Windows symlink-guard tests, identical on a clean tree).

🤖 Generated with Claude Code

…& prompt steps

A non-string `integration` on a command or prompt step is passed to
`get_integration()`, which uses it as a dict key: an unhashable list/dict
raises a raw `TypeError` there — and because neither `validate()` nor
`validate_workflow` checked the type, this crashes even a *validated* run,
not just an unvalidated one. A non-string `model` likewise reaches
`build_exec_args()` and is fed into the CLI argv.

Guard both fields in `validate()` (reject a literal non-string, mirroring the
existing 'command'/'prompt'/'input'/'options' checks) and in `execute()`
(fail the step cleanly rather than take down the whole run, mirroring the
'input'/'options' guards). An explicit YAML-null (inherit the workflow
default) and a "{{ ... }}" expression both stay valid.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds validation and runtime guards for non-string command/prompt integration and model values.

Changes:

  • Rejects invalid literal values during validation.
  • Returns failed step results for invalid resolved values.
  • Adds validation and execution tests.
Show a summary per file
File Description
src/specify_cli/workflows/steps/command/__init__.py Validates command dispatch fields.
src/specify_cli/workflows/steps/prompt/__init__.py Validates prompt dispatch fields.
tests/test_workflows.py Tests new validation and failure behavior.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 2
  • Review effort level: Medium

Comment on lines +64 to +65
# A non-string integration/model — a literal list/dict/number that
# skipped validation, an unvalidated workflow-level default, or an
Comment on lines +55 to +56
# A non-string integration/model — a literal list/dict/number that
# skipped validation, an unvalidated workflow-level default, or an
@mnriem

mnriem commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and resolve conflicts

Noor-ul-ain001 and others added 2 commits July 22, 2026 01:57
… guard

Address Copilot review: `config.get("integration") or context.default_integration`
(and the model equivalent) coerced a *falsey* non-string ([], {}, 0, False) into
the workflow default before the type guard ran. On an unvalidated execute() such a
step was silently accepted and — with a configured default — could dispatch using
the wrong integration/model instead of failing with the contract error.

Fall back to the workflow default only for genuinely-unset values (missing /
YAML-null / empty string) so every non-string reaches the guard. Add parametrized
falsey execute() cases ([], {}, 0, False) to both TestCommandStep and
TestPromptStep; with the fix stashed all 8 fail (swallowed into the default).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…integration-crash

# Conflicts:
#	tests/test_workflows.py
@Noor-ul-ain001

Copy link
Copy Markdown
Contributor Author

@
Addressed the Copilot feedback and resolved the merge conflicts.

Copilot feedback (falsey non-string bypass). Both flagged spots resolved the value with config.get("integration") or context.default_integration (and the model equivalent), which coerces a falsey non-string — [], {}, 0, False — into the workflow default before the new type guard runs. Since execute() is explicitly supported without prior validation, such a value was silently accepted and, with a configured default, could dispatch using the wrong integration/model instead of returning the contract error.

Fixed in both command and prompt steps by falling back to the workflow default only for a genuinely-unset value (missing / YAML-null / empty string):

integration = config.get("integration")
if integration is None or integration == "":
    integration = context.default_integration

so every non-string now reaches the guard.

Tests. Added parametrized execute() cases over [], {}, 0, False to both TestCommandStep and TestPromptStep, each with default_integration/default_model set so a regression would dispatch-with-the-default rather than fail-not-possible. Test-the-test: with the source fix stashed, all 8 fail; with the fix, they pass.

Conflicts. Merged upstream/main; the only conflict was in tests/test_workflows.py where both sides added new test methods to TestCommandStep (the incoming non-string command guard tests and these integration/model tests) — kept both.

Full tests/test_workflows.py: 677 passed, 7 skipped, 20 failed — the 20 are all the pre-existing Windows symlink-guard tests, identical on a clean tree and unrelated to this change.
@

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants