Skip to content

[FEATURE] 解析面缺少属性/模糊测试,未知输入没有覆盖 #280

Open

Description

@codeasier

功能描述

全部 140 个测试模块、5073 条断言都建立在手写样本与显式参数化上,没有属性测试或模糊测试。参数化只能证明“列出的输入是对的”,无法证明不变量,例如:

  • 任意合法 offset / limit / max_rows 组合下,分页拼接结果与一次性查询一致
  • 任意畸形 YAML 模板、缺失字段、未知字段都落到受控的域错误,而不是 traceback
  • 任意畸形 pickle 元数据(缺 key、类型错误、数值超范围)都进入受控错误路径

现有覆盖是枚举式样本,正确性取决于 reviewer 是否想到了对应样本。

使用场景

修改模板加载、参数校验、元数据解析时缺少自动生成的反例;#223 曾提出属性测试方向,但只针对 dataset 分片夹具且已关闭,其余解析面仍是空白。

期望的实现方式

No response

示例

$ grep -rho "pytest.mark.parametrize" tests --include=*.py | wc -l
384
$ grep -rln "hypothesis" tests --include=*.py | wc -l
0

参考资源

No response

验收标准

No response

Activity

  1. codeasier commented on Nov 1, 2026

    @codeasier
    OwnerAuthor

    Separately authorized runtime follow-up is now Draft PR #286, kept out of tests-only #282. It adds explicit input-boundary validation rather than accepting AttributeError/unchecked container failures as correct behavior. Minimal cases include QueryConfig.load_yaml_from_string("queries: []") and snapshot device_traces=[None] / [{}]; generated regressions require controlled errors, preserve old destinations, prevent new publication and check cleanup. It also covers missing required snapshot keys and real SQLite numeric overflow. CI is pending at 8df94284b2eb19bc1e6a3442a145d8244d55446d; no silent xfail or weakened publication assertion.

    This issue remains open. Existing compatible YAML extension fields and defaults (for example queries: {demo: {}}, omitted/null SQL, and empty documents) are currently valid rather than malformed. Rejecting every unknown/missing field would impose a new closed schema. That compatibility scope must be resolved explicitly before claiming the issue's universal wording is complete.

  2. codeasier commented on Nov 1, 2026

    @codeasier
    OwnerAuthor

    Compatibility clarification after checking the pre-existing docs/tests (not relying on the new properties): a template may legitimately omit query when declaring complete query_variants (src/pt_snap_cli/query/templates/AGENTS.md:29–31; existing test_from_dict_loads_query_variants). Optional defaults and empty configuration/template objects also have existing tests. Empty YAML/null SQL and ignored unknown root/query/parameter-definition fields are existing loader compatibility, not a newly promised general extension API; no conflicting closed-schema rejection rule was found.

    Accordingly, this minimal repair preserves those defaults/compatibility and requires controlled errors for illegal shapes/types and missing required fields. It does not invent a breaking closed schema merely from the issue's broad wording. Unknown runtime argument names and unknown output_schema fields remain rejected under their existing explicit contracts. The issue stays open for the actual #282/#286 acceptance matrix and CI, not for a presumed unresolved user design decision.

  3. codeasier commented on Nov 1, 2026

    @codeasier
    OwnerAuthor

    状态纠正:本 issue 曾在测试专批 PR #282 合入时(2026-11-01 15:45:46 UTC)被 GitHub 自动标为 completed。正文使用了带关闭关键字的否定句,GitHub 仍识别为关闭指令;这是状态管理失误,之前的“保持开放”描述未反映实际 GitHub 状态。现已于 17:06:50 UTC 重开,并移除 #282/#286 正文的自动关闭语法,不重写提交历史。

    #282 仅完成可独立交付的属性测试;另批最小运行时修复 #286 仍是 Draft。当前最终集成候选 bb2bde0e55cc7e7bf789755b3b94aa969b115186 等待精确提交普通 CI,以及必须实际执行的 manual installed-wheel dataset + 200-seed paired differential 验收,之后还需要正常合入与合后 CI。未把已知缺陷定义为正确行为,未使用 xfail/skip 掩盖。本 issue 保持开放,完整验收有证据后才显式完成。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions