An invalid pattern in a JSON schema throws and kills the whole run

[royalpinto007]

Desired outcome

An invalid pattern in a user's JSON schema produces a scorer failure, not a crash.

Why it matters

validate() in src/scorers/json-schema.ts compiles a user-supplied regex with no guard:

if (schema.pattern && !new RegExp(schema.pattern).test(value))
  errors.push(`${path}: does not match pattern ${schema.pattern}`);

schema comes straight from the eval file, so pattern is arbitrary user input. A typo such as "[a-z" makes new RegExp throw a SyntaxError, which propagates out of validate, out of the scorer, and up through the runner, taking the whole eval run with it. The user sees a raw stack trace mentioning RegExp and nothing that points at which case or which schema key is at fault.

The sibling scorer already handles this properly. src/scorers/regex.ts wraps compilation in try/catch and returns:

reason: `invalid regex: ${(err as Error).message}`,

so the same class of mistake is a clean, attributable failure in one scorer and a crash in the other.

The regex is also recompiled on every call, once per case per run, which is wasteful but secondary.

Steps

1. Wrap the new RegExp(schema.pattern) in try/catch inside validate().
2. On a compile error, push a descriptive entry into errors, for example `${path}: invalid pattern ${schema.pattern}: ${message}`, so the failure carries the JSON path and is reported like any other validation error.
3. Add a test in tests/scorers.test.ts with a schema containing an unclosed character class, asserting the scorer returns a failing result rather than throwing.

Small and self-contained, one function.

Claiming this

Comment below to claim it. A reply usually comes within a day.

[vsolano9]

I'd like to work on this reopened issue. I'll keep the fix inside the JSON Schema validator, add regression coverage proving an invalid nested pattern returns a path-attributed scorer failure instead of throwing, and run the full repository gate.

[github-actions]

It's yours, @vsolano9. Marked this as claimed.

Take your time, and shout in this thread if you get stuck. If there's no activity for 14 days the claim is released so someone else can pick it up.