Support multiple --text-group values in a single validation pass

[cmungall]

Feature request

Problem

When using validate text-file with a regex that has multiple text capture groups, only one can be validated per invocation. For structured reference assertions like:

- @ref PMID:32550677 "Multiple Acyl-CoA Dehydrogenase Deficiency" excerpt: "disorder of fatty acid and amino acid oxidation"

The regex @ref PMID:(\d+) "([^"]*)" excerpt: "([^"]*)" produces three groups:

1. PMID number (reference)
2. Title (text to validate)
3. Excerpt (text to validate)

To validate both title and excerpt, you must run the command twice:

# Validate titles
uvx linkml-reference-validator validate text-file doc.md \
  --regex '@ref PMID:(\d+) "([^"]*)" excerpt: "([^"]*)"' \
  --ref-group 1 --text-group 2

# Validate excerpts
uvx linkml-reference-validator validate text-file doc.md \
  --regex '@ref PMID:(\d+) "([^"]*)" excerpt: "([^"]*)"' \
  --ref-group 1 --text-group 3

Proposed solution

Allow --text-group to accept multiple values:

uvx linkml-reference-validator validate text-file doc.md \
  --regex '@ref PMID:(\d+) "([^"]*)" excerpt: "([^"]*)"' \
  --ref-group 1 --text-group 2 --text-group 3

This would validate both capture groups against the reference in a single pass, reporting results for each group separately.

Nice-to-have

A warning when the regex has more capture groups than are referenced by --text-group and --ref-group — this helps catch off-by-one errors in group numbering (which I hit during initial use).

Use case

Validating structured reference annotations in markdown analysis documents where each citation includes both a title and a supporting text excerpt that should both be verified against the cited publication.

[github-actions]

Issue Summary

Type: Feature request — UX improvement for multi-group regex validation

Status: Open, no prior discussion

---

Problem in brief

validate text-file currently only accepts a single --text-group integer. When a regex captures more than one text field (e.g. both a title and an excerpt), users must invoke the command twice, scanning and fetching the same reference twice — wasteful and annoying.

Motivating pattern:

- @ref PMID:32550677 "Multiple Acyl-CoA Dehydrogenase Deficiency" excerpt: "disorder of fatty acid and amino acid oxidation"

Regex @ref PMID:(\d+) "([^"]*)" excerpt: "([^"]*)" → 3 groups:

1. PMID (ref) 2. Title (text) 3. Excerpt (text)

---

What needs to change

Two files are involved:

1. src/linkml_reference_validator/etl/text_extractor.py

TextExtractor.__init__ takes text_group: int. This needs to become text_group: int | list[int]. extract_from_text then emits one ExtractedTextMatch per text group per regex match, so a 3-group pattern with --text-group 2 --text-group 3 produces 2 matches per line hit (one for the title, one for the excerpt).

2. src/linkml_reference_validator/cli/validate.py

The text_file_command parameter:

text_group: Annotated[int, typer.Option(...)] = 1

needs to become a List[int] so Typer accepts repeated --text-group flags:

text_group: Annotated[List[int], typer.Option(...)] = [1]

3. Nice-to-have (also requested)

After compiling the regex, count self._compiled_regex.groups and warn if that count exceeds len(text_groups) + 1 (for the ref group) — helps catch off-by-one mistakes in group numbering.

---

Design notes

• Each text group should produce an independent ExtractedTextMatch with its own supporting_text; the reference_id is the same for all groups from the same regex match.
• ValidationReport already aggregates multiple results, so the validation loop in text_file_command needs no structural change — just iterate over the expanded match list.
• Default behaviour (--text-group 1) stays unchanged (backward compatible).
• TDD approach: write tests for TextExtractor(text_group=[2, 3]) yielding two matches per hit before implementing.

---

Effort estimate

Small/medium — 2 files, no new dependencies, straightforward change with good test coverage.

@dragon-ai-agent please implement the feature described in issue #40: support multiple --text-group values in a single validation pass. Follow TDD — write tests first in tests/, then update TextExtractor in src/linkml_reference_validator/etl/text_extractor.py to accept text_group as int | list[int], and update the CLI in src/linkml_reference_validator/cli/validate.py to accept repeated --text-group flags. Also add the nice-to-have warning when the regex has more capture groups than are referenced by --text-group and --ref-group. Run just test to confirm all tests pass.

[cmungall]

Clarification: distinct validation semantics per group

On reflection, the multiple --text-group suggestion above doesn't fully capture the issue. The real problem is that structured reference assertions have fields with different validation semantics, but the tool treats everything as a substring search.

Consider this assertion format:

@ref PMID:32550677 "Multiple Acyl-CoA Dehydrogenase Deficiency" excerpt: "disorder of fatty acid and amino acid oxidation"

There are three things to validate, each with different semantics:

Field	What it is	How it should be validated
PMID:32550677	Reference ID	Resolve — does this PMID exist? Return metadata.
"Multiple Acyl-CoA Dehydrogenase Deficiency"	Title	Title match — is this the actual title of the paper? (Exact or fuzzy match against the paper's title metadata field.)
"disorder of fatty acid and amino acid oxidation"	Excerpt	Substring search — does this text appear somewhere in the paper's content?

Currently the tool only supports the third semantic (substring search on fetched content), and we end up abusing it for all three:

• Resolve works implicitly because fetching fails for non-existent PMIDs
• Title match works accidentally because PubMed happens to include the title in the fetched text — but it's really doing a substring search, not a title match. If the fetched content format changed, or if a paper's title happened to be a common phrase, this would silently give wrong results.

Proposed design

Support named group roles rather than just numbered text groups:

uvx linkml-reference-validator validate text-file doc.md \
  --regex '@ref (\S+) "([^"]*)" excerpt: "([^"]*)"' \
  --ref-group 1 \
  --title-group 2 \
  --text-group 3

Where:

• --ref-group: resolved via PubMed/DOI/etc. (existence check + metadata retrieval)
• --title-group: compared against the paper's title metadata (not substring search)
• --text-group: substring search in paper content (current behavior)

This would give clear, correct semantics for each field rather than collapsing them all into substring matching.