diff --git a/CHANGELOG.md b/CHANGELOG.md index f129664..ad2e649 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,68 @@ what has already shipped, and for nothing else. See `RELEASING.md`. ### Fixed +- **The law check carries mailradar's fixes of 2026-11-17 (maksimtech/mailradar#103).** `law_fetcher.py` + and `law_cache.py` were the copies mailradar had before that pull request, apart from the variable + `PATCHRADAR_HOME`; they are now its copies after it, with the same name kept. + - *A citation is "verified" only when the reference it cites is in the text downloaded now.* The + check looked at the article: with article 32 in Cellar's page and paragraph 2 missing from what was + parsed, the citation of `32(2)` for a critical CVE fell back to the cache, old SHA-256 and date + included, under "verified against Cellar", or stood beside `SHA256: not available` with an empty + cache. A missing reference now makes the act a fetch failure. + - *A failed download always reaches the EUR-Lex fallback and the cache*: a redirect to a host httpx + cannot encode (`http://xn--/DOC_1`) raised `idna.core.IDNAError` out of the whole law check. + - *Redirects stay on the source's host* (publications.europa.eu, eur-lex.europa.eu, www.normattiva.it, + default port), so the hosts in `docs/nis2-compliance.md` are the only ones the check can reach. + - *A page is read up to 32 MiB, and a fetch ends after 120 seconds*, redirects included. + - *A change in the law is reported even when the cache cannot be written*: the comparison was made + by the save, so with a read-only cache a changed text was cited without the warning and an unchanged + one was dated to the day of the audit. `law_cache.merge` now runs before the save. + - *Statuses name the source that was tried.* `ActStatus` carries `tried`, `failure` and `reason`, and + `scan` prints "Cellar answered without the text cited" or "Cellar and EUR-Lex unreachable" where it + printed "EUR-Lex unreachable" whatever had happened. + - *A deeply nested `law_cache.json` is an empty cache*, not a `RecursionError`; Normattiva's "-bis", + "-ter", ... paragraphs can be cited on their own. + +- **The law check no longer cites CVEs the same scan judged "not affected".** `scan` handed every CVE + found by name to the law check, verdict or not: for 7-Zip 26.06 it printed "not affecting it: 2" and + then cited CVE-2025-0411 (fixed in 24.09) and CVE-2026-14266 (fixed in 26.02) under GDPR art. 25 and + art. 32, and a critical unpatched one would have reached D.Lgs. 138/2024 art. 24 too. Only the CVEs + that affect the installed version, could not be evaluated, or belong to an entry matched by name are + cited now. The API runs no law check, so it had nothing to change. + +- **A CPE typed with capitals no longer turns every CVE into "not affected".** `add --cpe` and + `POST /api/watchlist/{name}?cpe=` accepted `cpe:2.3:a:Mozilla:Thunderbird` as typed, and the verdict + compared its vendor and product with NVD's lower-case `cpeMatch` names as strings: nothing matched, so + every CVE of the program was judged "not affected" and hidden from the table — a false all-clear. CPE + 2.3 names match without regard to case: the CPE is now stored in lower case, and the verdict compares + vendor and product without regard to case, which also covers entries stored before. + +- **"Affected" only when NVD's statement is about the version alone.** The verdict read the version of a + `cpeMatch` entry and ignored the rest: the fields after the version and the AND configuration around + it. On NVD's own records (2026-11-17), Chrome 150 on Windows was "affected" by CVE-2026-16424, which NVD + lists for Chrome *on Android* only, and Chrome 151 by CVE-2026-17709, macOS only; Firefox ESR 115.17.0 + was "affected, fix in 131.0.2" by CVE-2024-9680, whose ESR line was fixed in 115.16.1; and 7-Zip 4.43 + was affected by entries written `7-zip:4.43:beta`. The watchlist knows the version and the CPE, not the + operating system: a range that contains the version only on a platform an AND configuration requires, + or only for an update or edition the watched CPE does not name, now gives "not evaluable" with the + condition in the reason ("NVD lists it as affecting 150.0.7871.100 only on Android"). A CPE that names + its edition or update (`cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*`) is judged on those ranges only, + in the CLI and the API alike. A `-` counts as a condition only where the record names the product with + another value in the same field, so VS Code's lone `target_sw: -` changes nothing. + +- **The exploited CVEs of a product with a CPE are found whatever CISA calls it.** The KEV collector + matched the watchlist name against CISA's `vendorProject` and `product` only. CISA files Chrome under + "Google / Chromium V8", "Chromium Mojo", "Chromium" and so on: on the 2026-11-17 catalogue, "chrome" + matched 6 of the 72 exploited Chrome CVEs, and a scan reported the other 66 as not known to be + exploited. For an entry with a CPE the catalogue is now also read through NVD, which flags every CVE in + KEV and maps it to the products it affects (`hasKev` with `virtualMatchString=cpe:2.3::: + `, one paged request per entry): for `cpe:2.3:a:google:chrome` it returned the 72 and six more + whose CISA entry names no browser (Adobe Flash Player, WebRTC, Skia, Dawn). NVD's records come with + them, so those CVEs are judged without a lookup by id each. An alias table (chrome → Chromium) was the + alternative; it would have fixed one product and left every other one CISA names by a component as it + was. Entries without a CPE are still matched by name; if NVD cannot be asked, the name's matches stand + and the scan says it is incomplete. `docs/nis2-compliance.md` lists the new query. + - **The weekly mutation run gets past its stats phase again.** It had been red since 2026-10-24, four Saturdays, on `assert [13226] == []` in `tests/test_sql_is_not_formatted.py`. mutmut runs the suite from `mutants/`, where database.py holds every mutant beside the code, and one of them drops the diff --git a/README.md b/README.md index ae3912b..da7f82c 100644 --- a/README.md +++ b/README.md @@ -89,9 +89,12 @@ affected versions: - **affected** — listed in the table, with "fixed in X" only where NVD states it; - **not evaluable** — NVD has not yet said which products the CVE affects (the usual state for weeks after publication), NVD could not be asked about a CVE - that came from KEV or MSRC alone — those are looked up in NVD by id — or NVD's - versions belong to another scheme: listed, because "cannot tell" is not an - all-clear; + that came from KEV or MSRC alone — those are looked up in NVD by id — NVD's + versions belong to another scheme, or NVD lists the version only on another + platform (Chrome on Android) or for an update or edition the CPE does not name + (the ESR line of Firefox, a beta): listed, with the condition, because + "cannot tell" is not an all-clear. A CPE that names its edition, such as + `cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*`, is judged on that edition; - **not affected** — counted, and listed only with `--all`. One line gives the split, one says which update closes how many of the @@ -228,6 +231,13 @@ is worth more than a score: on a sample of 695 CVEs matched against installed software, sorting by CVSS put five 10.0 entries on top, none of them exploited, while the four listed in KEV scored 9.8, 8.8, 8.6 and 7.8. +CISA names products its own way, so for an entry added with `--cpe` the catalogue +is also read through NVD, which flags every CVE in KEV and knows which products +it affects: on 2026-11-17 the word "chrome" was in 6 of the 72 exploited Chrome +CVEs (the rest are "Chromium V8", "Chromium Mojo", ...), and NVD asked for +`cpe:2.3:a:google:chrome` returned all 72 and six more. Entries without a CPE +are matched by name only. + EPSS is the row marked *enrichment* because it finds nothing. The four sources above answer "which CVEs affect this software"; FIRST answers "how likely is this one to be exploited in the next 30 days", for a CVE you already have. So it diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 82e3250..3112f90 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -72,6 +72,11 @@ product, neither `*` nor `-`. A product written with a CPE special character unescaped — `notepad++` instead of `notepad\+\+` — is refused with the escaped form shown, because NVD would match nothing and the scan would read as clean. +CPE 2.3 names match without regard to case, and NVD writes them in lower case: +the CPE is stored in lower case, and the verdict compares vendor and product +without regard to case. Compared as typed, `cpe:2.3:a:Mozilla:Thunderbird` +matched nothing and made every CVE "not affected". + The version must be one `affected.py` can order. One it cannot read is refused at `add`, not discovered as "not evaluable" on every CVE of every scan. @@ -88,7 +93,14 @@ name is judged against the installed version with NVD's own `configurations`: which products the CVE affects (no `configurations`, the usual state of a CVE for weeks after publication), NVD could not be asked about a CVE that came from KEV or MSRC alone, or NVD's versions belong to another scheme than the - installed one (`2.07` against `16.0.14334`). + installed one (`2.07` against `16.0.14334`). Also a CVE whose range contains + the version only under a condition the watchlist cannot confirm, which the + reason names: an AND configuration with another product, mostly the + operating system (Chrome *on Android*), or an update or edition the watched + CPE does not name (`7-zip:4.43:beta`, Firefox `sw_edition: esr` beside `-`). + A CPE that names it (`cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*`) is judged + on that edition's ranges only. A `-` in a field counts as a condition only + when the record names the product with another value there too. A CVE that reached the scan only through KEV or MSRC, which carry no `cpeMatch`, is looked up in NVD by id (`cveId=`) for an entry that can be judged, diff --git a/docs/nis2-compliance.md b/docs/nis2-compliance.md index 3ea0434..e21721c 100644 --- a/docs/nis2-compliance.md +++ b/docs/nis2-compliance.md @@ -107,7 +107,7 @@ cosa ricevono: | Host | Quando | Cosa riceve | | --- | --- | --- | -| `services.nvd.nist.gov` | `scan`, da riga di comando o dall'interfaccia web | il nome di ogni software della watchlist (o quello passato a `scan`) come parola chiave di ricerca, con l'intervallo di date; per un software con versione installata e CPE, l'identificativo di ogni CVE trovata solo da CISA KEV o MSRC (`cveId`); la chiave `NVD_API_KEY`, se impostata, in un'intestazione | +| `services.nvd.nist.gov` | `scan`, da riga di comando o dall'interfaccia web | il nome di ogni software della watchlist (o quello passato a `scan`) come parola chiave di ricerca, con l'intervallo di date; per un software con versione installata e CPE, l'identificativo di ogni CVE trovata solo da CISA KEV o MSRC (`cveId`); per un software con CPE, la parte, il fornitore e il prodotto della CPE (`virtualMatchString`, con `hasKev`), per trovare le CVE del catalogo KEV che NVD attribuisce a quel prodotto; la chiave `NVD_API_KEY`, se impostata, in un'intestazione | | `api.msrc.microsoft.com` | `scan` | il mese dei bollettini richiesti; nessun nome di software, il filtro avviene sulla macchina | | `www.cisa.gov` | `scan` | nulla: scarica il catalogo KEV intero e lo filtra sulla macchina | | `security-tracker.debian.org` | `scan` dall'interfaccia web; `patchradar debian` senza `--file` | nulla: scarica il tracker intero e lo filtra sulla macchina | diff --git a/src/patchradar/affected.py b/src/patchradar/affected.py index 70b8ec6..2788dc3 100644 --- a/src/patchradar/affected.py +++ b/src/patchradar/affected.py @@ -38,6 +38,7 @@ from dataclasses import dataclass from patchradar.collectors.nvd import CVSS_METRIC_KEYS, scoring_entry +from patchradar.cpe import WILDCARDS, Cpe, split_fields from patchradar.cvss import severity_for # A dotted numeric version, optionally ending in lower-case letters the way @@ -119,6 +120,11 @@ class Range: # No exact, no start, no end: NVD naming the product with a `*` version, # which means every version of it. unbounded: bool = False + # What else NVD's statement needs, besides the version, that the watchlist + # cannot confirm: an update or edition the watched CPE does not name ("for + # update beta"), or another product an AND configuration requires ("on + # Android"). A range with conditions never makes a version "affected". + conditions: tuple[str, ...] = () def contains(self, version: str) -> bool: have = _parts(version) @@ -257,48 +263,159 @@ def _score_of(cve: dict) -> tuple[float | None, str | None, str]: return None, None, "UNKNOWN" -def ranges_in(cve: dict, *, vendor: str, cpe_product: str | None = None) -> list[Range]: - """The affected ranges this CVE declares for that vendor, and product. - - `cpe_product` is the product field of the CPE in its escaped form — the - `notepad\\+\\+` that the API wants, not the `Notepad++` a person reads. - Passing the readable name raises nothing and matches nothing, which reads as - "no vulnerabilities". +# The CPE 2.3 fields after the version, by their index in the formatted string, +# and how a reason names a value of each. +_FIELDS = { + 6: lambda v: "for the release with no update" if v == "-" else f"for update {v}", + 7: lambda v: f"for edition {v}", + 8: lambda v: f"for language {v}", + 9: lambda v: "for the standard edition" if v == "-" else f"for the {v} edition", + 10: lambda v: f"for target software {v}", + 11: lambda v: f"for target hardware {v}", + 12: lambda v: f"for {v}", +} + +# How the operating systems NVD's AND configurations require are called in a +# reason; any other product is named by its CPE product field. +_PLATFORM_NAMES = { + "android": "Android", "macos": "macOS", "mac_os_x": "macOS", "mac_os": "macOS", + "windows": "Windows", "linux_kernel": "Linux", "iphone_os": "iOS", "ipados": "iPadOS", + "chrome_os": "ChromeOS", "chromeos": "ChromeOS", +} + + +def _same(fields: list[str], vendor: str, product: str | None) -> bool: + return (len(fields) >= 6 and fields[3].lower() == vendor + and (product is None or fields[4].lower() == product)) + + +def _platform(entries: list[dict]) -> str: + """The products a node lists, as a reason names them: "on Android".""" + names, systems = [], True + for entry in entries: + fields = split_fields(str(entry.get("criteria", ""))) + if len(fields) < 5: + continue + product = fields[4].lower() + name = _PLATFORM_NAMES.get(product, product.replace("\\", "").replace("_", " ")) + systems = systems and fields[2] == "o" + if name not in names: + names.append(name) + listed = " or ".join(names) or "another product" + return f"on {listed}" if systems else f"with {listed}" + + +def _field_conditions(fields: list[str], seen: dict[int, set[str]], + watched: Cpe | None) -> list[str] | None: + """What the fields after the version require, or None when the watched CPE + names a different value: then the statement is about another update or + edition, and does not apply. + + A specific value (`beta`, `esr`) is always a condition. `-` ("none": the + release with no update, the standard edition) is one only when the record + also names the product with another value in that field — Firefox's `-` + beside `esr`; VS Code's lone `target_sw: -` chooses between nothing. """ - found: list[Range] = [] - for configuration in cve.get("configurations") or []: - if not isinstance(configuration, dict): + conditions = [] + for index, say in _FIELDS.items(): + value = fields[index].lower() if len(fields) > index else "*" + if value == "*": + continue + mine = watched.field(index) if watched is not None else "*" + if mine != "*": + if mine != value: + return None + continue + if value == "-" and not (seen.get(index, set()) - WILDCARDS): continue + conditions.append(say(value)) + return conditions + + +def _ranges(cve: dict, vendor: str, cpe_product: str | None, + watched: Cpe | None) -> tuple[list[Range], int]: + """The ranges for that vendor and product, and how many entries were left + out because the watched CPE names another update or edition.""" + vendor = vendor.lower() + product = cpe_product.lower() if cpe_product is not None else None + configurations = [c for c in cve.get("configurations") or [] if isinstance(c, dict)] + + # Which values each field takes across the product's vulnerable entries, + # which is what makes a `-` a choice between editions or not. + seen: dict[int, set[str]] = {} + for configuration in configurations: for node in configuration.get("nodes") or []: - if not isinstance(node, dict): - continue - for entry in node.get("cpeMatch") or []: - if not isinstance(entry, dict) or not entry.get("vulnerable", False): + for entry in (node.get("cpeMatch") or []) if isinstance(node, dict) else []: + if isinstance(entry, dict) and entry.get("vulnerable", False): + fields = split_fields(str(entry.get("criteria", ""))) + if _same(fields, vendor, product): + for index in _FIELDS: + seen.setdefault(index, set()).add(fields[index].lower() if len(fields) > index else "*") + + found: list[Range] = [] + excluded = 0 + for configuration in configurations: + nodes = [n for n in configuration.get("nodes") or [] if isinstance(n, dict)] + joined = str(configuration.get("operator", "")).upper() == "AND" + for node in nodes: + entries = [e for e in node.get("cpeMatch") or [] if isinstance(e, dict)] + node_joined = str(node.get("operator", "")).upper() == "AND" + for entry in entries: + if not entry.get("vulnerable", False): # `vulnerable: false` describes the context in which another # component is vulnerable. On CVE-2017-8803 the Notepad++ # node is exactly that, and counting it reports a CVE that # does not concern the program. continue - fields = str(entry.get("criteria", "")).split(":") - if len(fields) < 6 or fields[3] != vendor: + fields = split_fields(str(entry.get("criteria", ""))) + if not _same(fields, vendor, product): continue - if cpe_product is not None and fields[4] != cpe_product: + conditions = _field_conditions(fields, seen, watched) + if conditions is None: + excluded += 1 continue + # An AND of this node with others: Chrome *and* Android. The + # statement is about the product on, or with, what they list. + if joined: + conditions += [_platform([e for e in other.get("cpeMatch") or [] if isinstance(e, dict)]) + for other in nodes if other is not node] + if node_joined: + conditions += [_platform([other]) for other in entries if other is not entry] start = entry.get("versionStartIncluding") or entry.get("versionStartExcluding") end = entry.get("versionEndIncluding") or entry.get("versionEndExcluding") if start is None and end is None: version = fields[5] - found.append(Range(exact=version) if version not in _NOT_A_VERSION - else Range(unbounded=True)) + found.append(Range(exact=version, conditions=tuple(conditions)) + if version not in _NOT_A_VERSION + else Range(unbounded=True, conditions=tuple(conditions))) continue found.append(Range( start=start, start_inclusive="versionStartIncluding" in entry, end=end, end_inclusive="versionEndIncluding" in entry, + conditions=tuple(conditions), )) - return found + return found, excluded + + +def ranges_in(cve: dict, *, vendor: str, cpe_product: str | None = None, + watched: Cpe | None = None) -> list[Range]: + """The affected ranges this CVE declares for that vendor, and product. + + `cpe_product` is the product field of the CPE in its escaped form — the + `notepad\\+\\+` that the API wants, not the `Notepad++` a person reads. + Passing the readable name raises nothing and matches nothing, which reads as + "no vulnerabilities". Vendor and product are compared without regard to + case, as CPE 2.3 names are matched. + + `watched` is the watchlist entry's CPE: an entry whose update or edition it + contradicts is left out, and one whose update or edition it does not name + carries that as a condition (`Range.conditions`), as does one in an AND + configuration. + """ + return _ranges(cve, vendor, cpe_product, watched)[0] def _note_for(span: Range) -> str: @@ -326,7 +443,8 @@ def affects(payload: dict, *, version: str, vendor: str, if not isinstance(cve, dict): continue for span in ranges_in(cve, vendor=vendor, cpe_product=cpe_product): - if not span.contains(version): + # A range with conditions says "affected, if": not a hit + if span.conditions or not span.contains(version): continue score, cvss_version, severity = _score_of(cve) description = next((d.get("value", "") for d in cve.get("descriptions") or [] @@ -423,13 +541,20 @@ class Verdict: fixed_version: str | None = None -def judge(record: dict, *, version: str, vendor: str, cpe_product: str) -> Verdict: +def judge(record: dict, *, version: str, vendor: str, cpe_product: str, + watched: Cpe | None = None) -> Verdict: """The verdict on one scan record for the installed `version` (DESIGN §5.3). The record carries NVD's `configurations` when NVD returned it — an empty list when NVD has not analysed the CVE yet — and no such key when it came from another source alone. Only a statement NVD made about this product decides: anything short of that is "not evaluable", never "not affected". + + Nor "affected": a range that contains the version but holds only on a + platform, or for an update or edition, the watched CPE (`watched`) does not + name makes the CVE "not evaluable", with the condition in the reason. The + watchlist knows the version, not the operating system: Chrome on Windows + was "affected" by a CVE NVD lists for Chrome on Android only. """ configurations = record.get("configurations") if configurations is None: @@ -437,13 +562,23 @@ def judge(record: dict, *, version: str, vendor: str, cpe_product: str) -> Verdi if not isinstance(configurations, list) or not configurations: return Verdict(NOT_EVALUABLE, "NVD has not yet stated which products it affects") - spans = ranges_in({"configurations": configurations}, vendor=vendor, cpe_product=cpe_product) + spans, excluded = _ranges({"configurations": configurations}, vendor, cpe_product, watched) if not spans: + if excluded: + return Verdict(NOT_AFFECTED, f"NVD lists {vendor}:{cpe_product} only with an update or " + f"edition other than the one the watched CPE names") return Verdict(NOT_AFFECTED, f"NVD lists the products it affects, and " f"{vendor}:{cpe_product} is not among them") + conditional = None for span in spans: if span.contains(version): - return Verdict(AFFECTED, _note_for(span), span.fixed_version) + if not span.conditions: + return Verdict(AFFECTED, _note_for(span), span.fixed_version) + conditional = conditional or span + if conditional is not None: + return Verdict(NOT_EVALUABLE, f"NVD lists it as affecting {version} only " + f"{' and '.join(conditional.conditions)}, which the " + f"watchlist does not record") stated = "; ".join(span.describe() for span in spans) if not all(span.comparable(version) for span in spans): @@ -453,10 +588,12 @@ def judge(record: dict, *, version: str, vendor: str, cpe_product: str) -> Verdi f"{version} is not among them") -def mark(records: list[dict], installed: tuple[str, str, str] | None) -> None: +def mark(records: list[dict], installed: tuple[str, str, str] | None, + watched: Cpe | None = None) -> None: """Give each scan record its verdict for `installed` — (version, vendor, product), or None for an entry matched by name — and take NVD's - configurations off it, which nothing after the verdict reads. + configurations off it, which nothing after the verdict reads. `watched` is + the entry's CPE, whose update and edition fields the verdict reads too. The one place the CLI and the API judge, so the two cannot disagree. The record keeps the version it was judged for, which is what lets a stored @@ -465,7 +602,7 @@ def mark(records: list[dict], installed: tuple[str, str, str] | None) -> None: for record in records: if installed is not None: version, vendor, product = installed - verdict = judge(record, version=version, vendor=vendor, cpe_product=product) + verdict = judge(record, version=version, vendor=vendor, cpe_product=product, watched=watched) record.update(verdict=verdict.status, verdict_reason=verdict.reason, verdict_fixed_version=verdict.fixed_version, judged_version=version) record.pop("configurations", None) diff --git a/src/patchradar/api/main.py b/src/patchradar/api/main.py index 2c2bfd4..121a878 100644 --- a/src/patchradar/api/main.py +++ b/src/patchradar/api/main.py @@ -338,7 +338,8 @@ async def _scan_one(sw: str, days: int, errors: list[dict], entry: WatchEntry | lambda: nvd_fetch(sw, days_back=days), lambda: msrc_fetch(sw, days_back=days), lambda: debian_fetch(sw), - lambda: kev_fetch(sw), + # By the entry's CPE as well as by name, as in the CLI (collectors/kev.py) + lambda: kev_fetch(sw, cpe=entry.cpe) if entry is not None and entry.cpe else kev_fetch(sw), ] if vuldb_api_key(): fetches.append(lambda: vuldb_fetch(sw, days_back=days)) @@ -378,7 +379,8 @@ async def _scan_one(sw: str, days: int, errors: list[dict], entry: WatchEntry | "reason": exc.reason, "status": exc.status}) # Judged by the function the CLI judges with, before anything is stored, so # the verdict lands with the link and the two scans give one answer. - mark(collected, installed) + watched = parse_cpe(entry.cpe) if entry is not None and entry.cpe and installed is not None else None + mark(collected, installed, watched) if installed is not None and verdicts is not None: found = [cve["verdict"] for cve in collected] verdicts[sw] = {status: found.count(status) for status in (AFFECTED, NOT_EVALUABLE, NOT_AFFECTED)} diff --git a/src/patchradar/cli.py b/src/patchradar/cli.py index 3a87ab8..dc7d27a 100644 --- a/src/patchradar/cli.py +++ b/src/patchradar/cli.py @@ -1,5 +1,6 @@ import asyncio import contextlib +import functools import json import math import pathlib @@ -248,13 +249,17 @@ def _print_law_check(law, out=None) -> None: out.print("[bold]⚖️ Provisions applied[/bold]") for status in law.acts: act = status.act - name, source = escape(act.name), escape(act.source) + name = escape(act.name) + # Which source was tried, and whether it was unreachable or answered + # without the text cited: "EUR-Lex unreachable" whatever happened was + # wrong on both counts when Cellar had answered without the paragraph. + reason = escape(status.reason or "") if status.source == "verified": origin = escape(status.origin or act.source) out.print(f"[dim]{name}: verified against {origin} ({act.id_label} {escape(act.celex)})[/dim]") elif status.source == "cache": out.print( - f"[yellow]{name}: {source} unreachable, " + f"[yellow]{name}: {reason}, " "text from the cached copy, not re-verified[/yellow]" ) else: @@ -263,7 +268,7 @@ def _print_law_check(law, out=None) -> None: # for a bug in the hashing. There is no verified text to hash, and # printing one anyway would assert a verification never made. out.print( - f"[yellow]{name}: {source} unreachable and nothing cached, " + f"[yellow]{name}: {reason} and nothing cached, " "text not verifiable: the citations below have no SHA-256[/yellow]" ) if act.note: @@ -371,7 +376,8 @@ async def _scan_target( """Scan a single target for CVEs and return count. The CVEs found are also appended to `collected`, when given, for the law - check at the end of the scan, and the failures to `failed`: one source + check at the end of the scan — all of them but the ones judged not to affect + the installed version — and the failures to `failed`: one source refusing every target in a watchlist is a different event from one hiccup, and only the whole scan can tell them apart. """ @@ -386,7 +392,10 @@ async def _scan_target( # VulDB only with its key: without it every target came back "incomplete" # and no scan could ever end in an all-clear. A source nobody configured # is not one that failed; `scan` says once that it was not asked. - fetches = (fetch_cves, msrc_fetch, kev_fetch) + ((vuldb_fetch,) if vuldb_api_key() else ()) + # KEV by the entry's CPE as well as by name: CISA files most Chrome + # CVEs under "Chromium V8" and the like (collectors/kev.py). + kev_by_cpe = functools.partial(kev_fetch, cpe=entry.cpe) if entry is not None and entry.cpe else kev_fetch + fetches = (fetch_cves, msrc_fetch, kev_by_cpe) + ((vuldb_fetch,) if vuldb_api_key() else ()) for fetch in fetches: try: all_cves += await fetch(target, days_back=days) @@ -428,11 +437,17 @@ async def _scan_target( # configurations, and the ones that do not affect it are counted, not # listed, unless --all asks for them. judged = installed is not None - mark(all_cves, installed) + # The CPE too: its update and edition fields decide which of NVD's ranges apply + watched = parse_cpe(entry.cpe) if entry is not None and entry.cpe and installed is not None else None + mark(all_cves, installed, watched) for cve in all_cves: await save_cve(cve) if collected is not None: - collected.extend(all_cves) + # A CVE NVD states does not affect the installed version is no gap in + # the measures GDPR art. 25 and 32 or D.Lgs. 138/2024 art. 24 ask for: + # citing it put 7-Zip 26.06 under art. 25 for CVE-2025-0411, fixed in + # 24.09, in the same output that called it "not affecting it". + collected.extend(cve for cve in all_cves if cve.get("verdict") != NOT_AFFECTED) shown = all_cves if judged and not show_all: shown = [cve for cve in all_cves if cve["verdict"] != NOT_AFFECTED] diff --git a/src/patchradar/collectors/kev.py b/src/patchradar/collectors/kev.py index 849a055..e8fb63a 100644 --- a/src/patchradar/collectors/kev.py +++ b/src/patchradar/collectors/kev.py @@ -4,6 +4,7 @@ import httpx +from patchradar.collectors import nvd from patchradar.collectors.errors import BAD_PAYLOAD, CollectorError, from_http_error from patchradar.names import keyword_pattern @@ -161,15 +162,48 @@ def filter_catalogue(data: dict, keyword: str) -> list[dict]: return results -async def fetch_cves(keyword: str, days_back: int = 30) -> list[dict]: +async def fetch_cves(keyword: str, days_back: int = 30, *, cpe: str | None = None) -> list[dict]: """Fetch known-exploited CVEs for a given vendor or product keyword. `days_back` is accepted for signature compatibility with the other collectors and deliberately ignored: the catalogue is a standing list of what is being exploited, not a feed of recent publications, and dropping older entries would hide the ones that have been exploited the longest. + + With the watchlist entry's `cpe`, the catalogue is also read through NVD: + the entries whose CVE NVD flags as in KEV for that product + (`nvd.known_exploited_for`) are taken whatever CISA called the product, and + NVD's records of them come back too, so that the scan can judge them + without asking NVD again one id at a time. The keyword alone found 6 of the + 72 exploited Chrome CVEs: CISA files the rest under "Chromium V8", + "Chromium Mojo", "Chromium" and so on. A list of aliases would have fixed + Chrome and left every other product CISA names by a component (Win32k, + Exchange Server, ...) as it was; the CPE is the name NVD has already mapped. """ - return filter_catalogue(await get_catalogue(), keyword) + data = await get_catalogue() + results = filter_catalogue(data, keyword) + if cpe is None: + return results + try: + flagged = await nvd.known_exploited_for(cpe, keyword) + except CollectorError as exc: + # The keyword's matches stand; the ones only NVD could name are missing, + # and the error says so rather than the scan reading as complete. + raise CollectorError(exc.source, exc.reason, status=exc.status, detail=exc.detail, + partial=results) from exc + found = {str(record["id"]).upper() for record in results} + wanted = {str(record["id"]).upper() for record in flagged} + for entry in data.get("vulnerabilities", []): + if not isinstance(entry, dict): + continue + cve_id = entry.get("cveID") + if isinstance(cve_id, str) and cve_id.strip().upper() in wanted - found: + found.add(cve_id.strip().upper()) + results.append(_to_record(entry, keyword)) + # NVD's record first, so that the merged row keeps NVD's publication date + # and description, and CISA's adds the exploitation facts to it. + listed = [record for record in flagged if str(record["id"]).upper() in found] + return listed + results async def known_exploited(cve_id: str) -> dict | None: diff --git a/src/patchradar/collectors/nvd.py b/src/patchradar/collectors/nvd.py index f2b95b1..dc2e94b 100644 --- a/src/patchradar/collectors/nvd.py +++ b/src/patchradar/collectors/nvd.py @@ -8,6 +8,7 @@ import httpx from patchradar.collectors.errors import BAD_PAYLOAD, CollectorError, from_http_error +from patchradar.cpe import parse_cpe from patchradar.cvss import exploit_maturity from patchradar.priority import merge_by_cve @@ -297,10 +298,18 @@ async def _fetch_window( "keywordSearch": keyword, "pubStartDate": start.strftime("%Y-%m-%dT00:00:00.000"), "pubEndDate": end.strftime("%Y-%m-%dT23:59:59.999"), - # A string, because httpx's params type does not accept an int: it - # stringifies one anyway, so this changes the annotation, not the request. - "resultsPerPage": str(RESULTS_PER_PAGE), } + await _fetch_pages(client, params, keyword, pace, results) + + +async def _fetch_pages( + client: httpx.AsyncClient, params: dict[str, str], keyword: str, + pace: Callable[[], Awaitable[None]], results: list[dict], +) -> None: + """Every page of the answer to `params`, appended to `results`.""" + # A string, because httpx's params type does not accept an int: it + # stringifies one anyway, so this changes the annotation, not the request. + params = {**params, "resultsPerPage": str(RESULTS_PER_PAGE)} start_index = 0 while True: @@ -402,6 +411,32 @@ async def fetch_cves( +async def known_exploited_for( + cpe: str, keyword: str, *, + wait: Callable[[float], Awaitable[None]] | None = None, +) -> list[dict]: + """NVD's records of the CVEs it flags as in CISA's KEV catalogue (`hasKev`) + whose configurations name this product (`virtualMatchString`, the CPE's + `cpe:2.3:::`), whatever their date. + + How the scan finds the exploited CVEs of a product that has a CPE: CISA + names products its own way, and on 2026-11-17 the word "chrome" was in 6 of + the 72 Chrome entries of the catalogue — the others say "Chromium V8", + "Chromium Mojo", "Chromium". NVD maps each of them to the CPE it affects; + asked for google:chrome it returned 78, the 72 and six more whose CISA + entry names no browser at all (Adobe Flash Player, WebRTC, Skia, Dawn). + """ + parsed = parse_cpe(cpe) + key = api_key() + headers = {"apiKey": key} if key else {} + pace = _pacer(wait or _pace, KEYED_DELAY_SECONDS if key else KEYLESS_DELAY_SECONDS) + params = {"hasKev": "", "virtualMatchString": f"cpe:2.3:{parsed.part}:{parsed.vendor}:{parsed.product}"} + results: list[dict] = [] + async with httpx.AsyncClient(timeout=30.0, headers=headers) as client: + await _fetch_pages(client, params, keyword, pace, results) + return results + + async def records_for( cve_ids: list[str], keyword: str, *, wait: Callable[[float], Awaitable[None]] | None = None, diff --git a/src/patchradar/cpe.py b/src/patchradar/cpe.py index e94c0b5..1962cc2 100644 --- a/src/patchradar/cpe.py +++ b/src/patchradar/cpe.py @@ -17,7 +17,7 @@ _PARTS = {"a", "o", "h"} # ANY and NA. A vendor or product of either names no product to judge. -_WILDCARDS = {"*", "-", ""} +WILDCARDS = frozenset({"*", "-", ""}) class CpeError(ValueError): @@ -30,9 +30,17 @@ class Cpe: part: str vendor: str product: str + # Every field, as split from `text`: [0] "cpe", [1] "2.3", [2] the part, ... + fields: tuple[str, ...] = () + def field(self, index: int) -> str: + """The field at `index` of the CPE 2.3 formatted string — 6 update, + 9 sw_edition, 10 target_sw, ... — and `*` (ANY) where the name stops + before it, as CPE 2.3 reads a name written short.""" + return self.fields[index] if index < len(self.fields) and self.fields[index] else "*" -def _fields(text: str) -> list[str]: + +def split_fields(text: str) -> list[str]: """Split on the colons that are not escaped.""" fields, current, escaped = [], [], False for char in text: @@ -73,17 +81,24 @@ def _escaped(value: str) -> str: def parse_cpe(text: str) -> Cpe: - cleaned = text.strip() + """The CPE in `text`, in lower case. + + CPE 2.3 names match without regard to case, and NVD writes them in lower + case. Kept as typed, `cpe:2.3:a:Mozilla:Thunderbird` matched none of NVD's + names, and every CVE of the program was judged "not affected" — hidden from + the table, an all-clear made of a capital letter. + """ + cleaned = text.strip().lower() if not cleaned.startswith(PREFIX): raise CpeError(f"'{cleaned}' is not a CPE 2.3 name: it must start with '{PREFIX}'") - fields = _fields(cleaned) + fields = split_fields(cleaned) if len(fields) < 5: raise CpeError(f"'{cleaned}' names no product: cpe:2.3:::") part, vendor, product = fields[2], fields[3], fields[4] if part not in _PARTS: raise CpeError(f"'{cleaned}': the part is '{part}', and CPE 2.3 knows only a, o and h") for label, value in (("vendor", vendor), ("product", product)): - if value in _WILDCARDS: + if value in WILDCARDS: shown = f"'{value}'" if value else "empty" raise CpeError(f"'{cleaned}': the {label} is {shown}, which names no product") if _escaped(value) != value: @@ -91,4 +106,4 @@ def parse_cpe(text: str) -> Cpe: f"'{cleaned}': the {label} '{value}' has characters NVD writes escaped, " f"and as typed it would match nothing; NVD's form is '{_escaped(value)}'" ) - return Cpe(text=cleaned, part=part, vendor=vendor, product=product) + return Cpe(text=cleaned, part=part, vendor=vendor, product=product, fields=tuple(fields)) diff --git a/src/patchradar/law_cache.py b/src/patchradar/law_cache.py index 2a22832..54ebd3b 100644 --- a/src/patchradar/law_cache.py +++ b/src/patchradar/law_cache.py @@ -54,6 +54,29 @@ def default_cache_path() -> Path: return base.resolve() / "law_cache.json" +def merge( + cached: dict[Key, Provision], fresh: dict[Key, Provision] +) -> tuple[dict[Key, Provision], dict[Key, str]]: + """ + What the cache holds once `fresh` is merged into `cached`, saved or not. + + Returns: + (provisions, changed): an entry whose text did not change keeps its + fetched_at; for each one whose text changed, the SHA-256 of the + previous text. + """ + provisions = dict(cached) + changed = {} + for key, provision in fresh.items(): + old = provisions.get(key) + if old is not None and old.sha256 == provision.sha256: + continue + if old is not None: + changed[key] = old.sha256 + provisions[key] = provision + return provisions, changed + + class LawCache: def __init__(self, path: str | Path | None = None): @@ -67,7 +90,8 @@ def load(self) -> dict[Key, Provision]: """ try: data = json.loads(self.path.read_text(encoding="utf-8")) - except (OSError, ValueError): + # RecursionError: a deeply nested file, which used to end the law check + except (OSError, ValueError, RecursionError): return {} entries = data.get("entries") if isinstance(data, dict) else None if not isinstance(entries, list): @@ -94,15 +118,7 @@ def update( for each provision whose text differs from the cached one, the SHA-256 of the previous text. """ - provisions = self.load() - changed = {} - for key, provision in fresh.items(): - old = provisions.get(key) - if old is not None and old.sha256 == provision.sha256: - continue - if old is not None: - changed[key] = old.sha256 - provisions[key] = provision + provisions, changed = merge(self.load(), fresh) self._save(provisions, checked_at) return provisions, changed diff --git a/src/patchradar/law_checker.py b/src/patchradar/law_checker.py index e7c168a..72dea7a 100644 --- a/src/patchradar/law_checker.py +++ b/src/patchradar/law_checker.py @@ -10,13 +10,14 @@ """ from __future__ import annotations +import contextlib from dataclasses import dataclass, field from datetime import UTC, datetime import httpx from patchradar import law_fetcher -from patchradar.law_cache import Key, LawCache +from patchradar.law_cache import Key, LawCache, merge from patchradar.law_fetcher import DLGS_138_2024, GDPR, Act, LawFetchError, Provision # ─── Mapping: PatchRadar findings → provisions ──────────────────────────────── @@ -106,11 +107,27 @@ class Citation: @dataclass(frozen=True) class ActStatus: act: Act - # "verified": downloaded now; "cache": source unreachable, cached copy; "unavailable": no text at all + # "verified": every reference cited was in the text downloaded now; "cache": the source was + # unreachable or its text lacked a reference, cached copy; "unavailable": no text at all source: str error: str | None = None # Where a verified text came from: "Cellar", "EUR-Lex" (the fallback) or "Normattiva"; None otherwise origin: str | None = None + # When not verified: the source tried last ("Cellar", "EUR-Lex", "Normattiva", "Cellar and + # EUR-Lex"; None if unknown), and why it was not used: "unreachable" (no page) or "not_in_text" + # (a page, without the text cited) + tried: str | None = None + failure: str | None = None + + @property + def reason(self) -> str | None: + """Why the act was not verified, as a report says it: "Cellar and EUR-Lex unreachable".""" + if self.source == "verified": + return None + tried = self.tried or self.act.source + if self.failure == "not_in_text": + return f"{tried} answered without the text cited" + return f"{tried} unreachable" @dataclass @@ -156,36 +173,52 @@ def check( acts = list(dict.fromkeys(act for _, act, _ in cited)) # Keyed by (celex, article) — the cache's key, not the fetcher's. fresh: dict[Key, Provision] = {} - errors: dict[Act, str] = {} + errors: dict[Act, LawFetchError] = {} origins: dict[Act, str] = {} for act in acts: + refs = list(dict.fromkeys(ref for _, a, ref in cited if a == act)) # The articles cited ("32(1)(a)" is part of article 32) and ALSO_FETCH articles = tuple(dict.fromkeys( - [ref.split("(")[0] for _, a, ref in cited if a == act] + list(ALSO_FETCH.get(act, ())) + [ref.split("(")[0] for ref in refs] + list(ALSO_FETCH.get(act, ())) )) try: - by_article, origins[act] = law_fetcher.fetch_provisions_from(act, articles, now=now, client=client) + by_article, origin = law_fetcher.fetch_provisions_from(act, articles, now=now, client=client) + # The article being there is not enough: each reference cited is + # checked against the text just parsed. A paragraph the parser + # missed used to fall back to the cache, old hash and date included, + # under the "verified" label, or to no hash at all. + missing = [ref for ref in refs if ref not in by_article] + if missing: + raise LawFetchError( + f"{act.name} art. {', '.join(missing)} not found in the text from {origin}", + source=origin, reached=True, + ) except LawFetchError as e: - errors[act] = str(e) + errors[act] = e else: + origins[act] = origin fresh.update({p.key: p for p in by_article.values()}) - changed: dict[Key, str] = {} - try: - if fresh: - provisions, changed = cache.update(fresh, checked_at=law_fetcher.utc_stamp(now)) - else: - provisions = cache.load() - except OSError: - provisions = {**cache.load(), **fresh} # the text just downloaded can still be cited + # Merged here, not by the save: a cache that cannot be written (read-only, + # disk full) must not hide a change in the law, nor date an unchanged + # wording to today. + provisions, changed = merge(cache.load(), fresh) + if fresh: + # Unsaved, the text just downloaded can still be cited + with contextlib.suppress(OSError): + cache.update(fresh, checked_at=law_fetcher.utc_stamp(now)) statuses = [] for act in acts: if act not in errors: statuses.append(ActStatus(act, "verified", origin=origins[act])) else: + error = errors[act] cached = any((act.celex, ref) in provisions for _, a, ref in cited if a == act) - statuses.append(ActStatus(act, "cache" if cached else "unavailable", errors[act])) + statuses.append(ActStatus( + act, "cache" if cached else "unavailable", str(error), + tried=error.source, failure="not_in_text" if error.reached else "unreachable", + )) citations = [] for finding, act, ref in cited: diff --git a/src/patchradar/law_fetcher.py b/src/patchradar/law_fetcher.py index 5ab52fb..33be238 100644 --- a/src/patchradar/law_fetcher.py +++ b/src/patchradar/law_fetcher.py @@ -26,6 +26,7 @@ import hashlib import re +import time import unicodedata from collections.abc import Iterator from dataclasses import asdict, dataclass @@ -92,7 +93,17 @@ def id_label(self) -> str: class LawFetchError(Exception): - """The source could not be reached, or its page did not contain the articles.""" + """The source could not be reached, or its page did not contain the articles. + + `source` is the source tried ("Cellar", "EUR-Lex", "Normattiva", or "Cellar + and EUR-Lex" when the fallback failed too), and `reached` says it answered + with a page that lacked the text: a report must not call that "unreachable". + """ + + def __init__(self, message: str, *, source: str | None = None, reached: bool = False): + super().__init__(message) + self.source = source + self.reached = reached def text_sha256(text: str) -> str: @@ -239,7 +250,9 @@ def _text(node: _Element | str) -> str: # ─── Tree → provisions ──────────────────────────────────────────────────────── _PARAGRAPH_ID = re.compile(r"^\d{3}\.\d{3}$") #
is art. 5(1) -_PARAGRAPH = re.compile(r"^(\d+)\. ") # "1. I dati personali sono:" +# "1. I dati personali sono:", and the paragraphs Italian amendments insert: "3-bis. ", +# "4-ter. ", ... every Latin adverb after quater ends in -ies (quinquies, decies, terdecies) +_PARAGRAPH = re.compile(r"^(\d+(?:-(?:bis|ter|quater|[a-z]+ies))?)\. ") _NUMBERED = re.compile(r"^(\d+)\) ") # GDPR art. 4: "1) «dato personale»: ..." _LETTER = re.compile(r"^([a-z]{1,4})\)$") # table label cell: "a)" _POINT = re.compile(r"^([a-z]{1,4})\) ") # Normattiva point: "a) ingannevoli ..." @@ -319,7 +332,7 @@ def _parse_block(nodes: list, prefix: str) -> tuple[list[str], dict[str, str]]: return lines, points -def _parse_paragraphs(groups: list[tuple[int, list]], number: str) -> dict[str, str]: +def _parse_paragraphs(groups: list[tuple[str, list]], number: str) -> dict[str, str]: texts: dict[str, str] = {} lines = [] for paragraph, nodes in groups: @@ -340,7 +353,7 @@ def _parse_eli_article(article: _Element, number: str) -> dict[str, str]: ] if paragraphs: return _parse_paragraphs( - [(int(div.attrs["id"].split(".")[1]), div.children) for div in paragraphs], number + [(str(int(div.attrs["id"].split(".")[1])), div.children) for div in paragraphs], number ) lines, points = _parse_block(article.children, number) return {**points, number: "\n".join(lines)} @@ -364,14 +377,14 @@ def _parse_normattiva_article(root: _Element, number: str) -> dict[str, str]: for e in _elements(body) ): continue - groups: list[tuple[int | None, str, dict[str, str]]] = [] + groups: list[tuple[str | None, str, dict[str, str]]] = [] for comma in (e for e in _elements(body) if "art-comma-div-akn" in e.classes): text = _without_update_marks(_text(comma)) if not text: continue num = next((e for e in _elements(comma) if "comma-num-akn" in e.classes), None) - match = _PARAGRAPH.match(_text(num) + " ") if num is not None else None - paragraph = int(match.group(1)) if match else None + match = _PARAGRAPH.match(_without_update_marks(_text(num)) + " ") if num is not None else None + paragraph = match.group(1) if match else None # "3", "3-bis" points = {} for item in (e for e in _elements(comma) if "pointedList-rest-akn" in e.classes): point = _without_update_marks(_text(item)) @@ -421,12 +434,12 @@ def _parse_consolidated_article(root: _Element, number: str) -> dict[str, str]: def _split_flat_paragraphs(body: list, number: str) -> dict[str, str]: - groups: list[tuple[int, list]] = [] + groups: list[tuple[str, list]] = [] intro: list = [] for node in _flatten(body): match = _PARAGRAPH.match(_text(node)) if isinstance(node, _Element) and node.tag == "p" else None if match: - groups.append((int(match.group(1)), [node])) + groups.append((match.group(1), [node])) elif groups: groups[-1][1].append(node) else: @@ -439,10 +452,11 @@ def _split_flat_paragraphs(body: list, number: str) -> dict[str, str]: return _parse_paragraphs(groups, number) -def parse_articles(html: str, articles: tuple[str, ...]) -> dict[str, str]: +def parse_articles(html: str, articles: tuple[str, ...], source: str | None = None) -> dict[str, str]: """ Extract articles from an EUR-Lex page (Official Journal or consolidated - layout) or a Normattiva article page. + layout) or a Normattiva article page; `source` is who served it, for the + error message. Returns: Text by reference, for each article and each of its paragraphs and @@ -467,7 +481,8 @@ def parse_articles(html: str, articles: tuple[str, ...]) -> dict[str, str]: or _parse_consolidated_article(builder.root, number) ) if not parsed.get(number): - raise LawFetchError(f"Article {number} not found in the EUR-Lex page") + page = f"the {source} page" if source else "the page" + raise LawFetchError(f"Article {number} not found in {page}", source=source, reached=True) texts.update(parsed) return texts @@ -483,10 +498,33 @@ def _source_of(url: str) -> str: MAX_REDIRECTS = 5 +# The one host each source may redirect to: a text served from anywhere else +# would be labelled "verified against Cellar" without being Cellar's. +SOURCE_HOSTS = { + "Cellar": "publications.europa.eu", + "EUR-Lex": "eur-lex.europa.eu", + "Normattiva": "www.normattiva.it", +} + + +# A page is read up to this size, and a fetch, redirects included, ends at this +# deadline. The largest act cited, the GDPR from Cellar, was 823981 bytes on +# 2026-10-11. The timeout bounds each read, not the whole: without a deadline a +# server sending a byte every few seconds held the audit for as long as it liked. +MAX_RESPONSE_BYTES = 32 * 1024 * 1024 +FETCH_DEADLINE = 120.0 + +def _past(deadline: float, source: str) -> None: + if time.monotonic() > deadline: + raise LawFetchError(f"{source} took more than {FETCH_DEADLINE:g} s", source=source) -def _get_over_https(client: httpx.Client, url: str, headers: dict[str, str], source: str) -> httpx.Response: - """GET `url`, following redirects by hand and never over plain HTTP. + +def _get_over_https( + client: httpx.Client, url: str, headers: dict[str, str], source: str, deadline: float +) -> httpx.Response: + """GET `url`, following redirects by hand, never over plain HTTP and never + off the source's host. The response is streamed: the caller reads and closes it. Cellar answers its https:// URL with a 303 to an http:// one: recorded on 2026-10-11 for the GDPR, resource/celex/32016R0679 -> 303 -> @@ -496,15 +534,54 @@ def _get_over_https(client: httpx.Client, url: str, headers: dict[str, str], sou the same document, byte for byte, so such a hop is taken over HTTPS; if that fails, the caller's fallback to EUR-Lex applies, as for any other failure. """ + expected = SOURCE_HOSTS.get(source, "").encode("ascii") target = httpx.URL(url) for _ in range(MAX_REDIRECTS + 1): + _past(deadline, source) if target.scheme == "http": target = target.copy_with(scheme="https") - response = client.get(target, headers=headers, follow_redirects=False) + response = client.send( + client.build_request("GET", target, headers=headers), stream=True, follow_redirects=False + ) if not response.is_redirect: return response + response.close() target = response.url.join(response.headers["location"]) - raise LawFetchError(f"{source} redirected more than {MAX_REDIRECTS} times") + # raw_host: decoding the host is what raises on one like "xn--" + if target.raw_host != expected or target.port not in (None, 443): + host = target.raw_host.decode("ascii", "replace") + (f":{target.port}" if target.port else "") + raise LawFetchError( + f"{source} redirected to {host}, not to {expected.decode('ascii')}: not followed", + source=source, + ) + raise LawFetchError(f"{source} redirected more than {MAX_REDIRECTS} times", source=source) + + +def _read_capped(response: httpx.Response, source: str, deadline: float) -> str: + """The body as text, refused past MAX_RESPONSE_BYTES or past the deadline.""" + too_large = f"{source} sent more than {MAX_RESPONSE_BYTES} bytes" + declared = response.headers.get("Content-Length", "") + if declared.isdigit() and int(declared) > MAX_RESPONSE_BYTES: + raise LawFetchError(too_large, source=source) + body = bytearray() + for chunk in response.iter_bytes(): + body += chunk + if len(body) > MAX_RESPONSE_BYTES: + raise LawFetchError(too_large, source=source) + _past(deadline, source) + # As Response.text decodes: the charset declared, else the client's default + return body.decode(response.encoding or "utf-8", errors="replace") + + +def _download(client: httpx.Client, url: str, headers: dict[str, str], source: str) -> str: + deadline = time.monotonic() + FETCH_DEADLINE + response = _get_over_https(client, url, headers, source, deadline) + try: + if response.status_code != 200: + raise LawFetchError(f"{source} answered HTTP {response.status_code}", source=source) + return _read_capped(response, source, deadline) + finally: + response.close() def fetch_html( @@ -521,14 +598,13 @@ def fetch_html( try: if client is None: with httpx.Client(timeout=timeout) as own_client: - response = _get_over_https(own_client, url, headers, source) - else: - response = _get_over_https(client, url, headers, source) - except httpx.HTTPError as e: - raise LawFetchError(f"{source} unreachable: {e}") from e - if response.status_code != 200: - raise LawFetchError(f"{source} answered HTTP {response.status_code}") - return response.text + return _download(own_client, url, headers, source) + return _download(client, url, headers, source) + # Not only httpx.HTTPError: a Location httpx cannot follow raises + # httpx.InvalidURL, or idna.core.IDNAError (a ValueError) for a host such + # as "xn--", and either one used to skip the EUR-Lex fallback and the cache. + except (httpx.HTTPError, httpx.InvalidURL, ValueError) as e: + raise LawFetchError(f"{source} unreachable: {e}", source=source) from e def fetch_act_html( @@ -575,7 +651,9 @@ def _fetch_act_page( ) return page, "EUR-Lex" except LawFetchError as page_failed: - raise LawFetchError(f"{cellar_failed}; and {page_failed}") from page_failed + raise LawFetchError( + f"{cellar_failed}; and {page_failed}", source="Cellar and EUR-Lex" + ) from page_failed def fetch_provisions( @@ -607,11 +685,11 @@ def fetch_provisions_from( texts: dict[str, str] = {} for article in articles: html = fetch_html(NORMATTIVA_URL.format(urn=act.celex, article=article), client=client) - texts.update(parse_articles(html, (article,))) + texts.update(parse_articles(html, (article,), "Normattiva")) source = "Normattiva" else: html, source = _fetch_act_page(act, lang, client=client) - texts = parse_articles(html, articles) + texts = parse_articles(html, articles, source) fetched_at = utc_stamp(now or datetime.now(UTC)) return { ref: Provision.from_text(ref, text, fetched_at, act.celex) diff --git a/tests/fixtures/kev_chrome_excerpt.json b/tests/fixtures/kev_chrome_excerpt.json new file mode 100644 index 0000000..0ce90df --- /dev/null +++ b/tests/fixtures/kev_chrome_excerpt.json @@ -0,0 +1,135 @@ +{ + "_derived": "An excerpt, not the whole answer. The source is the live CISA KEV catalogue, https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json, downloaded on 2026-11-17 (catalogVersion 2026.11.17, dateReleased 2026-11-17T17:53:15.5664Z, 1782 entries; 72 of them name Chrome or Chromium, only 6 with the word 'chrome'). Kept, in the catalogue's order: CVE-2026-125350 (Google / Chromium); CVE-2026-58704 (Google / Pixel); CVE-2026-87491 (Google / Chromium V8); CVE-2026-5281 (Google / Dawn); CVE-2025-2783 (Google / Chromium Mojo); CVE-2025-0411 (7-Zip / 7-Zip); CVE-2016-7200 (Microsoft / Edge); CVE-2020-16017 (Google / Chrome). Every kept entry is copied verbatim; nothing was added or changed, only left out. The header fields are verbatim too, except `count`, which says how many entries this file holds.", + "title": "CISA Catalog of Known Exploited Vulnerabilities", + "catalogVersion": "2026.11.17", + "dateReleased": "2026-11-17T17:53:15.5664Z", + "count": 8, + "vulnerabilities": [ + { + "cveID": "CVE-2026-125350", + "vendorProject": "Google", + "product": "Chromium", + "vulnerabilityName": "Google Chromium Use-After-Free Vulnerability", + "dateAdded": "2026-11-10", + "shortDescription": "Google Chromium contains a use-after-free vulnerability in the XML component that could allow a remote attacker to execute arbitrary code inside the sandbox via a specially crafted webpage. This vulnerability could affect multiple web browsers that utilize Chromium, including, but not limited to, Google Chrome, Microsoft Edge, and Opera.", + "requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA’s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA’s “Forensics Triage Requirements” (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.", + "dueDate": "2026-11-24", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://chromereleases.googleblog.com/2026/11/stable-channel-update-for-desktop_018223192.html ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-125350", + "cwes": [ + "CWE-416" + ] + }, + { + "cveID": "CVE-2026-58704", + "vendorProject": "Google", + "product": "Pixel", + "vulnerabilityName": "Google Pixel Improper Authorization Vulnerability", + "dateAdded": "2026-09-16", + "shortDescription": "Google Pixel devices contain an improper authorization vulnerability in the cellular modem. A logic error may allow an attacker to bypass permission checks and escalate privileges.", + "requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA’s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA’s “Forensics Triage Requirements” (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.", + "dueDate": "2026-09-19", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "Yes", + "notes": "https://source.android.com/docs/security/bulletin/pixel/2026/2026-09-01 ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-58704", + "cwes": [ + "CWE-693" + ] + }, + { + "cveID": "CVE-2026-87491", + "vendorProject": "Google", + "product": "Chromium V8", + "vulnerabilityName": "Google Chromium V8 Out of Bounds Write Vulnerability", + "dateAdded": "2026-09-09", + "shortDescription": "Google Chromium V8 contains an out of bounds write vulnerability that allows a remote attacker to execute arbitrary code inside the sandbox via a crafted HTML page. This vulnerability could affect multiple web browsers that utilize Chromium, including, but not limited to, Google Chrome, Microsoft Edge, and Opera.", + "requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA’s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA’s “Forensics Triage Requirements” (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.", + "dueDate": "2026-09-23", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://chromereleases.googleblog.com/2026/09/stable-channel-update-for-desktop_0808145027.html ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-87491", + "cwes": [ + "CWE-787" + ] + }, + { + "cveID": "CVE-2026-5281", + "vendorProject": "Google", + "product": "Dawn", + "vulnerabilityName": "Google Dawn Use-After-Free Vulnerability", + "dateAdded": "2026-04-01", + "shortDescription": "Google Dawn contains an use-after-free vulnerability that could allow a remote attacker who had compromised the renderer process to execute arbitrary code via a crafted HTML page. This vulnerability could affect multiple Chromium-based products including, but not limited to, Google Chrome, Microsoft Edge, and Opera.", + "requiredAction": "Apply mitigations per vendor instructions, follow applicable BOD 22-01 guidance for cloud services, or discontinue use of the product if mitigations are unavailable.", + "dueDate": "2026-04-15", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "This vulnerability affects an open-source component, third-party library, protocol, or proprietary implementation that could be used by different products. For more information, please see: https://chromereleases.googleblog.com/2026/03/stable-channel-update-for-desktop_31.html ; https://nvd.nist.gov/vuln/detail/CVE-2026-5281", + "cwes": [ + "CWE-416" + ] + }, + { + "cveID": "CVE-2025-2783", + "vendorProject": "Google", + "product": "Chromium Mojo", + "vulnerabilityName": "Google Chromium Mojo Sandbox Escape Vulnerability", + "dateAdded": "2025-03-27", + "shortDescription": "Google Chromium Mojo on Windows contains a sandbox escape vulnerability caused by a logic error, which results from an incorrect handle being provided in unspecified circumstances. This vulnerability could affect multiple web browsers that utilize Chromium, including, but not limited to, Google Chrome, Microsoft Edge, and Opera.", + "requiredAction": "Apply mitigations per vendor instructions, follow applicable BOD 22-01 guidance for cloud services, or discontinue use of the product if mitigations are unavailable.", + "dueDate": "2025-04-17", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://chromereleases.googleblog.com/2025/03/stable-channel-update-for-desktop_25.html ; https://nvd.nist.gov/vuln/detail/CVE-2025-2783", + "cwes": [] + }, + { + "cveID": "CVE-2025-0411", + "vendorProject": "7-Zip", + "product": "7-Zip", + "vulnerabilityName": "7-Zip Mark of the Web Bypass Vulnerability", + "dateAdded": "2025-02-06", + "shortDescription": "7-Zip contains a protection mechanism failure vulnerability that allows remote attackers to bypass the Mark-of-the-Web security feature to execute arbitrary code in the context of the current user.", + "requiredAction": "Apply mitigations per vendor instructions or discontinue use of the product if mitigations are unavailable.", + "dueDate": "2025-02-27", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://www.7-zip.org/history.txt ; https://nvd.nist.gov/vuln/detail/CVE-2025-0411", + "cwes": [ + "CWE-693" + ] + }, + { + "cveID": "CVE-2016-7200", + "vendorProject": "Microsoft", + "product": "Edge", + "vulnerabilityName": "Microsoft Edge Memory Corruption Vulnerability", + "dateAdded": "2022-03-28", + "shortDescription": "The Chakra JavaScript scripting engine in Microsoft Edge allows remote attackers to execute remote code or cause a denial of service (memory corruption) via a crafted web site.", + "requiredAction": "Apply updates per vendor instructions.", + "dueDate": "2022-04-18", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://nvd.nist.gov/vuln/detail/CVE-2016-7200", + "cwes": [ + "CWE-119" + ] + }, + { + "cveID": "CVE-2020-16017", + "vendorProject": "Google", + "product": "Chrome", + "vulnerabilityName": "Google Chrome Use-After-Free Vulnerability", + "dateAdded": "2021-11-03", + "shortDescription": "Google Chrome contains a use-after-free vulnerability that allows a remote attacker, who has compromised the renderer process, to potentially perform a sandbox escape via a crafted HTML page.", + "requiredAction": "Apply updates per vendor instructions.", + "dueDate": "2022-05-03", + "knownRansomwareCampaignUse": "Unknown", + "forensicTriage": "No", + "notes": "https://nvd.nist.gov/vuln/detail/CVE-2020-16017", + "cwes": [ + "CWE-416" + ] + } + ] +} diff --git a/tests/fixtures/nvd_chrome_has_kev_excerpt.json b/tests/fixtures/nvd_chrome_has_kev_excerpt.json new file mode 100644 index 0000000..7c7d2ce --- /dev/null +++ b/tests/fixtures/nvd_chrome_has_kev_excerpt.json @@ -0,0 +1,823 @@ +{ + "_derived": "An excerpt, not the whole answer. The source is NVD's answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?hasKev=&virtualMatchString=cpe%3A2.3%3Aa%3Agoogle%3Achrome&resultsPerPage=2000&startIndex=0, recorded on 2026-11-17 (timestamp 2026-11-17T19:43:18.077, totalResults 78: every CVE NVD flags as in CISA KEV whose configurations name google:chrome). Kept, in NVD's order, the records of CVE-2020-16017, CVE-2025-2783, CVE-2026-5281, CVE-2026-87491, CVE-2026-125350, each copied verbatim. `resultsPerPage` and `totalResults` say how many records this file holds; every other header field is verbatim.", + "startIndex": 0, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T19:43:18.077", + "resultsPerPage": 5, + "totalResults": 5, + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2020-16017", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2021-01-08T19:15:12.727", + "lastModified": "2026-06-17T02:57:38.610", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Use after free in site isolation in Google Chrome prior to 86.0.4240.198 allowed a remote attacker who had compromised the renderer process to potentially perform a sandbox escape via a crafted HTML page." + }, + { + "lang": "es", + "value": "Un uso de la memoria previamente liberada en site isolation en Google Chrome versiones anteriores a 86.0.4240.198, permitió a un atacante remoto que había comprometido el proceso del renderizador llevar a cabo potencialmente un escape del sandbox por medio de una página HTML diseñada" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "unspecified", + "lessThan": "86.0.4240.198", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 9.6, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 6.0 + }, + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 9.6, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 6.0 + } + ], + "cvssMetricV2": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "cvssData": { + "version": "2.0", + "vectorString": "AV:N/AC:M/Au:N/C:P/I:P/A:P", + "baseScore": 6.8, + "accessVector": "NETWORK", + "accessComplexity": "MEDIUM", + "authentication": "NONE", + "confidentialityImpact": "PARTIAL", + "integrityImpact": "PARTIAL", + "availabilityImpact": "PARTIAL" + }, + "baseSeverity": "MEDIUM", + "exploitabilityScore": 8.6, + "impactScore": 6.4, + "acInsufInfo": false, + "obtainAllPrivilege": false, + "obtainUserPrivilege": false, + "obtainOtherPrivilege": false, + "userInteractionRequired": true + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2025-02-03T14:19:29.342779Z", + "id": "CVE-2020-16017", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2021-11-03", + "cisaActionDue": "2022-05-03", + "cisaRequiredAction": "Apply updates per vendor instructions.", + "cisaVulnerabilityName": "Google Chrome Use-After-Free Vulnerability", + "weaknesses": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + }, + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "86.0.4240.198", + "matchCriteriaId": "C62E5348-105F-4B98-A7B6-ACCA567941F3" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2020/11/stable-channel-update-for-desktop_11.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://crbug.com/1146709", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required", + "Vendor Advisory" + ] + }, + { + "url": "https://chromereleases.googleblog.com/2020/11/stable-channel-update-for-desktop_11.html", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://crbug.com/1146709", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Permissions Required", + "Vendor Advisory" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2020-16017", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + }, + { + "cve": { + "id": "CVE-2025-2783", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2025-03-26T16:15:23.693", + "lastModified": "2026-06-17T09:07:37.023", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Incorrect handle provided in unspecified circumstances in Mojo in Google Chrome on Windows prior to 134.0.6998.177 allowed a remote attacker to perform a sandbox escape via a malicious file. (Chromium security severity: High)" + }, + { + "lang": "es", + "value": "Un identificador incorrecto proporcionado en circunstancias no especificadas en Mojo en Google Chrome para Windows anterior a la versión 134.0.6998.177 permitió que un atacante remoto escapara de la zona protegida mediante un archivo malicioso. (Severidad de seguridad de Chromium: Alta)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "134.0.6998.177", + "lessThan": "134.0.6998.177", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 8.3, + "baseSeverity": "HIGH", + "attackVector": "NETWORK", + "attackComplexity": "HIGH", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 1.6, + "impactScore": 6.0 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2025-03-28T03:55:52.966374Z", + "id": "CVE-2025-2783", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2025-03-27", + "cisaActionDue": "2025-04-17", + "cisaRequiredAction": "Apply mitigations per vendor instructions, follow applicable BOD 22-01 guidance for cloud services, or discontinue use of the product if mitigations are unavailable.", + "cisaVulnerabilityName": "Google Chromium Mojo Sandbox Escape Vulnerability", + "weaknesses": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "description": [ + { + "lang": "en", + "value": "NVD-CWE-noinfo" + } + ] + } + ], + "configurations": [ + { + "operator": "AND", + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "134.0.6998.177", + "matchCriteriaId": "55EB6B03-2E87-4B74-A41D-1A48BAFDC687" + } + ] + }, + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": false, + "criteria": "cpe:2.3:o:microsoft:windows:-:*:*:*:*:*:*:*", + "matchCriteriaId": "A2572D17-1DE6-457B-99CC-64AFD54487EA" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2025/03/stable-channel-update-for-desktop_25.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes" + ] + }, + { + "url": "https://issues.chromium.org/issues/405143032", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2025-2783", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + }, + { + "cve": { + "id": "CVE-2026-5281", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2026-04-01T05:16:01.440", + "lastModified": "2026-07-24T21:10:00.143", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Use after free in Dawn in Google Chrome prior to 146.0.7680.178 allowed a remote attacker who had compromised the renderer process to execute arbitrary code via a crafted HTML page. (Chromium security severity: High)" + }, + { + "lang": "es", + "value": "Uso después de liberar en Dawn en Google Chrome versiones anteriores a 146.0.7680.178 permitía a un atacante remoto que había comprometido el proceso de renderizado ejecutar código arbitrario a través de una página HTML manipulada. (Gravedad de seguridad de Chromium: Alta)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "146.0.7680.178", + "lessThan": "146.0.7680.178", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H", + "baseScore": 8.8, + "baseSeverity": "HIGH", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 5.9 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-04-01T00:00:00+00:00", + "id": "CVE-2026-5281", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2026-04-01", + "cisaActionDue": "2026-04-15", + "cisaRequiredAction": "Apply mitigations per vendor instructions, follow applicable BOD 22-01 guidance for cloud services, or discontinue use of the product if mitigations are unavailable.", + "cisaVulnerabilityName": "Google Dawn Use-After-Free Vulnerability", + "weaknesses": [ + { + "source": "chrome-cve-admin@google.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + } + ], + "configurations": [ + { + "operator": "AND", + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "146.0.7680.177", + "matchCriteriaId": "55B0324F-7911-403B-8235-D71996B55DFB" + } + ] + }, + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": false, + "criteria": "cpe:2.3:o:apple:macos:-:*:*:*:*:*:*:*", + "matchCriteriaId": "387021A0-AF36-463C-A605-32EA7DAC172E" + }, + { + "vulnerable": false, + "criteria": "cpe:2.3:o:linux:linux_kernel:-:*:*:*:*:*:*:*", + "matchCriteriaId": "703AF700-7A70-47E2-BC3A-7FD03B3CA9C1" + }, + { + "vulnerable": false, + "criteria": "cpe:2.3:o:microsoft:windows:-:*:*:*:*:*:*:*", + "matchCriteriaId": "A2572D17-1DE6-457B-99CC-64AFD54487EA" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2026/03/stable-channel-update-for-desktop_31.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Vendor Advisory" + ] + }, + { + "url": "https://issues.chromium.org/issues/491518608", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Issue Tracking", + "Permissions Required" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2026-5281", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + }, + { + "cve": { + "id": "CVE-2026-87491", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2026-09-09T01:17:05.887", + "lastModified": "2026-09-21T13:17:11.283", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Out of bounds write in V8 in Google Chrome prior to 153.0.8010.36 allowed a remote attacker to execute arbitrary code inside the sandbox via a crafted HTML page. (Chromium security severity: Medium)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "153.0.8010.36", + "lessThan": "153.0.8010.36", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H", + "baseScore": 8.8, + "baseSeverity": "HIGH", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 5.9 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-09-10T03:55:21.039385Z", + "id": "CVE-2026-87491", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2026-09-09", + "cisaActionDue": "2026-09-23", + "cisaRequiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA’s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA’s “Forensics Triage Requirements” (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.", + "cisaVulnerabilityName": "Google Chromium V8 Out of Bounds Write Vulnerability", + "weaknesses": [ + { + "source": "chrome-cve-admin@google.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-787" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "153.0.8010.36", + "matchCriteriaId": "7BC1B89D-DEB1-48BD-8602-0869B89600FD" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2026/09/stable-channel-update-for-desktop_0808145027.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://issues.chromium.org/issues/543557673", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2026-87491", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + }, + { + "cve": { + "id": "CVE-2026-125350", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2026-11-10T18:17:35.237", + "lastModified": "2026-11-11T05:19:38.220", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Use after free in XML in Google Chrome prior to 157.0.8103.50 allowed a remote attacker to execute arbitrary code inside the sandbox via a crafted HTML page. (Chromium security severity: High)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "157.0.8103.50", + "lessThan": "157.0.8103.50", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H", + "baseScore": 8.8, + "baseSeverity": "HIGH", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 5.9 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-11-09T00:00:00+00:00", + "id": "CVE-2026-125350", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2026-11-10", + "cisaActionDue": "2026-11-24", + "cisaRequiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA’s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA’s “Forensics Triage Requirements” (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.", + "cisaVulnerabilityName": "Google Chromium Use-After-Free Vulnerability", + "weaknesses": [ + { + "source": "chrome-cve-admin@google.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "157.0.8103.50", + "matchCriteriaId": "6670B19D-D9FF-4276-B04E-F35E8F3BB029" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2026/11/stable-channel-update-for-desktop_018223192.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://issues.chromium.org/issues/582442907", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2026-125350", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + } + ] +} diff --git a/tests/fixtures/nvd_cve_2024_9680_recorded.json b/tests/fixtures/nvd_cve_2024_9680_recorded.json new file mode 100644 index 0000000..72e35bf --- /dev/null +++ b/tests/fixtures/nvd_cve_2024_9680_recorded.json @@ -0,0 +1,476 @@ +{ + "_recorded": "NVD API 2.0 answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2024-9680, recorded on 2026-11-17 (timestamp below), verbatim; only this key was added.", + "resultsPerPage": 1, + "startIndex": 0, + "totalResults": 1, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T19:24:56.386", + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2024-9680", + "sourceIdentifier": "security@mozilla.org", + "published": "2024-10-09T13:15:12.090", + "lastModified": "2026-08-04T05:16:32.530", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "An attacker was able to achieve code execution in the content process by exploiting a use-after-free in Animation timelines. We have had reports of this vulnerability being exploited in the wild. This vulnerability affects Firefox < 131.0.2, Firefox ESR < 128.3.1, Firefox ESR < 115.16.1, Thunderbird < 131.0.1, Thunderbird < 128.3.1, and Thunderbird < 115.16.0." + }, + { + "lang": "es", + "value": "Un atacante pudo ejecutar código en el proceso de contenido aprovechando un use-after-free en las líneas de tiempo de animación. Hemos recibido informes de que esta vulnerabilidad se ha explotado de forma activa. Esta vulnerabilidad afecta a Firefox < 131.0.2, Firefox ESR < 128.3.1 y Firefox ESR < 115.16.1." + } + ], + "affected": [ + { + "source": "security@mozilla.org", + "affectedData": [ + { + "vendor": "Mozilla", + "product": "Firefox", + "versions": [ + { + "version": "unspecified", + "lessThan": "131.0.2", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "Mozilla", + "product": "Firefox ESR", + "versions": [ + { + "version": "unspecified", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "Mozilla", + "product": "Firefox ESR", + "versions": [ + { + "version": "unspecified", + "lessThan": "115.16.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "Mozilla", + "product": "Thunderbird", + "versions": [ + { + "version": "unspecified", + "lessThan": "131.0.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "Mozilla", + "product": "Thunderbird", + "versions": [ + { + "version": "unspecified", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "Mozilla", + "product": "Thunderbird", + "versions": [ + { + "version": "unspecified", + "lessThan": "115.16.0", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + }, + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "affectedData": [ + { + "vendor": "mozilla", + "product": "firefox", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:firefox:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "131.0.2", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "mozilla", + "product": "firefox_esr", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:firefox_esr:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "115.16.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "mozilla", + "product": "firefox_esr", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:firefox_esr:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "115.16.1", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "mozilla", + "product": "thunderbird", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:thunderbird:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "131.0.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "115.16.0", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "mozilla", + "product": "thunderbird", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:thunderbird:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "131.0.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "115.16.0", + "versionType": "custom", + "status": "affected" + } + ] + }, + { + "vendor": "mozilla", + "product": "thunderbird", + "defaultStatus": "unknown", + "cpes": [ + "cpe:2.3:a:mozilla:thunderbird:*:*:*:*:*:*:*:*" + ], + "versions": [ + { + "version": "0", + "lessThan": "131.0.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "128.3.1", + "versionType": "custom", + "status": "affected" + }, + { + "version": "0", + "lessThan": "115.16.0", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", + "baseScore": 9.8, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "NONE", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 3.9, + "impactScore": 5.9 + }, + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", + "baseScore": 9.8, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "NONE", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 3.9, + "impactScore": 5.9 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2024-10-09T00:00:00+00:00", + "id": "CVE-2024-9680", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "yes" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2024-10-15", + "cisaActionDue": "2024-11-05", + "cisaRequiredAction": "Apply mitigations per vendor instructions or discontinue use of the product if mitigations are unavailable.", + "cisaVulnerabilityName": "Mozilla Firefox Use-After-Free Vulnerability", + "weaknesses": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + }, + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*", + "versionEndExcluding": "115.16.1", + "matchCriteriaId": "45244B45-832B-4C4A-8004-8D13C331E52B" + }, + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:firefox:*:*:*:*:-:*:*:*", + "versionEndExcluding": "131.0.2", + "matchCriteriaId": "07E39FE8-1685-46D2-9E3E-2613F3852132" + }, + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*", + "versionStartIncluding": "128.1.0", + "versionEndExcluding": "128.3.1", + "matchCriteriaId": "C4AEB9BD-EB47-4F96-BC72-949023ACE8ED" + }, + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:thunderbird:*:*:*:*:*:*:*:*", + "versionEndExcluding": "115.16.0", + "matchCriteriaId": "DB9A49A0-8233-46B2-894A-FAD4DC6ED563" + }, + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:thunderbird:*:*:*:*:*:*:*:*", + "versionStartIncluding": "128.0.1", + "versionEndExcluding": "128.3.1", + "matchCriteriaId": "F48D7DE7-477D-4026-B6BD-BFE2BC5382F3" + }, + { + "vulnerable": true, + "criteria": "cpe:2.3:a:mozilla:thunderbird:131.0:*:*:*:*:*:*:*", + "matchCriteriaId": "FD5E6249-26BF-4E13-87D8-B15EF63A859F" + } + ] + } + ] + }, + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:o:debian:debian_linux:11.0:*:*:*:*:*:*:*", + "matchCriteriaId": "FA6FEEC2-9F11-4643-8827-749718254FED" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://bugzilla.mozilla.org/show_bug.cgi?id=1923344", + "source": "security@mozilla.org", + "tags": [ + "Issue Tracking", + "Permissions Required" + ] + }, + { + "url": "https://msrc.microsoft.com/update-guide/en-US/vulnerability/CVE-2024-49039", + "source": "security@mozilla.org", + "tags": [ + "Not Applicable", + "Patch", + "Vendor Advisory" + ] + }, + { + "url": "https://www.mozilla.org/security/advisories/mfsa2024-51/", + "source": "security@mozilla.org", + "tags": [ + "Vendor Advisory" + ] + }, + { + "url": "https://www.mozilla.org/security/advisories/mfsa2024-52/", + "source": "security@mozilla.org", + "tags": [ + "Vendor Advisory" + ] + }, + { + "url": "https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=281992", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Issue Tracking" + ] + }, + { + "url": "https://lists.debian.org/debian-lts-announce/2024/10/msg00005.html", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Mailing List" + ] + }, + { + "url": "https://lists.debian.org/debian-lts-announce/2024/10/msg00006.html", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Mailing List" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2024-9680", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + } + ] +} diff --git a/tests/fixtures/nvd_cve_2025_0411_recorded.json b/tests/fixtures/nvd_cve_2025_0411_recorded.json new file mode 100644 index 0000000..24559ae --- /dev/null +++ b/tests/fixtures/nvd_cve_2025_0411_recorded.json @@ -0,0 +1,221 @@ +{ + "_recorded": "NVD API 2.0 answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2025-0411, recorded on 2026-11-17 (timestamp below), verbatim; only this key was added.", + "resultsPerPage": 1, + "startIndex": 0, + "totalResults": 1, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T18:36:38.337", + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2025-0411", + "sourceIdentifier": "zdi-disclosures@trendmicro.com", + "published": "2025-01-25T05:15:09.533", + "lastModified": "2026-06-17T08:26:25.627", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "7-Zip Mark-of-the-Web Bypass Vulnerability. This vulnerability allows remote attackers to bypass the Mark-of-the-Web protection mechanism on affected installations of 7-Zip. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.\n\nThe specific flaw exists within the handling of archived files. When extracting files from a crafted archive that bears the Mark-of-the-Web, 7-Zip does not propagate the Mark-of-the-Web to the extracted files. An attacker can leverage this vulnerability to execute arbitrary code in the context of the current user. Was ZDI-CAN-25456." + }, + { + "lang": "es", + "value": "Vulnerabilidad de omisión de 7-Zip Mark-of-the-Web. Esta vulnerabilidad permite a atacantes remotos omitir el mecanismo de protección de la marca de la Web en las instalaciones afectadas de 7-Zip. Se requiere la interacción del usuario para explotar esta vulnerabilidad, ya que el objetivo debe visitar una página maliciosa o abrir un archivo malicioso. La falla específica existe en el manejo de archivos comprimidos. Al extraer archivos de un archivo comprimido manipulado que lleva la marca de la Web, 7-Zip no propaga la marca de la Web a los archivos extraídos. Un atacante puede aprovechar esta vulnerabilidad para ejecutar código arbitrario en el contexto del usuario actual. Era ZDI-CAN-25456." + } + ], + "affected": [ + { + "source": "zdi-disclosures@trendmicro.com", + "affectedData": [ + { + "vendor": "7-Zip", + "product": "7-Zip", + "defaultStatus": "unknown", + "versions": [ + { + "version": "24.08 (x64)", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "nvd@nist.gov", + "type": "Primary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:L/AC:H/PR:N/UI:R/S:U/C:H/I:H/A:H", + "baseScore": 7.0, + "baseSeverity": "HIGH", + "attackVector": "LOCAL", + "attackComplexity": "HIGH", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 1.0, + "impactScore": 5.9 + } + ], + "cvssMetricV30": [ + { + "source": "zdi-disclosures@trendmicro.com", + "type": "Secondary", + "cvssData": { + "version": "3.0", + "vectorString": "CVSS:3.0/AV:L/AC:H/PR:N/UI:R/S:U/C:H/I:H/A:H", + "baseScore": 7.0, + "baseSeverity": "HIGH", + "attackVector": "LOCAL", + "attackComplexity": "HIGH", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "UNCHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 1.0, + "impactScore": 5.9 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2025-02-07T04:55:28.684912Z", + "id": "CVE-2025-0411", + "options": [ + { + "exploitation": "active" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "cisaExploitAdd": "2025-02-06", + "cisaActionDue": "2025-02-27", + "cisaRequiredAction": "Apply mitigations per vendor instructions or discontinue use of the product if mitigations are unavailable.", + "cisaVulnerabilityName": "7-Zip Mark of the Web Bypass Vulnerability", + "weaknesses": [ + { + "source": "zdi-disclosures@trendmicro.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-693" + } + ] + }, + { + "source": "nvd@nist.gov", + "type": "Primary", + "description": [ + { + "lang": "en", + "value": "NVD-CWE-noinfo" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:netapp:active_iq_unified_manager:-:*:*:*:*:windows:*:*", + "matchCriteriaId": "B55E8D50-99B4-47EC-86F9-699B67D473CE" + } + ] + } + ] + }, + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:7-zip:7-zip:*:*:*:*:*:*:*:*", + "versionEndExcluding": "24.09", + "matchCriteriaId": "AAFF445C-96F1-4328-A34E-A8C392B34BF3" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://www.zerodayinitiative.com/advisories/ZDI-25-045/", + "source": "zdi-disclosures@trendmicro.com", + "tags": [ + "Third Party Advisory", + "VDB Entry" + ] + }, + { + "url": "http://www.openwall.com/lists/oss-security/2025/01/24/6", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Mailing List" + ] + }, + { + "url": "https://security.netapp.com/advisory/ntap-20250207-0005/", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Third Party Advisory" + ] + }, + { + "url": "https://www.vicarius.io/vsociety/posts/cve-2025-0411-7-zip-mitigation-vulnerability", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Mitigation" + ] + }, + { + "url": "https://www.vicarius.io/vsociety/posts/cve-2025-0411-detection-7-zip-vulnerability", + "source": "af854a3a-2127-422b-91ae-364da2661108", + "tags": [ + "Mitigation" + ] + }, + { + "url": "https://www.cisa.gov/known-exploited-vulnerabilities-catalog?field_cve=CVE-2025-0411", + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "tags": [ + "US Government Resource" + ] + } + ] + } + } + ] +} diff --git a/tests/fixtures/nvd_cve_2026_16424_recorded.json b/tests/fixtures/nvd_cve_2026_16424_recorded.json new file mode 100644 index 0000000..48e6ef8 --- /dev/null +++ b/tests/fixtures/nvd_cve_2026_16424_recorded.json @@ -0,0 +1,155 @@ +{ + "_recorded": "NVD API 2.0 answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-16424, recorded on 2026-11-17 (timestamp below), verbatim; only this key was added.", + "resultsPerPage": 1, + "startIndex": 0, + "totalResults": 1, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T19:11:00.412", + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2026-16424", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2026-07-21T23:17:00.030", + "lastModified": "2026-10-24T17:10:00.220", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Use after free in GPU in Google Chrome on Android prior to 150.0.7871.182 allowed a remote attacker who had compromised the renderer process to potentially perform a sandbox escape via a crafted HTML page. (Chromium security severity: High)" + }, + { + "lang": "es", + "value": "Uso después de liberación en la GPU en Google Chrome para Android anterior a 150.0.7871.182 permitió a un atacante remoto que había comprometido el proceso de renderizado realizar potencialmente un escape de sandbox a través de una página HTML manipulada. (Gravedad de seguridad de Chromium: Alta)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "150.0.7871.182", + "lessThan": "150.0.7871.182", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 9.6, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 6.0 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-07-22T13:38:09.855415Z", + "id": "CVE-2026-16424", + "options": [ + { + "exploitation": "none" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "weaknesses": [ + { + "source": "chrome-cve-admin@google.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-416" + } + ] + } + ], + "configurations": [ + { + "operator": "AND", + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "150.0.7871.182", + "matchCriteriaId": "9A87274E-BC24-4123-9D89-B3D7C8C34F95" + } + ] + }, + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": false, + "criteria": "cpe:2.3:o:google:android:-:*:*:*:*:*:*:*", + "matchCriteriaId": "F8B9FEC8-73B6-43B8-B24E-1F7C20D91D26" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2026/07/stable-channel-update-for-desktop_0256605430.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://issues.chromium.org/issues/534858939", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required" + ] + } + ] + } + } + ] +} diff --git a/tests/fixtures/nvd_cve_2026_17709_recorded.json b/tests/fixtures/nvd_cve_2026_17709_recorded.json new file mode 100644 index 0000000..4cb299f --- /dev/null +++ b/tests/fixtures/nvd_cve_2026_17709_recorded.json @@ -0,0 +1,155 @@ +{ + "_recorded": "NVD API 2.0 answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-17709, recorded on 2026-11-17 (timestamp below), verbatim; only this key was added.", + "resultsPerPage": 1, + "startIndex": 0, + "totalResults": 1, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T19:11:08.031", + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2026-17709", + "sourceIdentifier": "chrome-cve-admin@google.com", + "published": "2026-07-30T01:16:33.390", + "lastModified": "2026-10-24T13:10:00.147", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Race in Downloads in Google Chrome on Mac prior to 151.0.7922.72 allowed a remote attacker who had compromised the renderer process to potentially perform a sandbox escape via a crafted HTML page. (Chromium security severity: High)" + }, + { + "lang": "es", + "value": "Una condición de carrera en Descargas en Google Chrome para Mac anterior a la versión 151.0.7922.72 permitía a un atacante remoto que había comprometido el proceso de renderizado realizar potencialmente un escape de sandbox a través de una página HTML manipulada. (Severidad de seguridad de Chromium: Alta)" + } + ], + "affected": [ + { + "source": "chrome-cve-admin@google.com", + "affectedData": [ + { + "vendor": "Google", + "product": "Chrome", + "versions": [ + { + "version": "151.0.7922.72", + "lessThan": "151.0.7922.72", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 9.6, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 6.0 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-07-30T15:39:57.613087Z", + "id": "CVE-2026-17709", + "options": [ + { + "exploitation": "none" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "weaknesses": [ + { + "source": "chrome-cve-admin@google.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-362" + } + ] + } + ], + "configurations": [ + { + "operator": "AND", + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:google:chrome:*:*:*:*:*:*:*:*", + "versionEndExcluding": "151.0.7922.72", + "matchCriteriaId": "5306E563-AFD7-43CB-AFC8-CE1F449BA94C" + } + ] + }, + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": false, + "criteria": "cpe:2.3:o:apple:macos:-:*:*:*:*:*:*:*", + "matchCriteriaId": "387021A0-AF36-463C-A605-32EA7DAC172E" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://chromereleases.googleblog.com/2026/07/stable-channel-update-for-desktop_0887107924.html", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Release Notes", + "Vendor Advisory" + ] + }, + { + "url": "https://issues.chromium.org/issues/519981494", + "source": "chrome-cve-admin@google.com", + "tags": [ + "Permissions Required" + ] + } + ] + } + } + ] +} diff --git a/tests/fixtures/nvd_cve_2026_81376_recorded.json b/tests/fixtures/nvd_cve_2026_81376_recorded.json new file mode 100644 index 0000000..a4a97e5 --- /dev/null +++ b/tests/fixtures/nvd_cve_2026_81376_recorded.json @@ -0,0 +1,136 @@ +{ + "_recorded": "NVD API 2.0 answer to GET https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-81376, recorded on 2026-11-17 (timestamp below), verbatim; only this key was added.", + "resultsPerPage": 1, + "startIndex": 0, + "totalResults": 1, + "format": "NVD_CVE", + "version": "2.0", + "timestamp": "2026-11-17T19:25:20.907", + "vulnerabilities": [ + { + "cve": { + "id": "CVE-2026-81376", + "sourceIdentifier": "secure@microsoft.com", + "published": "2026-09-08T18:20:53.813", + "lastModified": "2026-09-11T21:09:45.063", + "vulnStatus": "Analyzed", + "cveTags": [], + "descriptions": [ + { + "lang": "en", + "value": "Incomplete comparison with missing factors in Visual Studio Code allows an unauthorized attacker to bypass a security feature over a network." + } + ], + "affected": [ + { + "source": "secure@microsoft.com", + "affectedData": [ + { + "vendor": "Microsoft", + "product": "Visual Studio Code", + "versions": [ + { + "version": "1.0.0", + "lessThan": "1.136.2", + "versionType": "custom", + "status": "affected" + } + ] + } + ] + } + ], + "metrics": { + "cvssMetricV31": [ + { + "source": "secure@microsoft.com", + "type": "Secondary", + "cvssData": { + "version": "3.1", + "vectorString": "CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H", + "baseScore": 9.6, + "baseSeverity": "CRITICAL", + "attackVector": "NETWORK", + "attackComplexity": "LOW", + "privilegesRequired": "NONE", + "userInteraction": "REQUIRED", + "scope": "CHANGED", + "confidentialityImpact": "HIGH", + "integrityImpact": "HIGH", + "availabilityImpact": "HIGH" + }, + "exploitabilityScore": 2.8, + "impactScore": 6.0 + } + ], + "ssvcV203": [ + { + "source": "134c704f-9b21-4f2e-91b3-4a467353bcc0", + "ssvcData": { + "timestamp": "2026-09-09T04:26:44.058389Z", + "id": "CVE-2026-81376", + "options": [ + { + "exploitation": "none" + }, + { + "automatable": "no" + }, + { + "technicalImpact": "total" + } + ], + "role": "CISA Coordinator", + "version": "2.0.3" + } + } + ] + }, + "weaknesses": [ + { + "source": "secure@microsoft.com", + "type": "Secondary", + "description": [ + { + "lang": "en", + "value": "CWE-693" + }, + { + "lang": "en", + "value": "CWE-1023" + } + ] + } + ], + "configurations": [ + { + "nodes": [ + { + "operator": "OR", + "negate": false, + "cpeMatch": [ + { + "vulnerable": true, + "criteria": "cpe:2.3:a:microsoft:visual_studio_code:*:*:*:*:*:-:*:*", + "versionEndExcluding": "1.136.2", + "matchCriteriaId": "B2E65938-F774-4081-8D0C-35496C069561" + } + ] + } + ] + } + ], + "references": [ + { + "url": "https://msrc.microsoft.com/update-guide/vulnerability/CVE-2026-81376", + "source": "secure@microsoft.com", + "tags": [ + "Patch", + "Vendor Advisory" + ] + } + ] + } + } + ] +} diff --git a/tests/test_affected_conditions.py b/tests/test_affected_conditions.py new file mode 100644 index 0000000..7ef9b25 --- /dev/null +++ b/tests/test_affected_conditions.py @@ -0,0 +1,219 @@ +""" +PatchRadar — a verdict holds only when NVD's statement is about the version alone + +NVD states which versions a CVE affects in `cpeMatch` entries, and an entry can +say more than a version: + +- its other CPE fields: `7-zip:4.43:beta` is the 4.43 beta, not 4.43; Firefox's + `sw_edition` is `esr` for the extended-support line and `-` for the release; +- the configuration around it: an AND of Chrome and `vulnerable: false` Android + is Chrome *on Android*, and nothing about Chrome on Windows. + +The verdict read the version and nothing else, so a Windows install of Chrome was +"affected" by a CVE NVD lists for Android only, and Firefox ESR 115.17.0 was +"affected, fix in 131.0.2" by a CVE whose ESR line was fixed in 115.16.1. The +watchlist knows the version and the CPE, not the operating system or (unless the +CPE names them) the update and edition: a match that holds only under such a +condition is "not evaluable", with the condition named, and never "affected". + +The records are NVD's, recorded on 2026-11-17 (`nvd_cve_*_recorded.json`), and +the 7-Zip page recorded earlier (`nvd_7zip_page.json`). +""" +from __future__ import annotations + +import json +import pathlib + +import httpx +import pytest +import respx +from httpx import ASGITransport, AsyncClient +from typer.testing import CliRunner + +import patchradar.api.main as api +from patchradar import cli +from patchradar.affected import AFFECTED, NOT_AFFECTED, NOT_EVALUABLE, judge +from patchradar.collectors import msrc, nvd +from patchradar.collectors.nvd import _parse_item +from patchradar.cpe import parse_cpe + +FIXTURES = pathlib.Path(__file__).parent / "fixtures" + + +def record(name: str, cve_id: str | None = None) -> dict: + """The scan record NVD's answer in `name` gives for `cve_id` (or its only CVE).""" + items = json.loads((FIXTURES / name).read_text(encoding="utf-8"))["vulnerabilities"] + [item] = [i for i in items if cve_id in (None, i["cve"]["id"])] + return _parse_item(item, "test") + + +ANDROID_ONLY = record("nvd_cve_2026_16424_recorded.json") # Chrome < 150.0.7871.182 AND Android +MACOS_ONLY = record("nvd_cve_2026_17709_recorded.json") # Chrome < 151.0.7922.72 AND macOS +FIREFOX_9680 = record("nvd_cve_2024_9680_recorded.json") # ESR < 115.16.1, release < 131.0.2, ... +SEVEN_ZIP_BETAS = record("nvd_7zip_page.json", "CVE-2007-4725") # <= 4.42, and 4.43 to 4.52 beta + + +def _judge(rec: dict, version: str, vendor: str, product: str, cpe: str | None): + watched = {"watched": parse_cpe(cpe)} if cpe else {} + return judge(rec, version=version, vendor=vendor, cpe_product=product, **watched) + + +def chrome(rec: dict, version: str): + return _judge(rec, version, "google", "chrome", None) + + +def firefox(version: str, cpe: str | None = None): + return _judge(FIREFOX_9680, version, "mozilla", "firefox", cpe) + + +def seven_zip(version: str, cpe: str | None = None): + return _judge(SEVEN_ZIP_BETAS, version, "7-zip", "7-zip", cpe) + + +# ─── the platform an AND configuration requires ────────────────────────────── + +def test_the_records_say_what_the_tests_rely_on(): + for rec, platform in ((ANDROID_ONLY, "android"), (MACOS_ONLY, "macos")): + [configuration] = rec["configurations"] + assert configuration["operator"] == "AND" + [chrome_node, platform_node] = configuration["nodes"] + assert [m["vulnerable"] for m in platform_node["cpeMatch"]] == [False] + assert f":{platform}:" in platform_node["cpeMatch"][0]["criteria"] + + +def test_a_cve_listed_for_chrome_on_android_only_is_not_called_affected(): + verdict = chrome(ANDROID_ONLY, "150.0.7871.100") + + assert verdict.status == NOT_EVALUABLE, verdict + assert "only on Android" in verdict.reason + assert verdict.fixed_version is None + + +def test_a_cve_listed_for_chrome_on_macos_only_is_not_called_affected(): + verdict = chrome(MACOS_ONLY, "151.0.7922.10") + + assert verdict.status == NOT_EVALUABLE, verdict + assert "only on macOS" in verdict.reason + + +def test_a_version_past_the_fix_is_not_affected_whatever_the_platform(): + """The condition only matters for a version inside the range.""" + assert chrome(ANDROID_ONLY, "151.0.7922.10").status == NOT_AFFECTED + + +# ─── the edition ───────────────────────────────────────────────────────────── + +def test_firefox_esr_115_17_is_not_called_affected_with_the_release_fix(): + """The hunt's case: 115.17.0 is past the ESR fix (115.16.1) and inside the + release range (< 131.0.2); the verdict was "affected, fix in 131.0.2".""" + verdict = firefox("115.17.0") + + assert verdict.status == NOT_EVALUABLE, verdict + assert "edition" in verdict.reason + assert verdict.fixed_version is None + + +def test_a_cpe_that_names_the_esr_edition_is_judged_on_the_esr_ranges(): + assert firefox("115.17.0", "cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*").status == NOT_AFFECTED + verdict = firefox("115.16.0", "cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*") + assert (verdict.status, verdict.fixed_version) == (AFFECTED, "115.16.1") + + +def test_a_cpe_that_names_the_release_edition_is_judged_on_the_release_range(): + verdict = firefox("130.0", "cpe:2.3:a:mozilla:firefox:*:*:*:*:-:*:*:*") + assert (verdict.status, verdict.fixed_version) == (AFFECTED, "131.0.2") + assert firefox("131.0.2", "cpe:2.3:a:mozilla:firefox:*:*:*:*:-:*:*:*").status == NOT_AFFECTED + + +def test_a_version_outside_every_edition_s_range_is_not_affected(): + assert firefox("132.0").status == NOT_AFFECTED + + +def test_a_product_whose_entries_name_no_edition_is_judged_as_before(): + """Thunderbird in the same record: `sw_edition` is `*` throughout.""" + verdict = judge(FIREFOX_9680, version="115.15.0", vendor="mozilla", cpe_product="thunderbird") + assert (verdict.status, verdict.fixed_version) == (AFFECTED, "115.16.0") + + +# ─── the update ────────────────────────────────────────────────────────────── + +def test_the_4_43_beta_is_not_4_43(): + """`cpe:2.3:a:7-zip:7-zip:4.43:beta` was read as 4.43, so the final 4.43 was + "affected" by a CVE NVD states for the betas.""" + verdict = seven_zip("4.43") + + assert verdict.status == NOT_EVALUABLE, verdict + assert "update beta" in verdict.reason + + +def test_a_cpe_that_names_the_beta_is_affected(): + assert seven_zip("4.43", "cpe:2.3:a:7-zip:7-zip:*:beta").status == AFFECTED + + +def test_a_cpe_that_names_another_update_is_not_affected_by_the_betas(): + assert seven_zip("4.43", "cpe:2.3:a:7-zip:7-zip:*:-").status == NOT_AFFECTED + + +def test_a_range_with_no_update_still_decides_alone(): + """`versionEndIncluding: 4.42` with update `*`: no condition.""" + assert seven_zip("4.42").status == AFFECTED + + +def test_a_dash_no_other_entry_contradicts_is_no_condition(): + """`-` in a field that no entry of the product sets otherwise leaves nothing + to choose between: VS Code's only entry has `target_sw: -`.""" + rec = record("nvd_cve_2026_81376_recorded.json") + [configuration] = rec["configurations"] + [entry] = configuration["nodes"][0]["cpeMatch"] + assert entry["criteria"] == "cpe:2.3:a:microsoft:visual_studio_code:*:*:*:*:*:-:*:*" + + verdict = judge(rec, version="1.130.0", vendor="microsoft", cpe_product="visual_studio_code") + + assert (verdict.status, verdict.fixed_version) == (AFFECTED, "1.136.2") + + +# ─── through both scans ────────────────────────────────────────────────────── +# NVD answers with the recorded CVE-2024-9680, MSRC with no document, Debian +# (asked by the API only) with 503. + +FIREFOX_ESR = "cpe:2.3:a:mozilla:firefox:*:*:*:*:esr:*:*:*" + + +@pytest.fixture +def upstream_9680(): + payload = json.loads((FIXTURES / "nvd_cve_2024_9680_recorded.json").read_text(encoding="utf-8")) + with respx.mock(assert_all_called=False) as router: + router.get(url__startswith=nvd.NVD_API).mock(return_value=httpx.Response(200, json=payload)) + router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) + router.route().mock(return_value=httpx.Response(503)) + yield + + +def _cli_scan(cpe: str) -> str: + runner = CliRunner() + assert runner.invoke(cli.app, ["add", "firefox", "--version", "115.17.0", "--cpe", cpe]).exit_code == 0 + result = runner.invoke(cli.app, ["scan", "firefox", "--days", "7"], env={"COLUMNS": "250"}) + assert result.exit_code == 0, result.output + return " ".join(result.output.split()) + + +def test_the_cli_scan_names_the_edition_it_could_not_check(upstream_9680): + out = _cli_scan("cpe:2.3:a:mozilla:firefox") + + assert "affecting 115.17.0: 0; not evaluable: 1; not affecting it: 0" in out + assert "standard edition" in out + + +def test_the_cli_scan_reads_the_edition_of_the_watched_cpe(upstream_9680): + out = _cli_scan(FIREFOX_ESR) + + assert "affecting 115.17.0: 0; not evaluable: 0; not affecting it: 1" in out + + +async def test_the_api_scan_reads_the_edition_of_the_watched_cpe(upstream_9680): + async with AsyncClient(transport=ASGITransport(app=api.app), base_url="http://test") as client: + added = await client.post("/api/watchlist/firefox", params={"version": "115.17.0", "cpe": FIREFOX_ESR}) + assert added.status_code == 200, added.text + body = (await client.post("/api/scan", params={"days": 7})).json() + + assert body["verdicts"] == {"firefox": {"affected": 0, "not evaluable": 0, "not affected": 1}} diff --git a/tests/test_affected_verdict.py b/tests/test_affected_verdict.py index f01c9e4..f88b1c0 100644 --- a/tests/test_affected_verdict.py +++ b/tests/test_affected_verdict.py @@ -201,3 +201,22 @@ def test_every_version_is_comparable_with_every_version(): assert Range(unbounded=True).comparable("8.5.0") assert not Range(unbounded=True).comparable("latest") + + +# ─── case ──────────────────────────────────────────────────────────────────── +# NVD's recorded record of CVE-2025-0411: 7-Zip before 24.09, written by NVD in +# lower case as `cpe:2.3:a:7-zip:7-zip:*`. + +NVD_7ZIP_0411 = pathlib.Path(__file__).parent / "fixtures" / "nvd_cve_2025_0411_recorded.json" + + +@pytest.mark.parametrize(("vendor", "product"), [("7-Zip", "7-Zip"), ("7-ZIP", "7-zip")]) +def test_vendor_and_product_are_compared_without_regard_to_case(vendor, product): + """`ranges_in` compared the CPE fields as strings, so a vendor typed with a + capital letter matched none of NVD's and the CVE was "not affected".""" + [item] = json.loads(NVD_7ZIP_0411.read_text(encoding="utf-8"))["vulnerabilities"] + record = _parse_item(item, "7-zip") + + verdict = judge(record, version="24.08", vendor=vendor, cpe_product=product) + + assert (verdict.status, verdict.fixed_version) == (AFFECTED, "24.09") diff --git a/tests/test_api_versioned.py b/tests/test_api_versioned.py index dcbf121..fe3bd60 100644 --- a/tests/test_api_versioned.py +++ b/tests/test_api_versioned.py @@ -57,6 +57,15 @@ async def test_a_cpe_nvd_would_match_nothing_with_is_refused_with_its_form(clien assert await database.get_watchlist() == [] +async def test_a_cpe_sent_with_capitals_is_stored_as_nvd_writes_it(client): + response = await _add(client, version="8.5.0", cpe="cpe:2.3:a:Notepad-Plus-Plus:Notepad\\+\\+") + + assert response.status_code == 200, response.text + assert response.json()["cpe"] == NOTEPADPP + [entry] = await database.get_watch_entries() + assert entry.installed() == ("8.5.0", "notepad-plus-plus", "notepad\\+\\+") + + async def test_a_version_the_scan_cannot_order_is_refused(client): response = await _add(client, version="8.x-beta", cpe=NOTEPADPP) diff --git a/tests/test_cli_law.py b/tests/test_cli_law.py index 701ce1a..8f98c5c 100644 --- a/tests/test_cli_law.py +++ b/tests/test_cli_law.py @@ -1,11 +1,15 @@ """Tests for the law check that `patchradar scan` runs.""" import hashlib +import json from pathlib import Path +import httpx import pytest +import respx from typer.testing import CliRunner from patchradar import cli, law_fetcher +from patchradar.collectors import msrc, nvd FIXTURES = Path(__file__).parent / "fixtures" FIXTURE = FIXTURES / "gdpr_it_excerpt.html" @@ -138,3 +142,41 @@ def boom(*args, **kwargs): def test_cve_markup_in_evidence_is_not_interpreted(monkeypatch, eurlex): out = _scan(monkeypatch, [cve("CVE-[bold]x[/bold]", severity="CRITICAL")]) assert "CVE-[bold]x[/bold]" in out.output + + +# ─── only what may affect the installed version is cited ───────────────────── +# NVD answers with its recorded record of CVE-2025-0411 (7-Zip, fixed in 24.09: +# `versionEndExcluding: 24.09`; no reference tagged Patch, confidentiality +# impact HIGH), MSRC with no document. For 7-Zip 26.06 the same scan judges it +# "not affected", and the law check cited it under GDPR art. 25 and art. 32. + +NVD_7ZIP_0411 = json.loads((FIXTURES / "nvd_cve_2025_0411_recorded.json").read_text(encoding="utf-8")) +SEVEN_ZIP = "cpe:2.3:a:7-zip:7-zip" + + +def _judged_scan(version): + router = respx.mock(assert_all_called=False) + router.get(url__startswith=nvd.NVD_API).mock(return_value=httpx.Response(200, json=NVD_7ZIP_0411)) + router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) + with router: + assert runner.invoke(cli.app, ["add", "7-zip", "--cpe", SEVEN_ZIP, "--version", version]).exit_code == 0 + out = runner.invoke(cli.app, ["scan", "7-zip", "--days", "7"], env={"COLUMNS": "200"}) + assert out.exit_code == 0, out.output + return out.output + + +def test_a_cve_the_scan_judged_not_affecting_the_version_is_not_cited(): + out = _judged_scan("26.06") + + assert "not affecting it: 1" in " ".join(out.split()) + assert "CVE-2025-0411" not in out + assert "Provision applied" not in out + + +def test_a_cve_affecting_the_version_is_still_cited(): + out = _judged_scan("24.08") + law = out[out.index("Provisions applied"):] + + assert "CVE-2025-0411" in law + assert "Provision applied: GDPR art. 25" in law + assert "Provision applied: GDPR art. 32" in law diff --git a/tests/test_cpe.py b/tests/test_cpe.py index c7466c4..70eaa3e 100644 --- a/tests/test_cpe.py +++ b/tests/test_cpe.py @@ -79,3 +79,27 @@ def test_a_character_already_escaped_is_not_escaped_twice(): assert "notepad\\+\\+" in str(raised.value) assert "\\\\+" not in str(raised.value) + + +# ─── case ──────────────────────────────────────────────────────────────────── + +@pytest.mark.parametrize("text", [ + "cpe:2.3:a:Mozilla:Thunderbird", + "cpe:2.3:a:MOZILLA:THUNDERBIRD:*:*:*:*:*:*:*:*", +]) +def test_vendor_and_product_are_read_in_the_lower_case_nvd_writes(text): + """CPE 2.3 names match without regard to case, and NVD writes them in lower + case. `cpe:2.3:a:Mozilla:Thunderbird` was accepted as typed and then matched + no `cpeMatch`, so every CVE of Thunderbird came out "not affected" and was + hidden from the table: an all-clear made of a capital letter.""" + cpe = parse_cpe(text) + + assert (cpe.vendor, cpe.product) == ("mozilla", "thunderbird") + assert cpe.text == text.lower() + + +def test_an_escaped_character_survives_the_lower_case(): + cpe = parse_cpe("cpe:2.3:a:Notepad-Plus-Plus:Notepad\\+\\+") + + assert (cpe.vendor, cpe.product) == ("notepad-plus-plus", "notepad\\+\\+") + assert cpe.text == NOTEPADPP diff --git a/tests/test_judge_through_nvd.py b/tests/test_judge_through_nvd.py index bf09d24..29185b8 100644 --- a/tests/test_judge_through_nvd.py +++ b/tests/test_judge_through_nvd.py @@ -65,6 +65,9 @@ def nvd_router(): with respx.mock(assert_all_called=False) as router: router.get(url__startswith=nvd.NVD_API, params__contains={"keywordSearch": "7-zip"}).mock( return_value=httpx.Response(200, json=NVD_EMPTY)) + # The KEV collector asks NVD by the entry's CPE too (hasKev): nothing more there + router.get(url__startswith=nvd.NVD_API, params__contains={"hasKev": ""}).mock( + return_value=httpx.Response(200, json=NVD_EMPTY)) by_id = router.get(url__startswith=nvd.NVD_API, params__contains={"cveId": "CVE-2026-48092"}).mock( return_value=httpx.Response(200, json=CVE_48092)) router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) @@ -138,6 +141,9 @@ def test_when_nvd_cannot_be_asked_the_cve_stays_not_evaluable_and_the_scan_says_ with respx.mock(assert_all_called=False) as router: router.get(url__startswith=nvd.NVD_API, params__contains={"keywordSearch": "7-zip"}).mock( return_value=httpx.Response(200, json=NVD_EMPTY)) + # The KEV collector asks NVD by the entry's CPE too (hasKev): nothing more there + router.get(url__startswith=nvd.NVD_API, params__contains={"hasKev": ""}).mock( + return_value=httpx.Response(200, json=NVD_EMPTY)) router.get(url__startswith=nvd.NVD_API, params__contains={"cveId": "CVE-2026-48092"}).mock( return_value=httpx.Response(503)) router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) @@ -168,6 +174,9 @@ def test_a_cve_nvd_already_returned_is_not_asked_for_again(kev_lists_it): with respx.mock(assert_all_called=False) as router: router.get(url__startswith=nvd.NVD_API, params__contains={"keywordSearch": "7-zip"}).mock( return_value=httpx.Response(200, json=CVE_48092)) + # The KEV collector asks NVD by the entry's CPE too (hasKev): nothing more there + router.get(url__startswith=nvd.NVD_API, params__contains={"hasKev": ""}).mock( + return_value=httpx.Response(200, json=NVD_EMPTY)) by_id = router.get(url__startswith=nvd.NVD_API, params__contains={"cveId": "CVE-2026-48092"}).mock( return_value=httpx.Response(200, json=CVE_48092)) router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) @@ -187,6 +196,9 @@ async def test_the_api_lists_nvd_refusing_the_lookup_among_its_errors(kev_lists_ with respx.mock(assert_all_called=False) as router: router.get(url__startswith=nvd.NVD_API, params__contains={"keywordSearch": "7-zip"}).mock( return_value=httpx.Response(200, json=NVD_EMPTY)) + # The KEV collector asks NVD by the entry's CPE too (hasKev): nothing more there + router.get(url__startswith=nvd.NVD_API, params__contains={"hasKev": ""}).mock( + return_value=httpx.Response(200, json=NVD_EMPTY)) router.get(url__startswith=nvd.NVD_API, params__contains={"cveId": "CVE-2026-48092"}).mock( return_value=httpx.Response(503)) router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) diff --git a/tests/test_kev_by_cpe.py b/tests/test_kev_by_cpe.py new file mode 100644 index 0000000..e176b5b --- /dev/null +++ b/tests/test_kev_by_cpe.py @@ -0,0 +1,132 @@ +""" +PatchRadar — the exploited CVEs of a product with a CPE are found by its CPE + +CISA names products its own way. On 2026-11-17 the catalogue held 72 exploited +Chrome CVEs, and the word "chrome" was in 6 of them: the rest are filed as +"Google / Chromium V8", "Chromium Mojo", "Chromium" and so on. A scan of +`chrome` therefore reported 6 known-exploited CVEs where CISA lists 72. + +For an entry with a CPE the catalogue is now also read through NVD, which flags +each CVE in KEV (`hasKev`) and maps it to the products it affects +(`virtualMatchString`): asked for `cpe:2.3:a:google:chrome` it answered 78 — the +72, and six whose CISA entry names no browser at all (Adobe Flash Player, +WebRTC, Google Skia, Google Dawn). The keyword still applies, for entries +without a CPE and for CVEs NVD has not analysed yet. + +Fixtures, both excerpts of real answers, with what was left out said in their +`_derived` key: `kev_chrome_excerpt.json` (eight entries of the 2026.11.17 +catalogue: five Chrome CVEs under five CISA names, a Pixel, an Edge and a 7-Zip +one) and `nvd_chrome_has_kev_excerpt.json` (NVD's records of the five Chrome +CVEs, from its hasKev answer for google:chrome). +""" +from __future__ import annotations + +import json +import pathlib + +import httpx +import pytest +import respx +from typer.testing import CliRunner + +from patchradar import cli +from patchradar.collectors import kev, msrc, nvd +from patchradar.collectors.errors import CollectorError + +FIXTURES = pathlib.Path(__file__).parent / "fixtures" +CATALOGUE = json.loads((FIXTURES / "kev_chrome_excerpt.json").read_text(encoding="utf-8")) +HAS_KEV = json.loads((FIXTURES / "nvd_chrome_has_kev_excerpt.json").read_text(encoding="utf-8")) +NO_RESULTS = json.loads((FIXTURES / "nvd_no_results.json").read_text(encoding="utf-8")) +CHROME = "cpe:2.3:a:google:chrome" + +CHROME_CVES = {"CVE-2020-16017", "CVE-2026-125350", "CVE-2026-87491", "CVE-2025-2783", "CVE-2026-5281"} +NAMED_CHROME = {"CVE-2020-16017"} # the one entry whose CISA product has the word + + +def _nvd(request: httpx.Request) -> httpx.Response: + """NVD's hasKev answer for google:chrome; nothing for any other question.""" + params = request.url.params + if "hasKev" in params and params.get("virtualMatchString") == CHROME: + return httpx.Response(200, json=HAS_KEV) + return httpx.Response(200, json=NO_RESULTS) + + +@pytest.fixture +def sources(): + kev.clear_cache() + with respx.mock(assert_all_called=False) as router: + router.get(kev.KEV_URL).mock(return_value=httpx.Response(200, json=CATALOGUE)) + router.get(url__startswith=nvd.NVD_API, name="nvd").mock(side_effect=_nvd) + router.get(url__startswith=msrc.MSRC_API).mock(return_value=httpx.Response(404)) + yield router + kev.clear_cache() + + +def test_the_excerpts_hold_what_the_tests_rely_on(): + products = {e["cveID"]: e["product"] for e in CATALOGUE["vulnerabilities"]} + assert {cve for cve in CHROME_CVES if "chrome" in products[cve].lower()} == NAMED_CHROME + assert {v["cve"]["id"] for v in HAS_KEV["vulnerabilities"]} == CHROME_CVES + + +async def test_the_keyword_alone_finds_only_the_entry_that_says_chrome(sources): + """What a scan without a CPE still does, and the reason for the rest.""" + assert {r["id"] for r in await kev.fetch_cves("chrome")} == NAMED_CHROME + + +async def test_with_the_cpe_every_exploited_chrome_cve_is_found(sources): + records = await kev.fetch_cves("chrome", cpe=CHROME) + + exploited = {r["id"] for r in records if r.get("known_exploited")} + assert exploited == CHROME_CVES + # Pixel, Edge and 7-Zip are in the catalogue and not Chrome's + assert {r["id"] for r in records} == CHROME_CVES + + +async def test_nvd_is_asked_once_by_cpe_and_its_records_come_back(sources): + """NVD's records travel with CISA's, configurations included, so the scan + need not ask NVD about each of them again by id.""" + records = await kev.fetch_cves("chrome", cpe=CHROME) + + [call] = [c for c in sources.routes["nvd"].calls if "hasKev" in c.request.url.params] + assert call.request.url.params["virtualMatchString"] == CHROME + from_nvd = [r for r in records if r.get("source") == "NVD"] + assert {r["id"] for r in from_nvd} == CHROME_CVES + assert all("configurations" in r for r in from_nvd) + + +async def test_nvd_failing_leaves_the_keyword_matches_and_says_so(): + kev.clear_cache() + with respx.mock(assert_all_called=False) as router: + router.get(kev.KEV_URL).mock(return_value=httpx.Response(200, json=CATALOGUE)) + router.get(url__startswith=nvd.NVD_API).mock(return_value=httpx.Response(503)) + with pytest.raises(CollectorError) as raised: + await kev.fetch_cves("chrome", cpe=CHROME) + kev.clear_cache() + + assert raised.value.source == nvd.SOURCE + assert {r["id"] for r in raised.value.partial} == NAMED_CHROME + + +def test_a_scan_of_chrome_with_its_cpe_lists_every_exploited_cve(sources): + runner = CliRunner() + assert runner.invoke(cli.app, ["add", "chrome", "--cpe", CHROME]).exit_code == 0 + + result = runner.invoke(cli.app, ["scan", "chrome", "--days", "7"], env={"COLUMNS": "250"}) + + assert result.exit_code == 0, result.output + assert all(cve in result.output for cve in CHROME_CVES), result.output + assert "Total: 5 CVEs found" in " ".join(result.output.split()) + + +async def test_a_scan_over_http_reads_kev_by_the_cpe_too(sources): + """The API's scan, with Debian (which only it asks) refusing.""" + from httpx import ASGITransport, AsyncClient + + import patchradar.api.main as api + + sources.route().mock(return_value=httpx.Response(503)) + async with AsyncClient(transport=ASGITransport(app=api.app), base_url="http://test") as client: + assert (await client.post("/api/watchlist/chrome", params={"cpe": CHROME})).status_code == 200 + body = (await client.post("/api/scan", params={"days": 7})).json() + + assert body["by_software"] == {"chrome": 5} diff --git a/tests/test_law_cache.py b/tests/test_law_cache.py index 282a34b..54e5d80 100644 --- a/tests/test_law_cache.py +++ b/tests/test_law_cache.py @@ -101,6 +101,14 @@ def test_corrupt_file_is_empty(tmp_path): assert LawCache(path).load() == {} +def test_deeply_nested_file_is_empty(tmp_path): + """json.loads raises RecursionError here, which is neither OSError nor + ValueError: the corrupt cache took the whole law check down.""" + path = tmp_path / "law_cache.json" + path.write_text("[" * 200_000 + "]" * 200_000, encoding="utf-8") + assert LawCache(path).load() == {} + + def test_unexpected_structure_is_empty(tmp_path): path = tmp_path / "law_cache.json" path.write_text('["a", "b"]', encoding="utf-8") diff --git a/tests/test_law_checker.py b/tests/test_law_checker.py index 163915e..95cd5db 100644 --- a/tests/test_law_checker.py +++ b/tests/test_law_checker.py @@ -1,6 +1,8 @@ """Tests for mapping PatchRadar findings to GDPR provisions.""" from datetime import UTC, datetime +from pathlib import Path +import httpx import pytest from patchradar import law_fetcher @@ -179,3 +181,145 @@ def test_offline_without_cache(cache): def test_default_cache_location(tmp_path, online): check([cve(severity="CRITICAL")], now=DAY1) assert (tmp_path / "patchradar-home" / "law_cache.json").is_file() + + +# ─── a cited reference must be in the text downloaded now ──────────────────── +# Cellar is answered at the HTTP boundary with the recorded GDPR page +# (fixtures/gdpr_it_excerpt.html), or with a page derived from it: paragraph 2 +# of article 32 renamed out of the ELI paragraph ids, so that the article still +# parses and the paragraph cited for a critical CVE is not in it. + +GDPR_PAGE = (Path(__file__).parent / "fixtures" / "gdpr_it_excerpt.html").read_text(encoding="utf-8") +GDPR_PAGE_WITHOUT_32_2_DERIVED = GDPR_PAGE.replace('id="032.002"', 'id="032.002-derived"') +REAL_FETCH_HTML = law_fetcher.fetch_html + + +def _cellar(page: str | None) -> httpx.Client: + """Cellar answering `page` for the GDPR, or nothing at all (503) when None; + EUR-Lex, the fallback, answering 503 as well.""" + def answer(request: httpx.Request) -> httpx.Response: + if page is not None and str(request.url) == "https://publications.europa.eu/resource/celex/32016R0679": + return httpx.Response(200, text=page) + return httpx.Response(503) + + return httpx.Client(transport=httpx.MockTransport(answer)) + + +@pytest.fixture +def real_fetch(monkeypatch): + """conftest disables fetch_html so that no test reaches the network; the + client passed here answers only from the recording, so the real one goes back.""" + monkeypatch.setattr(law_fetcher, "fetch_html", REAL_FETCH_HTML) + + +def test_the_derived_page_lacks_only_the_paragraph_cited(): + """What the tests below rely on.""" + assert FINDING_ARTICLES["critical"] == ((GDPR, "32(2)"),) + assert GDPR_PAGE_WITHOUT_32_2_DERIVED != GDPR_PAGE + texts = law_fetcher.parse_articles(GDPR_PAGE_WITHOUT_32_2_DERIVED, ("32",)) + assert "32(1)" in texts and "32(2)" not in texts + + +def test_a_paragraph_missing_from_the_fresh_text_is_not_verified(real_fetch, cache): + """Cellar answered and article 32 parsed, but paragraph 2 was not in it: the + act used to be "verified against Cellar" beside "SHA256: not available".""" + law = check([cve(severity="CRITICAL")], cache=cache, now=DAY1, client=_cellar(GDPR_PAGE_WITHOUT_32_2_DERIVED)) + + [status] = law.acts + assert status.source == "unavailable", status + assert "32(2)" in (status.error or "") + assert law.citations[0].sha256 is None + + +def test_a_cached_paragraph_is_not_served_as_verified(real_fetch, cache): + """The same, with the paragraph cached by an earlier audit: its old SHA-256 + and date came back under the "verified" label.""" + first = check([cve(severity="CRITICAL")], cache=cache, now=DAY1, client=_cellar(GDPR_PAGE)) + assert [s.source for s in first.acts] == ["verified"] + assert first.citations[0].sha256 is not None + + law = check([cve(severity="CRITICAL")], cache=cache, now=DAY2, client=_cellar(GDPR_PAGE_WITHOUT_32_2_DERIVED)) + + [status] = law.acts + assert status.source == "cache", status + assert law.citations[0].sha256 == first.citations[0].sha256 + assert law.citations[0].version_date == "2026-09-19" + + +# ─── statuses name the source that was tried ───────────────────────────────── + +def test_a_status_says_the_source_answered_without_the_text(real_fetch, cache): + """The CLI printed "EUR-Lex unreachable" for this: Cellar was reached, and + only the paragraph cited was missing from its page.""" + check([cve(severity="CRITICAL")], cache=cache, now=DAY1, client=_cellar(GDPR_PAGE)) + [status] = check( + [cve(severity="CRITICAL")], cache=cache, now=DAY2, client=_cellar(GDPR_PAGE_WITHOUT_32_2_DERIVED) + ).acts + + assert (status.source, status.tried, status.failure) == ("cache", "Cellar", "not_in_text") + assert status.reason == "Cellar answered without the text cited" + + +def test_a_status_names_both_sources_when_neither_answered(real_fetch, cache): + [status] = check([cve(severity="CRITICAL")], cache=cache, now=DAY1, client=_cellar(None)).acts + + assert (status.source, status.tried, status.failure) == ("unavailable", "Cellar and EUR-Lex", "unreachable") + assert status.reason == "Cellar and EUR-Lex unreachable" + + +def test_a_verified_status_has_no_reason(real_fetch, cache): + [status] = check([cve(severity="CRITICAL")], cache=cache, now=DAY1, client=_cellar(GDPR_PAGE)).acts + + assert (status.source, status.tried, status.failure, status.reason) == ("verified", None, None, None) + + +# ─── a cache that cannot be written ────────────────────────────────────────── + +@pytest.fixture +def read_only_cache(cache): + """Call it after a first audit: it makes `cache` read-only, the file (what + stops os.replace on Windows) and its folder (what stops it on POSIX).""" + import os + import stat + + def freeze(): + cache.path.chmod(stat.S_IREAD) + cache.path.parent.chmod(stat.S_IREAD | stat.S_IEXEC) + try: + cache.update({}, checked_at="2026-10-01T09:30:00Z") + except OSError: + return + pytest.skip("this user can write a read-only file (root?)") + + yield freeze + os.chmod(cache.path.parent, stat.S_IRWXU) + if cache.path.exists(): + os.chmod(cache.path, stat.S_IREAD | stat.S_IWRITE) + + +def test_a_changed_text_is_reported_when_the_cache_cannot_be_written(cache, read_only_cache, monkeypatch): + """The change used to be computed only by a successful save: with the cache + read-only, a law whose text changed was cited with no warning.""" + monkeypatch.setattr(law_fetcher, "fetch_provisions_from", _fake_fetch()) + first = check([cve(severity="CRITICAL")], cache=cache, now=DAY1) + read_only_cache() + + monkeypatch.setattr(law_fetcher, "fetch_provisions_from", _fake_fetch(" (rettificato)")) + law = check([cve(severity="CRITICAL")], cache=cache, now=DAY2) + + assert law.changed == {"GDPR art. 32(2)": first.citations[0].sha256} + assert law.citations[0].sha256 != first.citations[0].sha256 + assert law.citations[0].version_date == "2026-10-01" + + +def test_an_unchanged_text_keeps_its_version_date_when_the_cache_cannot_be_written( + cache, read_only_cache, online +): + """The date of a wording is its first download, saved or not.""" + check([cve(severity="CRITICAL")], cache=cache, now=DAY1) + read_only_cache() + + law = check([cve(severity="CRITICAL")], cache=cache, now=DAY2) + + assert law.changed == {} + assert law.citations[0].version_date == "2026-09-19" diff --git a/tests/test_law_fetcher.py b/tests/test_law_fetcher.py index dd1d1e8..885536b 100644 --- a/tests/test_law_fetcher.py +++ b/tests/test_law_fetcher.py @@ -216,6 +216,123 @@ def fake(url, **kwargs): assert seen["Accept"] == law_fetcher.CELLAR_TYPE +@pytest.mark.parametrize("url", [ + "https://publications.europa.eu/resource/celex/32016R0679", + "https://eur-lex.europa.eu/legal-content/IT/TXT/HTML/?uri=CELEX:32016R0679", + "https://www.normattiva.it/uri-res/N2Ls?urn:nir:stato:decreto.legislativo:2024-09-04;138~art24!vig=", +]) +@pytest.mark.parametrize("location", [ + "https://example.org/page", + "https://normattiva.it/page", # not www.normattiva.it + "https://publications.europa.eu.example.org/page", + "https://eur-lex.europa.eu:8443/page", +]) +def test_a_redirect_off_the_source_host_is_refused(url, location): + """A text served from another host would be cited as "verified against + Cellar" (or EUR-Lex, or Normattiva) without being theirs.""" + seen = [] + + def handler(request): + seen.append(str(request.url)) + if len(seen) == 1: + return httpx.Response(302, headers={"Location": location}) + return httpx.Response(200, text="elsewhere") + + with pytest.raises(LawFetchError, match="redirected") as caught: + real_fetch_html(url, client=_client(handler)) + assert seen == [url] + assert caught.value.source == law_fetcher._source_of(url) + + +@pytest.mark.parametrize("url", [ + "https://eur-lex.europa.eu/legal-content/IT/TXT/HTML/?uri=CELEX:32016R0679", + "https://www.normattiva.it/uri-res/N2Ls?urn:nir:stato:decreto.legislativo:2024-09-04;138~art24!vig=", +]) +def test_a_redirect_on_the_source_host_is_followed(url): + """The http:// hop Cellar makes is in test_law_fetch_https.py.""" + def handler(request): + if request.url.path != "/moved": + return httpx.Response(302, headers={"Location": f"http://{request.url.host}/moved"}) + return httpx.Response(200, text="moved") + + assert real_fetch_html(url, client=_client(handler)) == "moved" + + +def test_a_host_httpx_cannot_encode_is_a_fetch_error(): + """httpx raises idna.core.IDNAError, a ValueError and no httpx.HTTPError, for + a host like "xn--": it used to skip the EUR-Lex fallback and the cache, and + end the law check.""" + client = _client(lambda request: httpx.Response(200, text="")) + with pytest.raises(LawFetchError, match="unreachable"): + real_fetch_html("https://xn--/DOC_1", client=client) + + +def test_a_redirect_to_a_host_httpx_cannot_encode_is_a_fetch_error(): + """The same host in a Location header: a 303 from Cellar to http://xn--/DOC_1.""" + client = _client(lambda request: httpx.Response(303, headers={"Location": "http://xn--/DOC_1"})) + with pytest.raises(LawFetchError, match="Cellar"): + real_fetch_html("https://publications.europa.eu/resource/celex/32016R0679", client=client) + + +# ─── size and time ──────────────────────────────────────────────────────────── +# The limits are lowered for these tests, so that they run in a second; set +# with raising=False, so that the tests fail on the behaviour, not on a name. + +CELLAR_GDPR = "https://publications.europa.eu/resource/celex/32016R0679" + + +def _streamed(chunks, pause=0.0): + import time + + def body(): + for chunk in chunks: + time.sleep(pause) + yield chunk + + return _client(lambda request: httpx.Response(200, content=body())) + + +def test_a_response_larger_than_the_limit_is_refused(monkeypatch): + monkeypatch.setattr(law_fetcher, "MAX_RESPONSE_BYTES", 64 * 1024, raising=False) + client = _streamed([b"x" * 1024] * 200) + with pytest.raises(LawFetchError, match="Cellar sent more than 65536 bytes"): + real_fetch_html(CELLAR_GDPR, client=client) + + +def test_a_response_within_the_limit_is_read_whole(monkeypatch): + monkeypatch.setattr(law_fetcher, "MAX_RESPONSE_BYTES", 64 * 1024, raising=False) + assert real_fetch_html(CELLAR_GDPR, client=_streamed([b"", b"ok", b""])) == "ok" + + +def test_a_server_that_drips_is_cut_off_at_the_deadline(monkeypatch): + """Each chunk arrives well within the read timeout, so only an overall + deadline stops it: 40 chunks, 0.1 s apart, against a 0.5 s deadline.""" + import time + + monkeypatch.setattr(law_fetcher, "FETCH_DEADLINE", 0.5, raising=False) + started = time.monotonic() + with pytest.raises(LawFetchError, match="Cellar took more than 0.5 s"): + real_fetch_html(CELLAR_GDPR, client=_streamed([b"x"] * 40, pause=0.1)) + assert time.monotonic() - started < 2 + + +def test_the_limits_are_sane(): + """Room for the largest act the Radars cite (the GDPR from Cellar was 823981 + bytes on 2026-10-11), and an end to a fetch that never finishes.""" + assert 8 * 1024 * 1024 <= law_fetcher.MAX_RESPONSE_BYTES <= 64 * 1024 * 1024 + assert 30 <= law_fetcher.FETCH_DEADLINE <= 300 + + +def test_an_article_missing_from_a_normattiva_page_names_normattiva(): + """"not found in the EUR-Lex page" was said of every source's page.""" + page = (FIXTURES / "normattiva_dlgs138_2024_art24_recorded.html").read_text(encoding="utf-8") + with pytest.raises(LawFetchError) as caught: + parse_articles(page, ("99",), "Normattiva") + assert "Normattiva" in str(caught.value) + assert "EUR-Lex" not in str(caught.value) + assert (caught.value.source, caught.value.reached) == ("Normattiva", True) + + def test_an_error_from_cellar_names_cellar(): client = httpx.Client(transport=httpx.MockTransport(lambda request: httpx.Response(503))) with pytest.raises(LawFetchError, match="Cellar answered HTTP 503"): diff --git a/tests/test_law_sources.py b/tests/test_law_sources.py index 386bdf4..4515912 100644 --- a/tests/test_law_sources.py +++ b/tests/test_law_sources.py @@ -96,6 +96,27 @@ def test_normattiva_update_marks_and_heading_are_dropped(): assert "vigore" not in text +def test_normattiva_bis_paragraphs_can_be_cited(): + """Art. 21 of the Consumer Code has paragraphs 3-bis and 4-bis: they were + part of the article's text, and could not be cited on their own.""" + texts = parse_articles(page("normattiva_cdc_art21.html"), ("21",)) + + assert texts["21(3-bis)"].startswith("3-bis. ") + assert texts["21(4-bis)"].startswith("4-bis. ") + assert texts["21"].split("\n") == [ + texts[f"21({n})"] for n in ("1", "2", "3", "3-bis", "4", "4-bis") + ] + + +@pytest.mark.parametrize("suffix", ["ter", "quater", "quinquies", "sexies", "decies", "terdecies"]) +def test_normattiva_later_latin_paragraphs_can_be_cited(suffix): + """The same page with paragraph 4-bis renumbered, as later amendments number theirs.""" + html = page("normattiva_cdc_art21.html").replace(">4-bis. <", f">4-{suffix}. <") + assert html != page("normattiva_cdc_art21.html") + + assert parse_articles(html, ("21",))[f"21(4-{suffix})"].startswith(f"4-{suffix}. ") + + def test_normattiva_articles_21_and_49(): assert parse_articles(page("normattiva_cdc_art21.html"), ("21",))["21(1)(a)"] == ( "a) l'esistenza o la natura del prodotto;" diff --git a/tests/test_law_unverifiable.py b/tests/test_law_unverifiable.py index c5ed8b8..bca5216 100644 --- a/tests/test_law_unverifiable.py +++ b/tests/test_law_unverifiable.py @@ -85,3 +85,29 @@ def test_an_act_read_from_the_cache_still_has_its_hash(): assert "cache" in out assert "not verifiable" not in out, out + + +# ── which source, and what went wrong with it ─────────────────────────────── + + +def test_a_source_that_answered_without_the_text_is_not_called_unreachable(): + """Cellar answered, and the paragraph cited was not in its page: the line + said "EUR-Lex unreachable", naming neither the source nor what happened.""" + out = render(LawCheckResult( + citations=[], notes=["n"], + acts=[ActStatus(GDPR, "cache", "GDPR art. 32(2) not found in the text from Cellar", + tried="Cellar", failure="not_in_text")], + )) + + assert "Cellar answered without the text cited" in out + assert "unreachable" not in out + + +def test_both_sources_unreachable_with_nothing_cached_say_which(): + out = render(LawCheckResult( + citations=[], notes=["n"], + acts=[ActStatus(GDPR, "unavailable", "Cellar answered HTTP 503; and EUR-Lex answered HTTP 202", + tried="Cellar and EUR-Lex", failure="unreachable")], + )) + + assert "Cellar and EUR-Lex unreachable and nothing cached" in out diff --git a/tests/test_nis2_evidence.py b/tests/test_nis2_evidence.py index e8c1fbb..16e464f 100644 --- a/tests/test_nis2_evidence.py +++ b/tests/test_nis2_evidence.py @@ -241,6 +241,9 @@ def test_msrc_is_asked_for_a_month_and_nothing_else(): def test_nvd_receives_the_watched_name_and_first_the_cve_ids(doc): nvd = (PACKAGE / "collectors" / "nvd.py").read_text(encoding="utf-8") assert '"keywordSearch": keyword' in nvd + # For an entry with a CPE: its part, vendor and product, to read KEV through NVD + assert '"virtualMatchString": f"cpe:2.3:{parsed.part}:{parsed.vendor}:{parsed.product}"' in nvd + assert "`virtualMatchString`, con `hasKev`" in doc assert 'headers = {"apiKey": key} if key else {}' in nvd epss = (PACKAGE / "epss.py").read_text(encoding="utf-8") assert 'params = {"cve": ",".join(chunk)' in epss @@ -280,7 +283,7 @@ def test_the_law_check_runs_from_the_command_line_scan_only(): def test_the_web_scan_asks_debian_and_the_command_line_scan_does_not(): assert "debian_fetch(sw)" in inspect.getsource(api_main._scan_one) command_line = inspect.getsource(cli._scan_target) - assert "fetches = (fetch_cves, msrc_fetch, kev_fetch)" in command_line + assert "fetches = (fetch_cves, msrc_fetch, kev_by_cpe)" in command_line assert "debian_fetch" not in command_line diff --git a/tests/test_watchlist_versions.py b/tests/test_watchlist_versions.py index a51a3e3..cc02df4 100644 --- a/tests/test_watchlist_versions.py +++ b/tests/test_watchlist_versions.py @@ -201,6 +201,18 @@ def test_a_cpe_with_an_unescaped_plus_is_refused_with_the_form_nvd_wants(): assert asyncio.run(database.get_watchlist()) == [] +def test_a_cpe_typed_with_capitals_is_stored_as_nvd_writes_it(): + """Stored as typed, `cpe:2.3:a:Mozilla:Thunderbird` matched none of NVD's + lower-case names and every CVE of the program read as "not affected".""" + result = CliRunner().invoke(cli.app, ["add", "thunderbird", "--version", "140.0", + "--cpe", "cpe:2.3:a:Mozilla:Thunderbird"]) + + assert result.exit_code == 0, result.output + [entry] = asyncio.run(database.get_watch_entries()) + assert entry.cpe == "cpe:2.3:a:mozilla:thunderbird" + assert entry.installed() == ("140.0", "mozilla", "thunderbird") + + def test_list_shows_the_version_and_the_cpe(): runner = CliRunner() runner.invoke(cli.app, ["add", "notepad", "--version", "8.5.0", "--cpe", NOTEPADPP])