Skip to content

feat: drive the browser through core's agent_browser.client when offered; typed timeouts/walls; comps cache (0.4.0) - #8

Merged
mabry1985 merged 3 commits into
mainfrom
feat/core-browser-client
Nov 9, 2026
Merged

mabry1985 merged 3 commits into
mainfrom
feat/core-browser-client

Conversation

@mabry1985

Copy link
Copy Markdown
Member

What

  • Use core's browser client when the host offers it. On a protoAgent core whose agent_browser plugin offers agent_browser.client (feat(agent_browser): a typed browser client for other plugins (agent_browser.client service) protoAgent#4941), every browser command goes through core's hardened client: output cap, process-group kill, CLI resolution including core's first-use download, the option guard, the per-session lock, and typed results. coreclient.CoreBrowser adapts it to the surface the tools already use (open / wait_for / eval_json / refocus).
    • Safe before the core release ships. The service is looked up per call. With no host, an older core, or agent_browser off, the lookup returns None and the tools run on today's browser.py.
    • Same browser either way. The session name is kept verbatim (instance_scoped=False), so the daemon and profile already signed in stay in use. The launch options are the same --profile, --headed and the one stealth flag when stealth is on, so switching paths never relaunches Chrome. The labelled tab is the same ebaytab. Stealth and anti-detection behaviour are unchanged.
    • open(..., check_walls=False): the plugin's own settle-and-classify loop still reads eBay's gate chain (search → /splashui/captcha → sign-in → back). A probe right after the navigation would catch that chain mid-way. Core's generic wall check runs only when a page has no results and no eBay reason, to name a Cloudflare / 403 / CAPTCHA page before blaming the markup.
  • Typed failures with next steps. Every tool error now carries error_kind (timeout / wall / unavailable / browser / page / bad_input) and a next_step. For timeout the next step says it is NOT a wall or a market finding, that the plugin already retried once in a fresh tab, and to retry at most once more. The skill doctrine says the same.
  • Shorter navigation timeout plus one automatic recovery, on both paths. nav_timeout_s (35 s, new config) replaces the 60 s per-command timeout for navigations. A timed-out navigation replaces the plugin's own tab and retries once, then raises a typed timeout.
  • Explicit wait timeouts (the main fix, on both paths). wait_for now always sends a 10 s timeout. It used to send none, so the daemon's default 30 s wait collided with the CLI's 30 s read timeout, and the CLI re-sent the wait five times.
  • Comps cache. ebay_price_check and ebay_price_and_profit answer a repeat of the same search (query, sold/active, condition) within cache_ttl_min (360, new config; 0 = off) from the last completed check. The answer is marked cached with its age, plus a note the model repeats. fresh=true bypasses the cache. It is stored in a new checks table beside the price history. The existing observations table carries no sold/active basis or condition, so it can't serve as the cache, and a cache hit is not recorded as a new observation. A failed search is never cached. merchantAgent's real store (…/merchantAgent-6604/marketplace/prices.db) shows the pattern this is for: 196 queries, the top ones checked 3–6 times each.
  • Test hygiene. Host-free runs defaulted the history file to ~/.protoagent/marketplace-prices.db, so the suite had been writing its fixture rows (x, a, switch oled) into the operator's home directory. Each test now gets its own file. (I deleted the 28 rows and the checks table my first test run added there; the file is back to its 1066 rows.)
  • Version 0.4.0 (manifest + pyproject). README and skill updated. No release cut.

Why: the evidence

The audit found 14% of ebay_price_check calls running ≥60 s, p95 126 s, and each failed retry taking about 151 s. agent.log showed "the plugin's browser tab would not take focus — replaced it". Timeouts were untyped, so the agent couldn't tell a stuck browser from a wall.

The 151 s is the CLI's own retry. agent-browser 0.27.1 reads the daemon's reply with a 30 s socket timeout and treats that timeout as transient, so it re-sends the command up to 5 times (cli/src/connection.rs). Measured on a private session:

$ agent-browser wait '#never-appears'      # what browser.py's wait_for sent
✗ Failed to read: Resource temporarily unavailable (os error 35) (after 5 retries - daemon may be busy or unresponsive)
rc=1 elapsed=152s

Every eBay gate page (no results ever appear) hit this in wait_for before the settle loop even started.

Verification

  • ruff check . && ruff format --check . → clean. pytest -q → 209 passed: 189 existing plus 20 in tests/test_core_client.py, covering:
    • path selection and the exact client options (no added flags, stealth flag only with stealth)
    • a rebuild on a new factory
    • typed timeout / wall / unavailable / cli_error, eBay's own walls typed as walls, and generic-wall naming
    • the cache (hit, fresh, sold vs. active, condition, price-and-profit sharing, off, failed searches never cached, the age window)
    • the fallback wait env timeout, nav-timeout recovery, and a typed second timeout
  • Live, real browser, private sessions only (abtest-ebay-* and abtest-ebayfb-*, closed after; the ebay and default sessions were never touched). This ran the plugin's real _read_page / _fetch over core's real BrowserClient (from the #4941 worktree) against a local stand-in results page:
    client: BrowserClient(session='abtest-ebay-1701', own_tab='ebaytab') flags: []
    read: found_container=True rows=2 in 9.2s
    listings: [('Nintendo Switch OLED White', 210.0, 12.99), ('Nintendo Switch OLED Neon', 250.0, 0.0)] dropped 0 meta {'headline_count': 2, ...}
    wait_for absent -> False in 2.4s
    tabs: [('t1', None, False), ('t2', 'ebaytab', True)]
    dead port -> BrowserError browser could not open http://127.0.0.1:9/: ... net::ERR_UNSAFE_PORT
    close: ok
    
    Fallback path: wait_for absent -> False 2.3s (it was ~150 s).
  • Not exercised against live ebay.com. That would mean driving the operator's signed-in ebay session, which merchantAgent is using.

Rollout

  • Mergeable and releasable on its own: with no agent_browser.client on the host, the fallback runs, and it gets the wait / nav-timeout / typed-error / cache fixes too.
  • The core-client path turns on once protoAgent#4941 ships in a core release and that core runs on merchantAgent with agent_browser enabled.

🤖 Generated with Claude Code

…red; typed timeouts/walls; comps cache (0.4.0)

- coreclient.py: when the host offers agent_browser.client (protoAgent core),
  every command runs through core's hardened client: output cap, process-group
  kill, CLI resolution, option guard, typed results. Same session (verbatim),
  same profile/headed/stealth flag, same `ebaytab`, so switching never
  relaunches Chrome. Looked up per call; older cores fall back to browser.py.
- Typed failures: every tool error carries error_kind (timeout / wall /
  unavailable / browser / page / bad_input) and a next_step for the model.
- Fallback browser.py: every selector wait carries an explicit 10 s timeout
  (an un-timed failing wait cost ~150 s: the CLI re-sends it 5x on its 30 s
  read timeout); navigation timeout nav_timeout_s (35 s) with one automatic
  recovery in a fresh own tab; typed BrowserTimeout/BrowserWall/
  BrowserUnavailable.
- Comps cache: a repeat of the same search (query, sold/active, condition)
  within cache_ttl_min (360) answers from the last completed check (new
  `checks` table beside the price history), marked `cached`; fresh=true
  bypasses. A cache hit is not a new history observation.
- A results-less page with no eBay reason is checked against core's generic
  wall detector before being reported as a markup change.
- Tests: the suite no longer writes into ~/.protoagent/marketplace-prices.db
  (each test gets its own history file). 209 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA panel review — WARN

code-review-structural · head 52d4508f22d1 · formal

All four findings were confirmed by the verifier with no disagreements. The change is low-risk overall: a comps cache layer, a CoreBrowser wrapper, and tighter timeouts. The one thing to fix first is the condition-case mismatch in history.py — it silently serves unfiltered comps as a "used" answer. The domain-not-in-cache-key issue in tools.py is the second-priority fix since it can serve cross-marketplace results. No coverage gaps; no refuted or uncertain items.

Findings

Severity Location Finding Verified
🟡 minor history.py:134 The comps cache keys on condition.lower(), but search_url only honours exact-case 'new'/'used'. A call like condition='Used' (or any unrecognised value) fe… confirmed
🟡 minor tools.py:526 The new comps cache is keyed only on normalized query, source, basis and condition. The configured eBay domain is not part of the key. A repeat search after … confirmed
🟡 minor coreclient.py:83 CoreBrowser.init stores self.factory = factory but nothing in the repo ever reads it — a dead instance attribute. confirmed
🟡 minor pyproject.toml:1 Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in a… confirmed
findings JSON (machine-readable)
[
  {
    "file": "history.py",
    "line": 134,
    "severity": "minor",
    "category": "correctness",
    "claim": "The comps cache keys on `condition.lower()`, but `search_url` only honours exact-case 'new'/'used'. A call like condition='Used' (or any unrecognised value) fetches the unfiltered 'any' results and stores them under the 'used' key. A later condition='used' call is then served those unfiltered comps as a cached 'used' answer.",
    "evidence": "history.py record_check/recent_check use `(condition or \"any\").lower()`; extract.py search_url uses `if cond := CONDITIONS.get(condition, \"\"):` with `CONDITIONS = {\"new\": \"3\", \"used\": \"4\", \"any\": \"\"}`, so 'Used' yields no LH_ItemCondition param while the cache row is stored as condition='used'.",
    "verdict": "confirmed",
    "note": "Re-read history.py: record_check stores `(condition or \"any\").lower()` and recent_check queries with `(condition or \"any\").lower()`. Re-read extract.py: `CONDITIONS = {\"new\": \"3\", \"used\": \"4\", \"any\": \"\"}` and `if cond := CONDITIONS.get(condition, \"\")` \u2014 'Used' is not a key, so no LH_ItemCondition param is added. The mismatch is real: unfiltered results stored under the 'used' key."
  },
  {
    "file": "tools.py",
    "line": 526,
    "severity": "minor",
    "category": "removed-behavior",
    "claim": "The new comps cache is keyed only on normalized query, source, basis and condition. The configured eBay `domain` is not part of the key. A repeat search after switching `ebay.domain` (e.g. www.ebay.com to www.ebay.co.uk), or from another agent sharing the default ~/.protoagent history DB, is answered from the other marketplace's listings and currency. The response carries the new domain's `url`. Before this PR every call read the configured domain live.",
    "evidence": "hit = h.recent_check(query, \"ebay\", basis=basis, condition=condition, max_age_min=cache_ttl_min)  \u2014 `recent_check`/`record_check` filter on `key = ? AND source = ? AND basis = ? AND condition = ?` and the checks table has no domain column, while `url = search_url(query, domain=domain, ...)` is recomputed from the current config and returned beside the cached listings.",
    "verdict": "confirmed",
    "note": "Re-read tools.py _comps: `hit = h.recent_check(query, \"ebay\", basis=basis, condition=condition, max_age_min=cache_ttl_min)` \u2014 no domain argument. Re-read history.py schema: checks table has key, source, basis, condition, seen_at, payload \u2014 no domain column. The url is recomputed from the current config's domain, so a cached hit returns the new domain's URL beside the old domain's listings."
  },
  {
    "file": "coreclient.py",
    "line": 83,
    "severity": "minor",
    "category": "conventions",
    "claim": "CoreBrowser.__init__ stores self.factory = factory but nothing in the repo ever reads it \u2014 a dead instance attribute.",
    "evidence": "coreclient.py:83 `self.factory = factory` (diff: `+        self.factory = factory`); a repo-wide search for `\\.factory` returns only this assignment (coreclient.py:83) with no reader, unlike the sibling attributes `session` (read by tools._session_lock) and `stealth` (read at tools.py:679).",
    "verdict": "confirmed",
    "note": "Searched the entire repo (24 files) at head for `\\.factory`: exactly 1 match \u2014 `coreclient.py:83: self.factory = factory`. No reader. Searched for `\\.session`: 6 matches including tools._session_lock. The factory attribute is stored but never read."
  },
  {
    "file": "pyproject.toml",
    "line": 1,
    "severity": "minor",
    "category": "conventions",
    "claim": "Missing `dependencies` field in [project] table; for a library-kind Python project that ships executable code, `pip install` would succeed without pulling in any runtime requirements.",
    "evidence": "The [project] table declares name, version, description, requires-python, and license but has no `dependencies` list. For a library-kind Python project that ships executable code (the README describes tool functions like ebay_price_and_profit and a browser wrapper), `pip install` of this package would succeed without pulling in any runtime requirements. If the plugin imports any third-party Python Fix: Add a `dependencies` list to the [project] table declaring all runtime third-party packages (even if the list is currently empty, an explicit `dependencies = []` documents the intent and avoids ambigu (protopatch confidence: medium)",
    "source": "protopatch",
    "verdict": "confirmed",
    "note": "Re-read pyproject.toml: [project] has name, version, description, requires-python, license \u2014 no dependencies field. Searched for third-party imports: `from langchain_core.tools import tool` at tools.py:473. A standalone `pip install` would not pull in langchain-core."
  }
]

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promoting the WARN verdict for head 52d4508f22d1: all checks terminal-green, zero unresolved review threads. (approve-on-green)

Open findings carried by this approval — non-blocking, but they did not go away:

  • minor history.py:134 — The comps cache keys on condition.lower(), but search_url only honours exact-case 'new'/'used'. A call like condition='Used' (or any unrecognised value) fetches the unfiltered 'any' results and st
  • minor tools.py:526 — The new comps cache is keyed only on normalized query, source, basis and condition. The configured eBay domain is not part of the key. A repeat search after switching ebay.domain (e.g. www.ebay.co
  • minor coreclient.py:83 — CoreBrowser.init stores self.factory = factory but nothing in the repo ever reads it — a dead instance attribute.
  • minor pyproject.toml:1 — Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in any runtime requirements.

Approving a WARN does not resolve its findings (issue #22).

…the eBay site; drop a dead attribute

QA panel (#8): condition='Used' searched unfiltered but cached under 'used', and a
repeat on another ebay.domain was answered with the first site's comps.
search_url now honours new/used in any case (effective_condition), the cache keys
on that, and its source is ebay@<domain>.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA panel review — WARN

code-review-structural · head 3a4f96f27643 · formal

Overall risk is low: two polish items remain, no blockers or majors. The p75 off-by-one in extract.py is the more substantive of the two (a real correctness asymmetry), but it only manifests when n is divisible by 4. Verification confirmed both findings against head 3a4f96f; nothing was refuted or downgraded. The four round-1 minor findings (condition case, domain in cache key, dead factory, missing dependencies) are not carried forward — the round-2 brief confirms three were fixed; the pyproject.toml dependencies gap was not explicitly addressed this round but is a minor packaging concern. No panel disagreement and no verification gaps.

Findings

Severity Location Finding Verified
🟡 minor tools.py:525 The new cache_source writes domain-qualified values (e.g. ebay@www.ebay.com) into checks.source, but the schema comment in history.py still documents that colu… confirmed
🟡 minor pyproject.toml:1 Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in a… confirmed
⚪ nit extract.py:239 The p75 index is computed as int(n * 0.75) without the correction that p25 applies, so when n is divisible by 4 it returns the maximum instead of the 75th perc… confirmed · nearby, not gating
findings JSON (machine-readable)
[
  {
    "file": "tools.py",
    "line": 525,
    "severity": "minor",
    "category": "conventions",
    "claim": "The new cache_source writes domain-qualified values (e.g. ebay@www.ebay.com) into checks.source, but the schema comment in history.py still documents that column as 'ebay | amazon', so the documented value set no longer matches what the code stores.",
    "evidence": "tools.py: `cache_source = f\"ebay@{str(domain).strip().lower()}\"` is passed as `source` to `record_check`/`recent_check`; history.py schema: `    source    TEXT NOT NULL,      -- ebay | amazon`.",
    "verdict": "confirmed",
    "note": "tools.py:525 `cache_source = f\"ebay@{str(domain).strip().lower()}\"` confirmed; history.py:49 checks table `source    TEXT NOT NULL,      -- ebay | amazon` confirmed \u2014 comment is stale."
  },
  {
    "file": "extract.py",
    "line": 239,
    "severity": "nit",
    "category": "correctness",
    "claim": "The p75 index is computed as int(n * 0.75) without the correction that p25 applies, so when n is divisible by 4 it returns the maximum instead of the 75th percentile (e.g. n=4, sorted [10,20,30,40] returns 40 instead of 30).",
    "evidence": "extract.py:239 \u2014 for n=4, int(4*0.75)=3, so totals[3]=40 (the max); the p25 path uses a `(1 if n % 4 == 0 else 0)` correction that p75 lacks.",
    "source": "protopatch",
    "verdict": "confirmed",
    "note": "extract.py:239 `\"p75\": round(totals[min(n - 1, int(n * 0.75))], 2)` confirmed; line 238 p25 has the `-(1 if n % 4 == 0 else 0)` correction that p75 lacks. For n=4, int(4*0.75)=3 \u2192 totals[3]=max. \u2014 nearby: in code this PR did not change (outside its changed lines and the functions they sit in) \u2014 reported, not gated (#232)",
    "nearby": true
  },
  {
    "file": "pyproject.toml",
    "line": 1,
    "severity": "minor",
    "category": "conventions",
    "claim": "Missing `dependencies` field in [project] table; for a library-kind Python project that ships executable code, `pip install` would succeed without pulling in any runtime requirements.",
    "evidence": "The [project] table declares name, version, description, requires-python, and license but has no `dependencies` list. For a library-kind Python project that ships executable code (the README describes tool functions like ebay_price_and_profit and a browser wrapper), `pip install` of this package would succeed without pulling in any runtime requirements. If the plugin imports any third-party Python Fix: Add a `dependencies` list to the [project] table declaring all runtime third-party packages (even if the list is currently empty, an explicit `dependencies = []` documents the intent and avoids ambigu (protopatch confidence: medium)",
    "source": "protopatch",
    "verdict": "confirmed",
    "note": "Re-read pyproject.toml: [project] has name, version, description, requires-python, license \u2014 no dependencies field. Searched for third-party imports: `from langchain_core.tools import tool` at tools.py:473. A standalone `pip install` would not pull in langchain-core. \u2014 carried unchanged: this file is byte-identical to the round that raised it (incremental, #313)",
    "carried": true,
    "carried_by": "incremental",
    "since": "52d4508f22d11ab2ee85c8ebbeee13dc1daff433"
  }
]

1 structural finding(s) are nearby notes, not part of the verdict: they sit in code this PR did not change (outside its changed lines and the functions those lines are in). Worth a look; not a request for this PR.

  • extract.py:238 (nit) — The p75 index is computed as int(n * 0.75) without the correction that p25 applies, so when n is divisible by 4 it returns the maximum instead of the 75th perce

Incremental re-review (#313): 7 of 11 file(s) changed since head 52d4508f22d1 (or depend on a change) and were reviewed. 4 unchanged file(s) were not re-read. 1 finding(s) on them are carried forward unchanged.
Comment @vera review for a full re-review.


Addressed since the last round (2 minor/nit): tools.py:526, coreclient.py:83 — no longer raised, and the PR changed the code around them since they were raised.

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promoting the WARN verdict for head 3a4f96f27643: all checks terminal-green, zero unresolved review threads. (approve-on-green)

Open findings carried by this approval — non-blocking, but they did not go away:

  • minor tools.py:525 — The new cache_source writes domain-qualified values (e.g. ebay@www.ebay.com) into checks.source, but the schema comment in history.py still documents that column as 'ebay | amazon', so the documented
  • nit extract.py:239 — The p75 index is computed as int(n * 0.75) without the correction that p25 applies, so when n is divisible by 4 it returns the maximum instead of the 75th percentile (e.g. n=4, sorted [10,20,30,40] re
  • minor pyproject.toml:1 — Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in any runtime requirements.

Approving a WARN does not resolve its findings (issue #22).

…when 4 | n); checks.source comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA panel review — WARN

code-review-structural · head 6d12180cd2c2 · formal

No findings were raised, so there was nothing to verify. The round-3 panel found no defects across all five angles; the verifier confirmed the empty array. The two open minor priors (pyproject.toml missing dependencies, tools.py:525 schema comment) remain open but are below the blocker/major threshold for disposition tracking. No coverage gaps.

Findings

Severity Location Finding Verified
🟡 minor pyproject.toml:1 Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in a… confirmed
findings JSON (machine-readable)
[
  {
    "file": "pyproject.toml",
    "line": 1,
    "severity": "minor",
    "category": "conventions",
    "claim": "Missing `dependencies` field in [project] table; for a library-kind Python project that ships executable code, `pip install` would succeed without pulling in any runtime requirements.",
    "evidence": "The [project] table declares name, version, description, requires-python, and license but has no `dependencies` list. For a library-kind Python project that ships executable code (the README describes tool functions like ebay_price_and_profit and a browser wrapper), `pip install` of this package would succeed without pulling in any runtime requirements. If the plugin imports any third-party Python Fix: Add a `dependencies` list to the [project] table declaring all runtime third-party packages (even if the list is currently empty, an explicit `dependencies = []` documents the intent and avoids ambigu (protopatch confidence: medium)",
    "source": "protopatch",
    "verdict": "confirmed",
    "note": "Re-read pyproject.toml: [project] has name, version, description, requires-python, license \u2014 no dependencies field. Searched for third-party imports: `from langchain_core.tools import tool` at tools.py:473. A standalone `pip install` would not pull in langchain-core. \u2014 carried unchanged: this file is byte-identical to the round that raised it (incremental, #313)",
    "carried": true,
    "carried_by": "incremental",
    "since": "52d4508f22d11ab2ee85c8ebbeee13dc1daff433"
  }
]

Incremental re-review (#313): 6 of 11 file(s) changed since head 3a4f96f27643 (or depend on a change) and were reviewed. 5 unchanged file(s) were not re-read. 1 finding(s) on them are carried forward unchanged.
Comment @vera review for a full re-review.

@protoreview protoreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promoting the WARN verdict for head 6d12180cd2c2: all checks terminal-green, zero unresolved review threads. (approve-on-green)

Open findings carried by this approval — non-blocking, but they did not go away:

  • minor pyproject.toml:1 — Missing dependencies field in [project] table; for a library-kind Python project that ships executable code, pip install would succeed without pulling in any runtime requirements.

Approving a WARN does not resolve its findings (issue #22).

@mabry1985
mabry1985 merged commit 1993486 into main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant