Skip to content

bsk click silently reports success while delivering no input when the Agent Window is in the background (--no-focus) #242

Closed

Description

@bailu8345315-tech

Summary

bsk click returns success — exit code 0 plus a plausible click coordinate — while no input is delivered to the page at all when the session's Agent Window is not in the foreground (session started with --no-focus). Nothing in the response distinguishes "clicked" from "did nothing", so an agent that trusts the CLI proceeds as if the interaction happened.

Environment

  • bsk: 0.2.1 (bsk update --check → bsk 0.2.1 is already up to date)
  • OS: Windows 11 (x64)
  • Browser: Chromium-based, BrowserSkill extension connected (bsk doctor: extension connected · 1 browser, browser protocol compatible)
  • Driver: a harness plugin that shells out to the bsk CLI (@wxg-prc-cpg/browser-skill-dsh-plugin 0.2.1). The plugin forwards the target verbatim (bsk click --session <id> <ref|selector>); the behaviour is reproducible from the CLI alone.
  • Page viewport at the time: 631x880 CSS px, devicePixelRatio 1.5
  • Test page: plain local HTML (no framework, no iframe)

Minimal reproduction

Test page — the two buttons and the four capture-phase listeners are all that matter:

<!doctype html>
<html><head><meta charset="utf-8"><title>bsk click probe</title>
<style>button{display:block;width:260px;padding:12px}#status{font-size:15px;font-weight:700;color:#b00;white-space:pre-wrap}
.f{height:60px;border-bottom:1px solid #ddd}</style></head>
<body>
  <button id="top" onclick="mark('TOP')">TOP</button>
  <div id="status">status: none</div>
  <div id="filler"></div>
  <button id="bottom" onclick="mark('BOTTOM')">BOTTOM</button>
  <script>
    const filler = document.getElementById('filler');
    for (let i = 0; i < 40; i++) { const d = document.createElement('div'); d.className = 'f'; filler.appendChild(d); }
    window.mark = n => { document.getElementById('status').textContent = 'status: clicked ' + n; };
    const log = [];
    for (const k of ['pointerdown', 'mousedown', 'mouseup', 'click']) {
      document.addEventListener(k, e => {
        log.push(`${k} client=(${Math.round(e.clientX)},${Math.round(e.clientY)}) target=${e.target.tagName}#${e.target.id||''}`);
        document.getElementById('status').textContent = 'events:\n' + log.slice(-6).join('\n');
      }, true);
    }
  </script>
</body></html>

Steps:

  1. bsk session start --no-focus → navigate the Agent Window to the page (background window, never brought to the front)
  2. bsk click --session <id> '#top' — CSS selector on purpose, to rule out snapshot-ref staleness
  3. Read the page: the status line is still status: none; none of the four listeners fired
  4. bsk session stop, then bsk session start (no --no-focus) → repeat the identical click → the status line becomes status: clicked TOP

Observed

Background window — success reported, nothing delivered:

$ bsk click --session <id> '#top'
clicked at (138, 39.33333396911622) on tab <tabId>

…while the page's event log stayed empty (status: none).

Foreground window — identical target, identical reported coordinate (138, 39.33), and the click lands (status: clicked TOP).

So the coordinates bsk computes are correct in both runs; only delivery differs, and neither the CLI output nor the exit code reflects it. Repeating the click (ref form, then selector form) does not help while the window is in the background — both attempts report success, neither is delivered.

Expected

Either:

  1. Deliver it: activate/raise the Agent Window (or the target tab) before dispatching input, so a click works regardless of how the session was started; or
  2. Fail loudly: a non-zero exit plus an explicit error such as input not delivered: Agent Window is not focused, so callers can focus and retry.

Reporting a coordinate plus success while delivering nothing is the dangerous middle ground.

Impact

We hit this while filling a 12-question multiple-choice form. The session was started with --no-focus to avoid stealing the user's focus. The clicks issued while the window was still in the background never registered (those four questions were left unanswered), while the ones issued later landed — and every single call reported success. The transcript looked perfect; the form was partially unset. That is a silent-corruption failure mode for any agent-driven form or UI task.

(The "window came to the foreground" transition is our inference from the session; the A/B test above is the hard evidence.)

What did not help

  • bsk click --help exposes --ref, --selector, --button, --click-count, --modifiers, --timeout, --tab-id — there is no activate / bring-to-front option.
  • bsk window only has resize; there is no focus / raise subcommand.
  • bsk tab select exists ("Activate (focus) a tab in the session's Agent Window") but I could not verify whether it also raises the containing window: the daemon is owned by the integrating harness, so from an outer shell bsk status / bsk session list answer ensure daemon is running (exit code 2). If tab select does raise the window, documenting "input commands require a foreground Agent Window" would already be a big improvement — and a bsk click --activate (or equivalent) would be the natural fix.

中文摘要

问题:Agent Window 处于后台时(会话以 --no-focus 启动),bsk click 会静默失效——返回成功、还打印一个完全正确的点击坐标,但页面收不到任何输入事件(pointerdown / mousedown / mouseup / click 一个都没触发)。把窗口置于前台后,同样的目标、同样的坐标就能点中。

对照证据:同一页面、同一目标 #top、同一坐标 (138, 39.33);后台窗口下页面事件日志为空,前台窗口下 status: clicked TOP。

期望:点击前自动激活 Agent Window;若确实无法投递输入,请报错并非零退出,而不是返回成功。现在这种"报成功却什么都没发生"的行为,会让 agent 在填表/操作 UI 时静默把任务做错——我们填 12 道选择题时就遇到:12 次点击全部报成功,实际有几道根本没选上。

Activity

  1. iuyo5678 commented on Sep 17, 2026

    @iuyo5678
    Collaborator

    Fixed in 0.3.0. Background clicks now prepare the page for input without bringing the window to the front, and readiness failures return an error. Please update the CLI and extension. Thanks for the clear reproduction!

  2. drakeo338 commented on Sep 18, 2026

    @drakeo338

    I went looking at this to fix it and I think it's already fixed on main — worth
    checking before anyone spends time on it.

    You're on 0.2.1. 0.3.0 shipped two days after you filed this, and its changelog
    has "Extension: background input filling, automatic append behavior, and
    verification of field values before reporting success" under Fixed.

    The mechanism lines up with what you saw. On main, handleClick in
    apps/extension/src/tools/interaction.ts routes through withInputReady
    (apps/extension/src/tools/input-readiness.ts). That reads
    document.visibilityState first, and when the target is hidden it turns on
    Emulation.setFocusEmulationEnabled and forces a compositor frame before the
    Input.dispatchMouseEvent goes out, then turns focus emulation off again in a
    finally. So a backgrounded Agent Window should now receive the click without
    the window being activated — which is the behaviour you wanted, rather than
    either of the two options you suggested.

    The reason it silently succeeded on 0.2.1 is that the dispatch was acknowledged
    by CDP even though the renderer never processed it, so there was nothing to
    report as a failure. The readiness wrapper also carries an effect_state on its
    errors now, which is what distinguishes "not delivered" from "outcome unknown"
    in the cases where it still can't confirm.

    Could you retest on 0.3.0? If it still happens there, that's a different bug
    from the one that got fixed and the repro would be worth having, particularly
    the devicePixelRatio 1.5 detail — most of the background-input handling doesn't
    touch scaling, so if it only reproduces at non-integer DPR that narrows it a
    lot.

    I haven't been able to verify this end to end myself: I don't have Windows or a
    browser in the environment I was looking at this in, so this is from reading the
    code and the changelog rather than from reproducing it.

  3. lymerin commented on Sep 21, 2026

    @lymerin
    Contributor

    I went looking at this to fix it and I think it's already fixed on main — worth checking before anyone spends time on it.

    You're on 0.2.1. 0.3.0 shipped two days after you filed this, and its changelog has "Extension: background input filling, automatic append behavior, and verification of field values before reporting success" under Fixed.

    The mechanism lines up with what you saw. On main, handleClick in apps/extension/src/tools/interaction.ts routes through withInputReady (apps/extension/src/tools/input-readiness.ts). That reads document.visibilityState first, and when the target is hidden it turns on Emulation.setFocusEmulationEnabled and forces a compositor frame before the Input.dispatchMouseEvent goes out, then turns focus emulation off again in a finally. So a backgrounded Agent Window should now receive the click without the window being activated — which is the behaviour you wanted, rather than either of the two options you suggested.

    The reason it silently succeeded on 0.2.1 is that the dispatch was acknowledged by CDP even though the renderer never processed it, so there was nothing to report as a failure. The readiness wrapper also carries an effect_state on its errors now, which is what distinguishes "not delivered" from "outcome unknown" in the cases where it still can't confirm.

    Could you retest on 0.3.0? If it still happens there, that's a different bug from the one that got fixed and the repro would be worth having, particularly the devicePixelRatio 1.5 detail — most of the background-input handling doesn't touch scaling, so if it only reproduces at non-integer DPR that narrows it a lot.

    I haven't been able to verify this end to end myself: I don't have Windows or a browser in the environment I was looking at this in, so this is from reading the code and the changelog rather than from reproducing it.

    Adding a more specific follow-up finding to the discussion.

    The original 0.2.1 reproduction was fixed in 0.3.0. The issue below is a separate integration problem introduced by the persistent background-execution mechanism added afterwards.

    On the current main branch, a controlled page tool is prepared by the dispatcher through prepareBackgroundExecution, which establishes a persistent focus/visibility policy before the handler runs.

    The handler then enters withInputReady, but that layer cannot determine whether the current session already owns the persistent policy. If document.visibilityState is still reported as hidden, withInputReady performs its temporary sequence:

    1. explicitly enable Emulation.setFocusEmulationEnabled;
    2. dispatch the input;
    3. explicitly disable Emulation.setFocusEmulationEnabled in finally.

    Those explicit toggles do not update BackgroundExecution's ownership/applied state. The persistent state can therefore become inconsistent:

    • BackgroundExecution still records the persistent policy as enabled;
    • the renderer has just been explicitly disabled by the temporary cleanup;
    • a later ensureAttached → synchronize sees the cached policy as already applied and does not re-enable it;
    • tools that rely on the persistent background policy but do not use withInputReady can then run without the expected background-input override.

    This is a state-consistency bug, not a reappearance of the original 0.2.1 click failure. It is also not covered by the current click browser tests: the relevant tests invoke the handler directly and bypass the dispatcher’s prepareBackgroundExecution step, so they cannot observe the lease/handler interaction.

    The four production withInputReady call sites do receive the complete dependency object, and the target-scoped CDP wrapper is not passed into withInputReady; the missing piece is specifically that the readiness layer has no read-only way to query persistent ownership.

    I am preparing a follow-up PR for this post-0.3.0 integration issue, together with dispatcher-level regression coverage. The PR will contain the implementation details and test results.

  4. iuyo5678 commented on Sep 26, 2026

    @iuyo5678
    Collaborator

    Closing: the original 0.2.1 behaviour (background clicks acknowledged by CDP but never delivered) was fixed in 0.3.0 by #261, and the fix is included in 0.3.1.

    The follow-up @lymerin reported above, where the temporary focus-emulation toggles in withInputReady desync from the persistent background-execution lease added in #249, is a separate issue. It is tracked in #311.

    @bailu8345315-tech if you still see silent click failures on 0.3.1, please reopen with bsk --version, the extension version, and the repro steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions