diff --git a/docs/PICK.md b/docs/PICK.md index 7610ef6..0c502f2 100644 --- a/docs/PICK.md +++ b/docs/PICK.md @@ -1,16 +1,16 @@ # Pick a tool for a request, or abstain -`plexus pick` reads a plain-language request and returns a probability for each tool in the mesh, scored against what each tool's manifest says it emits and consumes. When no tool matches, or the top probability is under a threshold, it abstains, so the request goes to a person or a larger model in place of a guessed tool. `plexus route` still answers the wiring question, how one tool's output reaches another. +`plexus pick` reads a plain-language request and returns a probability for each tool in the mesh, scored against what each tool's manifest says it emits and consumes. When the best tool scores zero or its probability falls under a threshold, it abstains and hands the request to a person or a larger model. `plexus route` still answers the wiring question, how one tool's output reaches another. ## Bar, set before the run This bar was committed before the picker was written and before its author saw the labelled set. -**Labels.** 100 requests written by a separate labeller who saw only tool names, their capability titles and a one-line purpose, never code, each with one gold tool (`tests/fixtures/plexus_pick_labels.json`). The local routing-accuracy records hold 2 records, too few to use. +**Labels.** 100 requests, each with one gold tool (`tests/fixtures/plexus_pick_labels.json`), written by a separate labeller who saw tool names, their capability titles and a one-line purpose. The local routing-accuracy records hold 2 records, too few to use. **Split.** Items are split in half by a seeded hash of their id. The threshold is chosen on the dev half only: the lowest threshold whose dev abstain rate is at least 10%. All bar numbers come from the test half. -**Baseline.** Plexus had no request picker. The baseline is the majority tool on the dev half, always picked. +**Baseline.** The picker is new to plexus, so the baseline always picks the majority tool on the dev half. - **P1, accuracy.** Top-1 accuracy is at least the baseline's plus 0.20. - **P2, abstain rate.** Under 15% of test items abstain. diff --git a/docs/explainer/index.html b/docs/explainer/index.html index 63cbb7a..498f1ed 100644 --- a/docs/explainer/index.html +++ b/docs/explainer/index.html @@ -171,11 +171,11 @@

plexus

What it does for you

-

MCP tells an agent which tools exist. plexus tells it how their outputs plug into each other's inputs. Each tool ships a small manifest; plexus builds the wiring graph, plans the pipeline that feeds the tool you want, and reports what does not connect. A plan carries a receipt tied to the exact manifests it came from, so a changed tool shows up as drift.

+

MCP tells an agent which tools exist. plexus tells it how their outputs plug into each other's inputs. Each tool ships a small manifest; plexus builds the wiring graph, plans the pipeline that feeds the tool you want, and reports the loose ends. A plan carries a receipt tied to the exact manifests it came from, so a changed tool shows up as drift.

@@ -212,13 +212,13 @@

How it works, one step at a time

  • 02

    Discover the edge

    plexus discover matches capability strings. review consumes what notes emits, so one edge forms: notes to review via notes.summary/1.

    -

    The edge is tagged evidence: declared. plexus copies the producer's module pointer into via but never imports or runs it. The citation is a claim for you to check.

    +

    The edge is tagged evidence: declared. plexus copies the producer's module pointer into via as a citation for you to check, and leaves the module itself unloaded.

    Source: src/plexus/mesh.py; README.md, "Declared, not probed"

  • - 03

    Name what does not connect

    -

    Nothing in the set emits style.guide/1, so it is an unmet input: an external or human input, or a tool that is not here. Nothing consumes review.verdict/1, so it is a terminal output.

    -

    Both come from the same comparison. An unmet input means only that no manifest in this set produces it.

    + 03

    Name the loose ends

    +

    In this set, style.guide/1 has a consumer and zero producers, so it is an unmet input: it comes from a person, an outside source, or a tool you have yet to add. review.verdict/1 has a producer and zero consumers, so it is a terminal output.

    +

    Both come from one comparison of what the manifests in this set emit and consume.

    Source: src/plexus/mesh.py, orphans

  • @@ -245,7 +245,7 @@

    How it works, one step at a time

    Walkthrough

    Install it, run it once, then use the main feature. Each command below is real, and so is its output.

      -
    1. Install

      Install from PyPI. Python 3.11 or newer; discovery runs no tool.

      +
    2. Install

      Install from PyPI. Python 3.11 or newer; discovery reads manifests and leaves the tools idle.

      $ python -m pip install plexus-mesh
    3. First run: is there a route?

      Ask whether one tool's output can reach another's input through the built-in manifests.

      $ plexus route --from gather --to crucible --builtin
      @@ -273,23 +273,23 @@ 

      Walkthrough

  • -

    What it does not do

    +

    Limits

    -

    Source: README.md at d889be9, "Declared, not probed" and "How a tool plugs in"; src/plexus/receipt.py

    +

    Source: README.md at d889be9, the sections on declared edges and on how a tool plugs in; src/plexus/receipt.py

    Check what stuck

    Answer each one in your head before you open it.

    When does an edge from A to B form?

    When B consumes a capability that A emits, directly or through A's consumable_as list.

    -
    What does evidence: declared mean on an edge?

    The edge comes from what the manifests say. plexus copied the producer's module pointer but did not follow or run it.

    -
    style.guide/1 shows up as an unmet input. Does that mean no such tool exists?

    No. It means no manifest in the set plexus was given emits it.

    -
    A tool's manifest changes after you saved a plan. What does verify do?

    It re-derives the receipt from the current manifests, the hashes no longer agree, and it exits 1.

    +
    What does evidence: declared mean on an edge?

    The edge comes from what the manifests say. plexus copied the producer's module pointer as a citation and left the module unloaded.

    +
    style.guide/1 shows up as an unmet input. What does that tell you?

    That the manifests plexus was given consume it and leave it unproduced. A tool outside that set may well produce it.

    +
    A tool's manifest changes after you saved a plan. What does verify do?

    It re-derives the receipt from the current manifests, the hashes differ, and it exits 1.