diff --git a/docs/PICK.md b/docs/PICK.md index 7610ef6..0c502f2 100644 --- a/docs/PICK.md +++ b/docs/PICK.md @@ -1,16 +1,16 @@ # Pick a tool for a request, or abstain -`plexus pick` reads a plain-language request and returns a probability for each tool in the mesh, scored against what each tool's manifest says it emits and consumes. When no tool matches, or the top probability is under a threshold, it abstains, so the request goes to a person or a larger model in place of a guessed tool. `plexus route` still answers the wiring question, how one tool's output reaches another. +`plexus pick` reads a plain-language request and returns a probability for each tool in the mesh, scored against what each tool's manifest says it emits and consumes. When the best tool scores zero or its probability falls under a threshold, it abstains and hands the request to a person or a larger model. `plexus route` still answers the wiring question, how one tool's output reaches another. ## Bar, set before the run This bar was committed before the picker was written and before its author saw the labelled set. -**Labels.** 100 requests written by a separate labeller who saw only tool names, their capability titles and a one-line purpose, never code, each with one gold tool (`tests/fixtures/plexus_pick_labels.json`). The local routing-accuracy records hold 2 records, too few to use. +**Labels.** 100 requests, each with one gold tool (`tests/fixtures/plexus_pick_labels.json`), written by a separate labeller who saw tool names, their capability titles and a one-line purpose. The local routing-accuracy records hold 2 records, too few to use. **Split.** Items are split in half by a seeded hash of their id. The threshold is chosen on the dev half only: the lowest threshold whose dev abstain rate is at least 10%. All bar numbers come from the test half. -**Baseline.** Plexus had no request picker. The baseline is the majority tool on the dev half, always picked. +**Baseline.** The picker is new to plexus, so the baseline always picks the majority tool on the dev half. - **P1, accuracy.** Top-1 accuracy is at least the baseline's plus 0.20. - **P2, abstain rate.** Under 15% of test items abstain. diff --git a/docs/explainer/index.html b/docs/explainer/index.html index 63cbb7a..498f1ed 100644 --- a/docs/explainer/index.html +++ b/docs/explainer/index.html @@ -171,11 +171,11 @@
MCP tells an agent which tools exist. plexus tells it how their outputs plug into each other's inputs. Each tool ships a small manifest; plexus builds the wiring graph, plans the pipeline that feeds the tool you want, and reports what does not connect. A plan carries a receipt tied to the exact manifests it came from, so a changed tool shows up as drift.
+MCP tells an agent which tools exist. plexus tells it how their outputs plug into each other's inputs. Each tool ships a small manifest; plexus builds the wiring graph, plans the pipeline that feeds the tool you want, and reports the loose ends. A plan carries a receipt tied to the exact manifests it came from, so a changed tool shows up as drift.
plexus plan --goal orders the tools that feed one target and names its sources.plexus verify re-derives a plan's receipt from the current manifests and exits 1 on drift.plexus discover matches capability strings. review consumes what notes emits, so one edge forms: notes to review via notes.summary/1.
The edge is tagged evidence: declared. plexus copies the producer's module pointer into via but never imports or runs it. The citation is a claim for you to check.
The edge is tagged evidence: declared. plexus copies the producer's module pointer into via as a citation for you to check, and leaves the module itself unloaded.
Source: src/plexus/mesh.py; README.md, "Declared, not probed"
Nothing in the set emits style.guide/1, so it is an unmet input: an external or human input, or a tool that is not here. Nothing consumes review.verdict/1, so it is a terminal output.
Both come from the same comparison. An unmet input means only that no manifest in this set produces it.
+ 03In this set, style.guide/1 has a consumer and zero producers, so it is an unmet input: it comes from a person, an outside source, or a tool you have yet to add. review.verdict/1 has a producer and zero consumers, so it is a terminal output.
Both come from one comparison of what the manifests in this set emit and consume.
Source: src/plexus/mesh.py, orphans
Install it, run it once, then use the main feature. Each command below is real, and so is its output.
Install from PyPI. Python 3.11 or newer; discovery runs no tool.
+Install from PyPI. Python 3.11 or newer; discovery reads manifests and leaves the tools idle.
$ python -m pip install plexus-meshAsk whether one tool's output can reach another's input through the built-in manifests.
$ plexus route --from gather --to crucible --builtin
@@ -273,23 +273,23 @@ Walkthrough
- What it does not do
+ Limits
- - Edges come from declarations. Discovery never runs a tool, so an edge says two manifests agree. Whether the tools work together is a separate test.
probe_lane checks that a lane's server is live.
- - An unmet input means no manifest in the set you gave produces it. plexus reasons about nothing outside that set.
- verify checks a saved plan's receipt against one re-derived from the current manifests. Act on the re-derived plan it reports, not on a saved file you have not re-derived.
- - The built-in manifests are kept in plexus. Their presence does not show that each tool publishes its own manifest.
+ - Edges come from declarations, so an edge says two manifests agree. Test that the tools work together separately;
probe_lane checks that a lane's server is live.
+ - plexus reasons over the set of manifests you gave it. An unmet input is unmet within that set.
+ verify checks a saved plan's receipt against one re-derived from the current manifests. Act on the re-derived plan it reports.
+ - The built-in manifests are kept in plexus itself. Each tool can also publish its own.
- Source: README.md at d889be9, "Declared, not probed" and "How a tool plugs in"; src/plexus/receipt.py
+ Source: README.md at d889be9, the sections on declared edges and on how a tool plugs in; src/plexus/receipt.py
Check what stuck
Answer each one in your head before you open it.
When does an edge from A to B form?
When B consumes a capability that A emits, directly or through A's consumable_as list.
- What does evidence: declared mean on an edge?
The edge comes from what the manifests say. plexus copied the producer's module pointer but did not follow or run it.
- style.guide/1 shows up as an unmet input. Does that mean no such tool exists?
No. It means no manifest in the set plexus was given emits it.
- A tool's manifest changes after you saved a plan. What does verify do?
It re-derives the receipt from the current manifests, the hashes no longer agree, and it exits 1.
+ What does evidence: declared mean on an edge?
The edge comes from what the manifests say. plexus copied the producer's module pointer as a citation and left the module unloaded.
+ style.guide/1 shows up as an unmet input. What does that tell you?
That the manifests plexus was given consume it and leave it unproduced. A tool outside that set may well produce it.
+ A tool's manifest changes after you saved a plan. What does verify do?
It re-derives the receipt from the current manifests, the hashes differ, and it exits 1.