diff --git a/.claude/commands/beelocal-up.md b/.claude/commands/beelocal-up.md new file mode 100644 index 00000000..e624868f --- /dev/null +++ b/.claude/commands/beelocal-up.md @@ -0,0 +1,77 @@ +--- +description: Bring up the beelocal substrate (k3d cluster + local registry + geth-swap) that beekeeper deploys onto +argument-hint: [--with-bee-image] [--bee-repo ] +--- + +Bring up the substrate a local Bee cluster needs: the k3d cluster `bee`, the local +registry, and the `geth-swap` devchain. It stops there — `/cluster-up` deploys the Bee +nodes afterwards. + +`make beelocal` lives in the bee repo. Resolve it once, up front — first match wins: a +`--bee-repo ` argument, then `$BEE_REPO`, then the sibling checkout the README +quick start produces: + +``` +BEE_REPO="${BEE_REPO:-$(git rev-parse --show-toplevel)/../bee}" +``` + +If the result is not a bee checkout, ask for the path instead of guessing. + +Everything here is local-only; never run `ACTION=destroy` — that belongs to +`/cluster-down --destroy`. + +1. **Check what is already up** and pick the cheapest path: + + ``` + k3d cluster list bee + curl -s http://k3d-registry.localhost:5000/v2/_catalog + kubectl get pods -n local -l app.kubernetes.io/name=geth-swap + ``` + + | State | Do | + |---|---| + | cluster running, registry + geth up | nothing — report and point at `/cluster-up` | + | cluster exists, servers stopped | `make beelocal ACTION=start` | + | no cluster | full prepare (step 3) | + +2. **`/etc/hosts`** — beelocal maps `*.localhost` (registry, geth-swap, bee-N, + bootnode-N, light-N) to `127.0.0.1`. The step is idempotent and already satisfied + when `grep -q 'swarm bee' /etc/hosts` matches. If it does not match, STOP: it needs + an interactive sudo. Ask the user to run it themselves from the bee repo (in Claude + Code, prefix with `!`): + + ``` + make beelocal ACTION=add-hosts + ``` + +3. **Prepare** — from `$BEE_REPO`: + + ``` + make beelocal ACTION=prepare SETUP_CONTRACT_IMAGE_TAG=0.9.4 OPTS='skip-local' + ``` + + - `prepare` = check tooling → add-hosts → create k3d cluster + registry → helm + install `geth-swap` (waits for the `setupcontracts` job to reach `Completed`). + - `SETUP_CONTRACT_IMAGE_TAG` pins `ethersphere/bee-localchain`; it must match bee + CI (`.github/workflows/beekeeper.yml`, currently `0.9.4`) or the deployed contract + addresses differ from what the tooling expects. + - `OPTS='skip-local'` skips beelocal's own bee build, which is slow and **not + CI-parity** (no `REACHABILITY_OVERRIDE_PUBLIC`, no `.github/patches`). + `/cluster-up --rebuild-bee` builds the correct image. Pass `--with-bee-image` in + "$ARGUMENTS" only if the user explicitly wants beelocal's build — then use + `OPTS='skip-vet'` and warn that the image is not CI-parity. + - On an already-running cluster `prepare` falls through to that build, which is why + step 1 matters. + +4. **Verify** — substrate only, there are no Bee nodes yet: + - `k3d cluster list bee` servers running; `kubectl get nodes` reaches the API. + - registry catalog answers. + - `kubectl get pods -n local` → `geth-swap-*` Running, setupcontracts `Completed`. + - `curl -s -X POST http://geth-swap.localhost -H 'content-type: application/json' --data '{"jsonrpc":"2.0","id":1,"method":"eth_chainId"}'` → `0x3039`. + +5. **Report** PASS/FAIL per item and the next command: `/cluster-up local-dns + --rebuild-bee` on a fresh substrate, or plain `/cluster-up local-dns` if the + registry already lists `ethersphere/bee`. + +Echo each command before running it and surface the tail of any failure instead of +pushing on. Do not commit anything. diff --git a/.claude/commands/cluster-down.md b/.claude/commands/cluster-down.md new file mode 100644 index 00000000..8ec41401 --- /dev/null +++ b/.claude/commands/cluster-down.md @@ -0,0 +1,34 @@ +--- +description: Tear down the beekeeper Bee cluster, optionally destroying the whole beelocal substrate +argument-hint: [cluster-name] [--destroy] [--bee-repo ] +--- + +Tear down the local Bee test cluster. Cluster name: "$1" if given, otherwise +`local-dns`. This is destructive and local-only. + +Before removing anything, report what is actually running (`k3d cluster list`, +`kubectl get pods -n local`) and say what you are about to remove. Decide the scope +from "$ARGUMENTS": + +1. **Delete the Bee cluster** (default) — from this repo: + + ``` + ./dist/beekeeper delete bee-cluster --cluster-name= + ``` + + This removes the Bee nodes but leaves the k3d cluster, registry and geth-swap + running, so `/cluster-up` can redeploy quickly. + +2. **Destroy the substrate** — only when `--destroy` was passed. After the delete + above (or if the cluster is already gone), from the bee repo — resolved as + `--bee-repo `, else `$BEE_REPO`, else + `$(git rev-parse --show-toplevel)/../bee`: + + ``` + make beelocal OPTS='skip-vet' ACTION=destroy + ``` + + This deletes the k3d cluster and the local registry. The next `/beelocal-up` will + have to prepare everything from scratch. + +Echo each command before running it. Do not touch git. diff --git a/.claude/commands/cluster-up.md b/.claude/commands/cluster-up.md new file mode 100644 index 00000000..ee41c2c1 --- /dev/null +++ b/.claude/commands/cluster-up.md @@ -0,0 +1,91 @@ +--- +description: Deploy a Bee cluster with beekeeper onto an already-running beelocal substrate +argument-hint: [cluster-name] [--rebuild-bee] [--bee-repo ] +--- + +Deploy a local Bee test cluster with beekeeper. Cluster name: "$1" if given, otherwise +`local-dns` (names are defined in `config/local.yaml`). + +Run beekeeper commands from this repo. The bee repo is needed only to build the image. +Resolve it once, up front — first match wins: a `--bee-repo ` argument, then +`$BEE_REPO`, then the sibling checkout the README quick start produces: + +``` +BEE_REPO="${BEE_REPO:-$(git rev-parse --show-toplevel)/../bee}" +``` + +If the result is not a bee checkout, ask for the path instead of guessing. + +The steps are heavy but local-only: check whether each is already satisfied and skip if +so, never tear anything down, and stop and report on failure rather than pushing on. + +1. **Substrate precondition (verify only)** — `k3d cluster list` shows a running + cluster, `curl -s http://k3d-registry.localhost:5000/v2/_catalog` answers, and + `kubectl get nodes` reaches the API server. If not, STOP and run `/beelocal-up` + first. + +2. **Bee image — must be CI-parity.** Build and push only when `--rebuild-bee` was + passed in "$ARGUMENTS", the registry catalog has no `ethersphere/bee` yet, or the + user has bee changes they want tested (`git -C "$BEE_REPO" status --short`). + Otherwise reuse the registry image. From `$BEE_REPO`: + + ``` + patch pkg/api/postage.go .github/patches/postage_api.patch + patch pkg/retrieval/retrieval.go .github/patches/retrieval.patch + make docker-build PLATFORM=linux/$(go env GOARCH) \ + BEE_IMAGE=k3d-registry.localhost:5000/ethersphere/bee:latest \ + REACHABILITY_OVERRIDE_PUBLIC=true BATCHFACTOR_OVERRIDE_PUBLIC=2 + docker push k3d-registry.localhost:5000/ethersphere/bee:latest + patch -R pkg/api/postage.go .github/patches/postage_api.patch + patch -R pkg/retrieval/retrieval.go .github/patches/retrieval.patch + ``` + + Always revert both patches and delete any `*.orig`, even if the build fails — + leaving them applied silently poisons the next bee build. + + All four pieces are what bee CI uses (`.github/workflows/beekeeper.yml`): + - `REACHABILITY_OVERRIDE_PUBLIC=true` (ldflag, defaults to `false`) — without it + AutoNAT never resolves inside k3d, `/topology` reports `reachability: Unknown`, + and the pushsync handler never stores (gate: + `IsReachable() && Proximity >= storageRadius`). Symptoms: direct uploads take + ~30s per chunk, "could not push chunk" / "context deadline exceeded" everywhere, + cross-node downloads fail. Checks look hung rather than failed. + - `BATCHFACTOR_OVERRIDE_PUBLIC=2` — batch depth factor for a small cluster. + - `postage_api.patch` — allows batches with depth < 17. + - `retrieval.patch` — disables multiplexed forwarding, as in CI. + +3. **Beekeeper binary** — `make binary` if `./dist/beekeeper` is missing or older than + recent source changes. + +4. **Deploy**: + + ``` + ./dist/beekeeper create bee-cluster --cluster-name= --log-verbosity=debug + ``` + + `create` funds the nodes over the chain configured by `geth-url`, + `bzz-token-address`, `eth-account` and `wallet-key`. If the active + `~/.beekeeper.yaml` points at a remote chain (e.g. a testnet), override those four + for this one command instead of editing the user's global config — Viper reads env + with prefix `BEEKEEPER_`, `-`→`_`. Do not pass `--config`; the binary's flag parse + rejects it. The local values are in `config/beekeeper-local.yaml`: + + ``` + BEEKEEPER_GETH_URL=http://geth-swap.localhost \ + BEEKEEPER_BZZ_TOKEN_ADDRESS=0x6aab14fe9cccd64a502d23842d916eb5321c26e7 \ + BEEKEEPER_ETH_ACCOUNT=0x62cab2b3b55f341f10348720ca18063cdb779ad5 \ + BEEKEEPER_WALLET_KEY=4663c222787e30c1994b59044aa5045377a6e79193a8ead88293926b535c722d \ + ./dist/beekeeper create bee-cluster --cluster-name= --log-verbosity=debug + ``` + + These are the beelocal devchain defaults (public ci-stake key, chainId `0x3039`) — + local use only. A good run logs `fund options, eth: 0.1, bzz: 100`. + +5. **Verify** — run `/cluster-verify `, then report PASS/FAIL and the next + command, e.g. + `./dist/beekeeper check --cluster-name= --checks=ci-pingpong --log-verbosity=debug`. + Long checks need an explicit `--timeout`: it defaults to 30m and bounds the whole + run, not each check. + +Echo each command before running it and surface the tail of any failing output. Do not +commit anything. diff --git a/.claude/commands/cluster-verify.md b/.claude/commands/cluster-verify.md new file mode 100644 index 00000000..8a1c7e96 --- /dev/null +++ b/.claude/commands/cluster-verify.md @@ -0,0 +1,51 @@ +--- +description: Verify the local beelocal + beekeeper cluster is healthy and ready for checks +argument-hint: [cluster-name] +--- + +Read-only health check of the local k3d substrate and the Bee nodes beekeeper deployed +onto it. Cluster name: "$1" if given, otherwise `local-dns`. Do not modify the cluster. + +Take the namespace, api-scheme and api-domain from the cluster's definition in +`config/local.yaml` rather than assuming; for `local-dns` that is namespace `local` and +`http://.localhost` (port 1633) through the ingress, with node groups +`bootnode-0`, `bee-0..N` and `light-0..M`. + +Work bottom-up and stop drilling only when a layer hard-fails in a way that blocks the +next one (no k3d cluster ⇒ nothing above can pass). + +1. **Tooling** — `kubectl`, `k3d`, `docker`, `curl`, `jq` present, and + `./dist/beekeeper` exists (note `make binary` if missing; do not build unless asked). + +2. **Substrate** — `k3d cluster list` shows all nodes up; the registry + (`curl -s http://k3d-registry.localhost:5000/v2/_catalog`) lists `ethersphere/bee`; + the geth-swap pods are Running. + +3. **Workloads** — `kubectl get pods -n -o wide`: bootnode, every `bee-*` and + `light-*` pod Running and Ready, none in `CrashLoopBackOff`/`Error`/`Pending`. Each + pod's image is the locally-pushed `k3d-registry.localhost:5000/ethersphere/bee:…` — + confirm the tag is the one under test. Skim `kubectl logs -n --tail=50` + of one full node for fatal errors. + +4. **Ingress** — routes exist (`kubectl get ingress,ingressroute -n `) and each + node answers `/health` and `/readiness` through them. + +5. **Per-node baseline** — for every full node, via `http://.localhost`: + - `/addresses` → overlay, + - `/topology` → `connected`, `population`, `depth` (a small cluster should be a near + full mesh; `connected > 0` everywhere), + - `/status` → `storageRadius`, `reserveSize`, `reserveSizeWithinRadius`, + `pullsyncRate`, `committedDepth`, `isReachable`, `beeMode`, + - `/reservestate` → `radius`, `storageRadius`, `commitment`. + + A fresh local cluster normally shows `storageRadius: 0` and a small reserve. This + snapshot is the baseline for any reserve/radius work. + +6. **Report** — one PASS/WARN/FAIL line per layer with its single most useful piece of + evidence, a per-node table + (`node | overlay | connected | storageRadius | reserveSize | pullsyncRate`), and a + verdict: ready for checks, or not ready plus the blocking item. If ready, suggest + the next check, e.g. + `./dist/beekeeper check --cluster-name= --checks=ci-pingpong --log-verbosity=debug`. + +Keep it concise and evidence-driven; quote the actual output behind any FAIL. diff --git a/.claude/skills/change-storage-price/SKILL.md b/.claude/skills/change-storage-price/SKILL.md new file mode 100644 index 00000000..ba583bf0 --- /dev/null +++ b/.claude/skills/change-storage-price/SKILL.md @@ -0,0 +1,95 @@ +--- +name: change-storage-price +description: >- + How to change the Bee storage price (the postage price oracle) on a local + beekeeper/beelocal cluster, so postage batches start expiring and reserve/radius + behavior can be exercised. Uses the storage-incentives repo's hardhat `changeprice` + task (`npx hardhat changeprice --price --network localhost`) against the + PriceOracle contract, with a `cast` fallback. Use whenever the task involves raising + or lowering the chain storage price, activating pricing so batches expire, batch + TTL / expiry on the local chain, the price oracle, or `setPrice`. +--- + +# Changing the Bee storage price on a local cluster + +Goal: set the on-chain storage **price** the local Bee nodes read from the postage +price oracle. The local devchain (`bee-localchain`) boots with `currentPrice = 0`, +which makes every batch's TTL **infinite** (`batchTTL = -1`) — so nothing ever expires. +Raising the price activates pricing and starts the expiry clock; that's the lever for +exercising batch expiry → reserve eviction → commitment/radius changes. + +> Values below are for the beelocal devchain (`bee-localchain:0.9.4`). Verify the +> addresses and the branch still exist before relying on them. + +## The command + +```bash +cd "${STORAGE_INCENTIVES_REPO:-$(git rev-parse --show-toplevel)/../storage-incentives}" +git checkout ci/set-price # branch that carries tasks/changeprice.ts +npm install # first time only +npx hardhat changeprice --price 30000 --network localhost +``` + +It reads and logs `currentPrice()`, calls `setPrice()` on the PriceOracle, waits +for the tx, then logs the new price. Bee picks up the new price at +`/chainstate` `currentPrice` within a few blocks. + +Override the target oracle with `--contract 0x…` (defaults to the cluster oracle below). + +## What the pieces are + +- **Task**: `tasks/changeprice.ts` on storage-incentives branch **`ci/set-price`** + (registered as the `changeprice` hardhat task). +- **`--network localhost`** → `url: http://geth-swap.localhost`, `chainId 12345` + (from `hardhat.config.ts`). This is the beelocal geth-swap RPC exposed via traefik. + (`localcluster` = `http://geth-swap:8545`, the in-cluster form; `localhost` is what + you run from your host.) +- **Signer**: the `localhost` network defines no `accounts`, so hardhat uses geth-swap's + own remote accounts — i.e. the devchain admin **`0x62CAb2b3B55f341F10348720ca18063cDB779ad5`** + (private key `4663c222787e30c1994b59044aa5045377a6e79193a8ead88293926b535c722d` — the + public ci-stake key, safe to hardcode). It holds the role `setPrice` requires. +- **PriceOracle (default target)**: **`0x538E6dE1D876BBCD5667085257bc92F7c808A0F3`** — + `setPrice(uint32)` / `currentPrice()`, wired to the active PostageStamp + (`0x657241f4494A2F15Ba75346E691d753A978C72Df`). Deterministic on + `bee-localchain:0.9.4`. + - Note: bee's configured `price-oracle-address 0x5aFE06…` is a *different, unused* + instance — do **not** target it. + +## `cast` fallback (no Node / hardhat) + +Same effect, straight to the contract: + +```bash +cast send 0x538E6dE1D876BBCD5667085257bc92F7c808A0F3 "setPrice(uint32)" 30000 \ + --private-key 4663c222787e30c1994b59044aa5045377a6e79193a8ead88293926b535c722d \ + --rpc-url http://geth-swap.localhost +``` + +Read the current price: +```bash +cast call 0x538E6dE1D876BBCD5667085257bc92F7c808A0F3 "currentPrice()(uint256)" \ + --rpc-url http://geth-swap.localhost +``` + +## Gotchas (measured) + +- **uint32 overflow — hard cap 4194303.** `setPrice` does `_price << 10` in uint32, so + any `_price > 4194303` silently **wraps** (`setPrice(5000000)` → `805696`). Keep the + price ≤ 4194303; use `4000000` when you want "large". +- **Price 0 ⇒ infinite TTL.** With `currentPrice = 0` every batch (amount 1) has + `batchTTL = -1` and never expires — dilution/expiry is a no-op. Run `changeprice` once + (e.g. `--price 10000`) just to *activate* pricing before expecting any expiry. +- **24h minimum validity once priced.** After pricing is active, the PostageStamp + contract rejects batches whose amount buys < 24h of validity ("insufficient amount for + 24h minimum validity"). So a batch's `postage-ttl` must be ≥ 24h; pushing it to expiry + then needs many dilution/depth increments. +- **PostageStamp `lastPrice()` starts at 0** — the first `setPrice` is also what + activates pricing on the stamp contract. +- **Verify it landed** on a Bee node: `curl http://bee-0.localhost/chainstate` and check + `currentPrice`. + +## Reset + +Price is chain state — it **persists** across cluster restarts on the same devchain. +To lower it back, run `changeprice` again with the smaller value (or `--price 0` to fully +deactivate pricing / restore infinite TTL). diff --git a/.gitignore b/.gitignore index 6f0ed895..0cb314e4 100644 --- a/.gitignore +++ b/.gitignore @@ -24,7 +24,17 @@ _test dist/ .idea/ .vscode/ -.claude/ +# Claude Code: local by default — only the entries un-ignored below are shared +.claude/* +!.claude/skills/ +.claude/skills/* +!.claude/skills/change-storage-price/ +!.claude/commands/ +.claude/commands/* +!.claude/commands/beelocal-up.md +!.claude/commands/cluster-up.md +!.claude/commands/cluster-verify.md +!.claude/commands/cluster-down.md .vs/ .DS_Store tmp/ diff --git a/AGENTS.md b/AGENTS.md index 6b281c28..329cbc62 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,6 +26,27 @@ CI (`.github/workflows/go.yml`) runs `make vet`, `make check-whitespace`, golang Requires Go 1.26. For a full local cluster (K3s/k3d + Geth), follow the **Local Development** quick start in `README.md`. +### Local cluster helpers (`.claude/`) + +`.claude/commands/` holds slash commands for that workflow — `/beelocal-up` (k3d +cluster, registry, geth-swap), `/cluster-up`, `/cluster-verify`, `/cluster-down` — and +`.claude/skills/` holds reference material they lean on. + +They default to the sibling checkouts the quick start produces: + +``` +/beekeeper /bee /storage-incentives +``` + +A sibling repo is resolved first match wins — a `--bee-repo ` argument, then +`$BEE_REPO`, then `$(git rev-parse --show-toplevel)/../bee`. Use the argument for a +one-off checkout, the environment variable when your layout differs permanently +(`STORAGE_INCENTIVES_REPO` likewise). + +Only the commands and skills listed above are shared; everything else under `.claude/` +is gitignored, so local settings and personal skills stay out of the repo. To +contribute a new one, add the file and un-ignore it explicitly in `.gitignore`. + ## How a run is wired together The root command (`cmd/beekeeper/cmd/cmd.go`) builds shared dependencies in `PersistentPreRunE` before any subcommand runs: diff --git a/README.md b/README.md index 38f894dc..f0f31269 100644 --- a/README.md +++ b/README.md @@ -78,10 +78,12 @@ git clone https://github.com/ethersphere/bee cd bee # Install K3s cluster and Geth node -make beelocal ACTION=prepare SETUP_CONTRACT_IMAGE_TAG=0.9.2 OPTS='skip-vet' +make beelocal ACTION=prepare SETUP_CONTRACT_IMAGE_TAG=0.9.4 OPTS='skip-vet' +``` -> **Important:** The `SETUP_CONTRACT_IMAGE_TAG=0.9.2` parameter is required and must match exactly. This ensures compatibility with our CI pipeline and production environment. See our [CI workflow](https://github.com/ethersphere/bee/blob/3a5de30aba477560bfc503632479f4793d68dcef/.github/workflows/beekeeper.yml#L15) for reference. +> **Important:** The `SETUP_CONTRACT_IMAGE_TAG` parameter is required and must match the value Bee's CI uses, currently `0.9.4`. This ensures compatibility with our CI pipeline and production environment. See the [CI workflow](https://github.com/ethersphere/bee/blob/master/.github/workflows/beekeeper.yml) for the current value. +```bash # Deploy Bee nodes locally cd ../beekeeper ./dist/beekeeper create bee-cluster --cluster-name=local-dns @@ -90,6 +92,12 @@ cd ../beekeeper ./dist/beekeeper check --cluster-name=local-dns --checks=ci-pingpong ``` +**AI assistant helpers:** this repo ships slash commands for the workflow above — +`/beelocal-up` (k3d cluster, registry, geth-swap), `/cluster-up`, `/cluster-verify`, +`/cluster-down` — plus reference skills, under `.claude/`. They assume `bee` is a +sibling checkout. See [AGENTS.md](AGENTS.md) for how paths are resolved and how to add +your own. + **Need help?** See the [Bee Deployment Guide](https://github.com/ethersphere/bee-staging) for detailed step-by-step instructions. ## Requirements