Repository navigation
Wasmtime 48 - #162
Merged
Merged
Wasmtime 48#162
Conversation
…server Remove the 'metrics' cargo feature and the prometheus/lazy_static dependencies from the public fastedge-lib crates: - runtime::util::metrics is now a backend-agnostic hook: embedders install a process-global sink via set_sink(); with none installed (fastedge-run CLI) reporting is a no-op - runtime::instances keeps live/peak counts in plain atomics; flush_peak() now returns the drained peak instead of setting a gauge - AppResult variants and http-service call sites are no longer feature-gated The Prometheus implementation (fastedge_call_count, fastedge_error_total_count, fastedge_request_duration, fastedge_wasm_memory_used, fastedge_wasm_instances_live/peak) moves to server/src/metrics.rs; the sink is installed at startup and the instance gauges are set on each scrape (same semantics). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ve consistency in `runtime::instances`
Expose the four missing key-value read metrics on the Prometheus scrape:
- fastedge_kv_reads_total{result="miss"|"hit"} splits reads into direct
(backing-store) and in-process cache hits. Commands that are never
cached (scan/zscan/zrange_by_score/bf_exists) count as misses.
- fastedge_redis_read_retries_total{store,command} counts retried read
attempts only, so rate() gives retries since the previous scrape.
- fastedge_redis_read_duration_seconds{store,command} times uncached
reads; the average since the previous scrape is rate(_sum)/rate(_count).
Cache hits are excluded (in-process lookup) and failed reads are not
observed, since their timeout-dominated duration would skew the
latency of served reads.
The key-value store lives in the fastedge-lib workspace and must stay
free of a metrics backend, so it reports retries through a new
set_read_retry_observer hook that the server installs at startup.
All new series are materialised at 0 by redis_metrics::init() and are
documented with PromQL in docs/METRICS.md.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the guest trapped before invoking `response-outparam::set`, the
error was rebuilt with `bail!("...: {e:?}")`, which flattens the cause
into a string and drops the source chain. `map_err`/`fail_reason_of`
classify errors by downcasting `root_cause()` to `wasmtime::Trap` or
`Elapsed`, so every such failure fell through to the generic branch and
returned 530/INTERNAL_STATUS_EXECUTE_ERROR instead of 532/3010 - even
though the stats row was correctly recorded as TIMEOUT.
Use `.context()` to attach the message while keeping the chain intact.
Also stop logging the same backtrace twice: the spawned task logged
"incoming handler" and the error was then logged again as "execute"
after propagating up. The task-side log now only fires when the error
cannot propagate (response headers already flushed), which was the only
case it was needed for.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lls_total` Replace `fastedge_call_count` and `fastedge_error_total_count` with a single `fastedge_calls_total` metric that tracks all app call outcomes under an `outcome` label. Add support for the new `SUSPENDED` app result. Update HTTP service to emit metrics for requests with `DISABLED`, `RATE_LIMITED`, and `SUSPENDED` statuses. Add unit test coverage to ensure correct metric allocation across outcomes.
…backend integration
- Added `fastedge_kv_cache_bytes`, `_entries`, and `_max_bytes` metrics for tracking in-process key-value store cache occupancy. - Implemented lifecycle management to dynamically remove metrics for idle/disconnected stores. - Updated configuration to accurately define default cache sizes in bytes and ensured compatibility with Prometheus scraping. - Enhanced test coverage for cache behavior, including capacity, eviction, oversized values, and label sanitization.
…_id and metric_label
…ations - Enhanced error classification for Redis read failures with clearer differentiation between transient and permanent errors. - Introduced `retry_backoff`, `is_guest_fault`, and `read_error` helpers to streamline retry logic. - Optimized retry mechanism with capped backoff for `WAIT_AND_RETRY`. - Improved guest-facing error messages for deterministic errors. - Updated unit tests to cover new error handling logic comprehensively. - Integrated `read_status_of` in ProxyWasm host to map errors to appropriate status codes.
- Updated default cache TTL from 300 seconds to 60 seconds to enable quicker expiration.
…andling Add `epoch_grace` module to refine epoch timeout decisions by distinguishing kernel-parked threads from CPU-exhausted guests. Track grace outcomes with the new `fastedge_epoch_stall_grace_total` metric. Update store functionality to utilize CPU-aware deadlines, enhancing reliability under memory pressure.
…upport Enhance error classification during app initialization (`_initialize`, `on_context_create`) to distinguish between initialization-phase timeouts and request-phase timeouts. Introduce new `CallPhase` enum to provide phase-specific context for metrics. Update metrics labels to differentiate `init_timeout` from standard `timeout`.
Implement `rebase` functionality to redefine baseline for credited deadline extensions, enhancing CPU usage accuracy while maintaining stall grant allowances. Introduce and refine supporting tests for the new mechanism.
Remove the `epoch_grace` module and its related `fastedge_epoch_stall_grace_total` metric. Eliminate CPU-aware grace handling and its integration in the runtime. Update tests, documentation, and metrics initialization accordingly.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Client-spoofable stats exclusion and several incomplete accounting and logging paths must be corrected.
8 open findings
What changed in this PR
Upgrades the runtime to Wasmtime 48 while expanding request observability, metrics hooks, cache accounting, and Redis resilience.
Changes:
- Migrates WASI and HTTP integrations to Wasmtime 48 APIs.
- Adds trace context, access logging, and backend-neutral metrics.
- Introduces Redis read retries and cache-hit tracking APIs.
| File | Description |
|---|---|
src/key_value.rs |
Updates stats stub. |
src/context.rs |
Adapts runtime context interfaces. |
crates/runtime/src/util/stats.rs |
Adds stats-record discard support. |
crates/runtime/src/util/mod.rs |
Makes metrics always available. |
crates/runtime/src/util/metrics.rs |
Replaces Prometheus coupling with a sink. |
crates/runtime/src/trace.rs |
Generates W3C trace identifiers. |
crates/runtime/src/store.rs |
Adapts store construction and errors. |
crates/runtime/src/registry.rs |
Corrects registry capacity documentation. |
crates/runtime/src/logger.rs |
Removes obsolete WASI file support. |
crates/runtime/src/limiter.rs |
Uses Wasmtime error types. |
crates/runtime/src/lib.rs |
Migrates WASI-HTTP hooks and outbound logging. |
crates/runtime/src/instances.rs |
Decouples instance counters from Prometheus. |
crates/runtime/src/app.rs |
Changes KV cache capacity to bytes. |
crates/runtime/src/access_log.rs |
Defines system access-log records and sink. |
crates/runtime/Cargo.toml |
Updates runtime dependencies and features. |
crates/key-value-store/src/redis_impl.rs |
Adds Redis read retries and classification. |
crates/key-value-store/src/lib.rs |
Adds cache-origin and retry-observer APIs. |
crates/key-value-store/Cargo.toml |
Enables Tokio for Redis retries. |
crates/http-service/src/state.rs |
Exposes outbound log handles. |
crates/http-service/src/lib.rs |
Adds request logging, tracing, and stats filtering. |
crates/http-service/src/executor/wasi_http.rs |
Migrates the WASI-HTTP executor. |
crates/http-service/src/executor/mod.rs |
Updates Wasmtime body imports. |
crates/http-service/src/executor/http.rs |
Adapts HTTP execution APIs. |
crates/http-service/Cargo.toml |
Removes obsolete dependencies and features. |
crates/http-backend/src/lib.rs |
Logs outbound requests. |
crates/http-backend/src/access_log.rs |
Defines the outbound logging seam. |
crates/cache/src/lib.rs |
Exposes reusable cache operations. |
CLAUDE.md |
Documents Wasmtime 48 and metrics architecture. |
Cargo.toml |
Pins the Wasmtime 48 fork. |
Cargo.lock |
Refreshes the dependency graph. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Port the access-log work from feat/integration-with-log-sender onto current main (that branch is ~210 commits behind and carries the FastEdge-lib subtree at lib/ instead of fastedge-lib/). Emit exactly one record per processed request — for every outcome, including early rejections — describing how the edge server handled it: traceparent, final status, X-CDN-Internal-Status, app/client ids, method, uri and execution time. Transport is abstracted behind `runtime::access_log::AccessLogSender`; the server implements it as best-effort RFC 3164 syslog datagrams over UDP to the trace stream (default 127.0.0.1:9514, `--access-log-addr` / `--access-log-role`). Init failure falls back to the no-op sink. Covered paths: - ProxyWasm (v1/v2/v2a/v2b/v2c/v2d/v3), one record per lifecycle callback, tagged with the trigger and the initiating service. - Preview2 http-handler and standard wasi:http. - Outbound requests (`proxy_http_call` and the WASI/http-backend fetch client) as their own `fetch` records, correlated by traceparent via a dependency-free `ExtRequestLog` seam in http-backend. Requests without an inbound `traceparent` get a freshly generated W3C id (`runtime::trace::new_traceparent`) instead of a nanoid, and the resolved id is pinned onto the request so outbound records join the inbound one. `traceparent` is added to the default propagate-headers set. nanoid is dropped from proxywasm and http-service. Deviations from the source branch, for current main: - Initiator/version mapping extended to V2d (core-proxy) and V3 (nginx), matching the connection-gauge classification. - ProxyWasm record duration reads the per-call `Instant` already used by the metrics path (ungated from the `metrics` feature) rather than the stats row, whose `get_time_elapsed()` is a request-level accumulator and would over-report on streaming body callbacks. - The cdn-config/LMDB commit on that branch is a separate feature and is not included.
- Upgraded to Wasmtime 48.0.0, adjusting dependencies and addressing breaking changes. - Refactored `WasiHttpHooks` implementation to align with Wasmtime 48's new `HttpHooks` design. - Streamlined epoch-related bookkeeping for HTTP requests and improved resource table handling. - Updated `Cargo.lock` with new versions and removed unused or outdated dependencies.
- Introduced `EpochRefund`, `ResponseBodyEpochRefund`, and `RequestBodyEpochRefund` to improve epoch pause accounting for network wait times during HTTP request and response processing. - Enhanced `WasiHttpHooks` to ensure accurate epoch refunds for guest blocking on network I/O. - Updated `Cargo.toml` and `Cargo.lock` to include `http-body` dependency.
…gement - Updated `update-preview1.sh` for improved Wasmtime version tracking and script reliability. - Updated `Cargo.lock` to include Wasmtime 48.0.2 and dependency adjustments. - Refactored references to dependencies for consistency and removed outdated sources.
qrdl
approved these changes
Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.