Feature: dual execution profiles — default HTTP, opt-in stateful Lambda per agent

[danny-avila]

Summary

Allow an operator to run two Code API execution profiles concurrently and route the built-in code tools per agent:

• Default profile: the existing HTTP/libkrun sandbox, with no AWS account, configuration, or runtime dependency.
• Stateful profile: an opt-in AWS Lambda MicroVM deployment using affinity and a stable conversation session hint.

The existing LibreChat Stateful code sessions toggle should select the stateful profile. Agents without that opt-in must continue using the default HTTP profile.

This should be the first post-#16 product follow-up because it creates the routing boundary needed for session-scope controls, an ephemeral escape hatch, environment/image selection, and cost/quota policy.

Current behavior

PR #16 introduced a SandboxBackend seam, but each Code API process still selects one process-wide singleton through CODEAPI_SANDBOX_BACKEND=http|lambda-microvm.

Within a Lambda deployment, runtime_session_hint only selects between:

• a warm stateful Lambda MicroVM, and
• a fresh one-shot Lambda MicroVM.

It never selects the legacy HTTP/libkrun backend.

The Agents/LibreChat stateful toggle currently controls only whether runtime_session_hint is sent. The remote Code API base URL remains global. In mixed multi-agent runs, stateful enablement can also be promoted to run scope rather than remaining scoped to the agent that requested it.

Queued Code API jobs carry runtime-session mode and identity, but no backend/profile target. Therefore HTTP and Lambda workers cannot safely consume the same existing BullMQ queues: selection would be nondeterministic, and an HTTP worker correctly rejects a stateful job it cannot honor.

Desired behavior

agent.stateful_code_sessions != true
  -> default Code API endpoint/profile
  -> HTTP/libkrun worker pool
  -> no AWS operational dependency

agent.stateful_code_sessions == true
  -> stateful Code API endpoint/profile
  -> Lambda MicroVM affinity worker pool
  -> stable conversation-scoped runtime session hint

Code API remains the control-plane protocol in both cases. Lambda MicroVM is a sandbox backend behind one execution profile, not a replacement for Code API.

Recommended first slice

Use two isolated Code API deployments/worker queues and add a trusted, server-configured profile/endpoint selector in Agents + LibreChat:

1. Configure a default remote Code API profile and an optional stateful remote profile.
2. Make the Code API endpoint/transport injectable per agent/run instead of resolving one module-global remote base URL.
3. Derive the selected profile server-side from the admin capability and the agent's stateful_code_sessions setting. Do not accept an arbitrary model- or client-supplied URL.
4. Propagate the selected profile consistently through execute_code, bash_tool, file-authoring/read tools, background code execution, and agent handoffs.
5. Keep queues/worker pools isolated unless Code API later adds an explicit trusted backend target and backend-specific queues.

A future single-front-door hybrid router is possible, but is not required for the first useful version.

Cross-repository work

• ClickHouse/code-interpreter: queue namespace/isolation support, profile-aware observability, and two-profile deployment documentation.
• @librechat/agents: a trusted request/run-scoped Code API endpoint override propagated through every remote code-tool execution path.
• LibreChat: configuration and capability gating, resolution for the agent actually executing the tool, and propagation through prewarm, file operations, background work, and handoffs.

Acceptance criteria

• A normal installation can run the default HTTP/libkrun profile without AWS credentials, AWS resources, or Lambda configuration.
• Operators can configure and run the HTTP and Lambda profiles concurrently.
• Toggle off routes all code-session tools only to the default HTTP profile and sends no runtime-session hint.
• Toggle on routes all code-session tools only to the Lambda profile and sends a stable conversation-scoped hint.
• Selection is genuinely per agent. Two agents with different settings in the same graph/run do not promote one another to a shared execution profile.
• Background executions and handoffs preserve the originating agent's selected profile.
• If the stateful profile is not configured or healthy, the toggle is unavailable or the request fails clearly. It must not claim statefulness while using the default profile.
• A request belonging to an established stateful workspace never silently falls back to HTTP. An explicit switch starts a fresh workspace and communicates that boundary.
• Persisted/harvested artifacts can be supplied after an explicit profile transition; live workspace-only state is documented as non-portable.
• Metrics/logs identify the selected execution profile and actual backend without exposing user-controlled high-cardinality values.
• Cross-repo tests cover toggle-off routing, toggle-on routing, mixed-agent routing, background/handoff propagation, missing-profile failure, and explicit profile transition semantics.
• Deployment documentation explains queue isolation, authentication, health checks, and rollback.

Non-goals for v1

• Automatic migration of processes, installed packages, databases, or unsurfaced files between live workspaces.
• Silent capacity/failure fallback from a stateful session to an HTTP sandbox.
• A general user-selectable image/environment catalog.
• Model-selected execution URLs or arbitrary backend targets.
• Warm-session PTC; programmatic tools need an explicit routing policy, but binding PTC rounds to the warm workspace remains separate work.

Follow-ups enabled by this seam

• Per-invocation ephemeral execution from a stateful agent.
• Conversation/session-scope selection.
• Multiple runtime/image environments.
• Cost, quota, and placement policies.
• Controlled canary rollout of Lambda execution.
• Explicit fallback/reset UX when the stateful profile is unavailable.

Related work

• feat: optional AWS Lambda MicroVM stateful sandbox backend #16 — optional Lambda MicroVM backend and stateful runtime sessions.
• feat: stateful sandbox sessions via toolExecution.sandbox sub-config danny-avila/agents#291 — stateful session hint support in the remote sandbox engine.
• ✨ feat: stateful_code_sessions capability for warm Code API sandbox sessions (experimental) danny-avila/LibreChat#14150 — admin capability and per-agent stateful-session toggle.
• Per-agent scoping was explicitly deferred in LibreChat #14150 review.