A self-hosted agent workspace for your own LLM servers
LangGraph deep agents · MCP tools · multimodal RAG · terminals + fleet SSH · scheduled tasks · local voice · image generation — in a llama.cpp-style web UI

Quick start · Features · Screenshots · Architecture · Full feature notes · Security

Warning

Read Security before running this. The agent gets an unsandboxed shell and full file access on the host, with no approval prompt and no authentication. It's built as a personal tool for a trusted LAN/tailnet — never expose it to the internet.

About

LangBang is what you get if llama.cpp's built-in web UI grew an agent: point it at any OpenAI-compatible server — sglang, vLLM, llama-server, Ollama, OpenAI, OpenRouter — and you get a persistent, multi-thread chat where the model can plan with a todo list, run shell commands, read and edit files, call MCP tools, generate images, read its answers aloud, and keep working on a schedule while you're away. Around the chat sit the tools you'd otherwise open separately: a multimodal semantic index of your files and media, a file editor, browser terminals that SSH into the rest of your fleet, and a phone line.

It's a single Python process (FastAPI + LangGraph) with a no-build vanilla-JS frontend in a dark "terminal HUD" theme. Everything stateful lives in SQLite and plain files under data/. It was built for a home lab of NVIDIA DGX Sparks running Qwen3.8, and every rough edge it hit along the way — long-context blowups, mangled tool calls, stuck scheduled runs — is documented in the feature notes.

Features

Agent

• Deep agent (deepagents) on LangGraph: write_todos planning, sub-agent delegation, file tools, shell, web crawl — with a live to-do panel
• Plan mode + human gates — the agent asks you structured questions (ask_user) and, in plan mode, investigates read-only and waits for you to approve the plan
• MCP servers (stdio / SSE / streamable-HTTP) managed in the UI — test connection, enable/park, import from Claude Desktop configs
• Skills (Agent Skills spec) — the agent writes and improves its own skills, and can read your existing Hermes skill tree
• Memory + learning loop — durable memories across threads, and a post-run review that turns hard-won procedures into skills
• Robust on local models — repairs mangled tool-call args, keeps the final answer honest about its todo list, survives MCP servers dying mid-call

Models & context

• Providers + per-thread override — configure several backends, pick a global default, override provider/model per thread (or per scheduled task)
• Reasoning — thinking on/off, reasoning effort, and optionally keep reasoning in context across tool steps
• Context auto-compaction anchored on the backend's real token counts — long agentic runs fold their history instead of hitting the context wall
• Decision endpoint — POST /v1/systemone answers typed questions (choice / score / noul) about any text or JSON with probabilities, on the TypeSafe Jev wire protocol. A local Laya System 1 model answers in ~0.5 s on CPU; when it's down (or a call asks for it) the chat model answers the same schema, and low-confidence answers can escalate to it — docs/FEATURES.md
• Trajectory view — every model call, tool call, compaction and error with timings and token counts, copyable as text

Workspace

• Scheduled tasks — cron or one-off; runs are server-owned, so they keep going with no tab open. System jobs (the RAG node's nightly rescan) show on the same panel
• ▮ TERM — real terminals in the browser (xterm.js) on tmux sessions the server keeps: close the tab, switch devices or restart LangBang and the shell is still there. A fleet list imports hosts from ~/.ssh/config, probes them (load, memory, disk, GPU), opens SSH tabs and runs one command on many hosts at once
• Semantic search (RAG node) — built-in multimodal EmbeddingGemma 2 index of your files, media, generated outputs, chats, skills and memories; text finds images/video/audio too; federates across a fleet of nodes (fan-out + replication) — docs/RAG.md
• Notifications — in-app toasts, desktop popups, and phone push via ntfy (self-hostable)
• ✎ FILES — browse and edit any file with syntax highlighting, Markdown preview, media viewer, and an AI edit bar scoped to that file; audio opens in a visualizer (six music-sync modes) with its tags/lyrics, and ✨ IMAGES FROM AUDIO illustrates the track from its metadata plus measured tempo, key and energy
• Image generation & editing — Qwen-Image via ComfyUI; results render inline
• Voice — read-aloud with local streaming Pocket TTS (CPU, voice cloning) or gTTS / Google Cloud, with a text normalizer so dates, times, money and tables are spoken correctly; 🎙 speak your messages, or go hands-free with TALK mode (listen → send on a pause → read the answer → listen again)
• Phone assistant (IVR) — /api/ivr/* for a telephony bridge: speech in, speech out, backed by a separate, locked-down agent (its own model and tool allowlist, no shell or file writes) that can search your knowledge base and take messages
• Per-thread tools + features (⚙ TOOLS, /tools) — restrict or force tools per thread, ∅ bare mode (no tools at all, like llama-server), switch memory / auto-learn / skills / auto-search / compaction / ◈ thinking per thread; ⌛ temp threads (/temp, /bare) skip learning, stay out of other threads' hints and are deleted after 24 h idle
• ▤ BENCH — benchmark any provider/model on GPQA Diamond (dataset downloaded at runtime with your HF token, never committed), IFEval (Google's instruction checkers, vendored in server/ifeval/) and Terminal-Bench 2.0 (89 containerized shell tasks via Harbor's terminus-2 agent; uv tool install harbor + Docker) and SWE-bench Verified (500 GitHub issues, mini-swe-agent or terminus-2 via Harbor; Harbor runs can go to any SSH runner host with Docker + Harbor, e.g. an x86 box with disk for the 1-3 GB images); per-answer persistence, resumable runs, leaderboard
• Per-thread prompts, chat search, recap, inline media for any cited file path, and a hidden X-SIM platformer for waiting on slow models

See docs/FEATURES.md for the full reference.

Screenshots

Chat — tool calls fold into cards, answers render as Markdown	Image generation — cited files render inline
Trajectory — every step with timing and tokens	✎ FILES — editor with highlighting and an AI edit bar
Providers — any OpenAI-compatible backend	Per-thread model — override provider, model and reasoning per thread

Quick start

Requirements: Python 3.12+, uv, and an OpenAI-compatible chat endpoint serving a tool-calling model. Optional: ffmpeg (streaming Pocket TTS), a ComfyUI server (image generation), a Hugging Face login with Kyutai's terms accepted (Pocket TTS weights are gated).

git clone https://github.com/TyrelCB/langbang.git
cd langbang
uv sync                                   # installs deps (CPU-only PyTorch for Pocket TTS)

# point it at your backend for the first run (or configure later in the UI)
export LANGBANG_BASE_URL=http://localhost:8000/v1
export LANGBANG_MODEL=your-model-name
export LANGBANG_API_KEY=none              # or a real key for hosted APIs

uv run uvicorn server.main:app --host 127.0.0.1 --port 8123

Open http://localhost:8123, then ⚙ CONFIG → MODEL to add more providers, pick defaults, and enable the features you want. Use --host 0.0.0.0 only on a network you trust (see Security).

Environment variable	Purpose
LANGBANG_BASE_URL, LANGBANG_MODEL, LANGBANG_API_KEY	first-run default provider (afterwards: CONFIG)
LANGBANG_DATA_DIR	where settings, threads, trajectory, uploads and TTS cache live (default ./data)
LANGBANG_CRAWL4AI_URL, LANGBANG_CRAWL_TIMEOUT	crawl4ai service behind the crawl_url tool
LANGBANG_RAG_MCP_URL	optional RAG MCP server registered by default
LANGBANG_HERMES_SKILLS	an existing skill tree to load read-only (default ~/.hermes/skills)
LANGCHAIN_API_KEY + LANGCHAIN_TRACING_V2=true	LangSmith tracing

The built-in defaults (backend, ComfyUI, crawl4ai, MCP URLs) point at the author's lab hosts — override them with the variables above or in CONFIG.

Architecture

flowchart LR
    UI["Browser UI<br/>web/ · vanilla JS"] <-->|"REST + SSE"| API["FastAPI<br/>server/main.py"]
    API --> HUB["Detached run hubs<br/>server/runs.py"]
    SCHED["Scheduler<br/>server/schedule.py"] --> HUB
    HUB --> AGENT["LangGraph deep agent<br/>server/agent.py"]
    AGENT <-->|"OpenAI-compatible"| LLM[("sglang · vLLM · llama-server<br/>Ollama · OpenAI …")]
    AGENT --> TOOLS["Local tools<br/>shell · files · crawl · images · schedules"]
    AGENT <--> MCP["MCP servers"]
    AGENT --- MEM["Skills · memory · learning<br/>server/learning.py"]
    AGENT --> DB[("SQLite<br/>threads · checkpoints · trajectory")]
    API --> VOICE["Voice<br/>Pocket TTS · gTTS · normalizer"]
    TOOLS --> COMFY[("ComfyUI")]
    API <-->|"WebSocket"| TERM["▮ TERM<br/>server/term.py · tmux"]
    TERM -->|"ssh"| FLEET[("fleet hosts")]
    PHONE["telephony bridge"] -->|"/api/ivr"| IVR["IVR agent<br/>server/ivr.py"]
    IVR --> VOICE
    TOOLS <-->|"HTTP"| RAG["RAG node :8900<br/>rag/ · EmbeddingGemma 2"]
    RAG <-->|"fan-out · replication"| PEERS[("fleet peers")]

Loading

Runs belong to the server, not the browser tab: a dropped connection, a reload or a closed laptop never cancels work — only ■ STOP does. Tabs re-attach to a live run and replay what they missed.

Module	Role
server/main.py	FastAPI app: chat/stream, threads, files, voice, images, schedules, terminals, settings
server/agent.py	agent construction, middleware (gates, compaction, todo reconcile, tool-arg repair), streaming runner, thread storage
server/runs.py	detached run hubs with replayable event streams
server/schedule.py	cron + one-off scheduled tasks
server/config.py	settings, providers, per-run resolution
server/learning.py	per-thread prompts, memory, post-run skill/memory review
server/mcp.py	MCP client with per-server timeouts and isolation
server/local_tools.py	shell, files, crawl, image generation, scheduling tools
server/voice.py, server/ttsjobs.py, server/ttsnorm.py	TTS providers, streaming jobs, text normalization
server/comfy.py	ComfyUI workflow builder (Qwen-Image text-to-image + edit)
server/audioart.py	audio metadata, tempo/key/energy analysis, images-from-audio prompts
server/term.py	▮ TERM: tmux-backed terminal sessions, fleet hosts, probes, run-on-many
server/decide.py	/v1/systemone decision endpoint: System 1 backends (Laya) in order, chat-model fallback/escalation
server/ivr.py	phone assistant: STT → restricted agent → TTS, call log
server/notify.py	toasts / desktop / ntfy push
server/rag.py	RAG node client: search tools backend + fire-and-forget ingest pushes
rag/	the RAG node (own service): embedder backends, vector store, indexer, federation, search UI
server/fileassist.py, server/sfxgen.py	FILES AI edit bar; UI sound-effect regeneration
web/	index.html, app.js, style.css, term.js (▮ TERM), viz.js (audio visualizer), game.js (X-SIM), sfx.js, vendored JS, sounds
tools/	A/B harness, sglang parser patch, TTS normalizer checks, demo MCP server

Security

The built-in tools give the agent unsandboxed shell and file access on the machine running the server (run_bash = bash -lc). There is no approval prompt and no login — by design, this is a personal LAN tool.

• Never expose it to the internet. Bind to 127.0.0.1, or to a LAN/tailnet you fully trust. Don't put an auth-bypassing proxy in front of it.
• Anyone who can reach /api/chat, /api/shell (the !cmd route) or ▮ TERM (/api/term, /api/fleet/exec) can run commands as you — and, through ▮ TERM, on every fleet host your SSH keys open. The terminal WebSocket only accepts pages served by LangBang itself (Origin check), since browsers let any site open a WebSocket to your LAN. GET /api/media?path= serves any absolute path for the inline players — the same read access the shell already grants.
• The RAG node (:8900) serves search results, snippets and indexed files (/media) to the LAN without auth, as the standalone search UI did; its write/export routes need loopback or its bearer token.
• /v1/systemone (and /api/decide) has no auth either: any LAN client can spend Laya and chat-model time through it. Laya itself listens on loopback only.
• MCP servers you add run with your privileges too (stdio servers are spawned as local processes).

Docs

• docs/FEATURES.md — full feature reference and design notes
• docs/RAG.md — the RAG node: indexing, search tools, federation, adding fleet nodes
• docs/BACKUP.md — nightly tar incrementals + rsync off-box, restore
• SOUND_DESIGN.md — the UI sound pack and how it's generated
• tools/sglang-patches/ — a parser fix for tag-less single-parameter tool calls
• tools/ab/ — A/B harness (keep-reasoning-in-context experiment)

License & credits

LangBang is licensed under the Apache License 2.0. See NOTICE for attributions and THIRD_PARTY_NOTICES.md for bundled third-party code.

Built on LangChain, LangGraph and deepagents (MIT), served by FastAPI — installed as dependencies, not redistributed. Text-to-speech uses Pocket TTS by Kyutai (code MIT; model weights CC-BY-4.0, gated on Hugging Face and downloaded by each user — not included here). The UI sound effects in web/sounds/ were generated with Stability AI's Stable Audio 3 small-sfx model — Powered by Stability AI. tools/sglang-patches/ contains files derived from SGLang (Apache-2.0). Screenshot content was generated by LangBang itself (Qwen3.8 for the chats, Qwen-Image 2.1 via ComfyUI for the pixel-art image).