NetOps Copilot

An LLM agent that answers infrastructure and network-operations questions in natural language, grounded in your own runbooks, device state, and incident history — instead of hallucinating a plausible-sounding answer.

Built by a network/NOC engineer, for network/NOC engineers: the goal is to close the gap between "the answer exists in a runbook or a senior engineer's head" and "the on-call engineer can actually get it at 3am."

Why this exists

Most LLM demos answer general questions well and infra-specific questions badly, because the model has never seen your topology, your runbooks, or your past incidents. NetOps Copilot fixes that with retrieval-augmented generation (RAG) over your own operational documents, plus a small set of read-only tools the agent can call to check live state.

Architecture

                        ┌─────────────────────┐
   User question  ─────▶│   Agent Orchestrator │
                        │  (planning + tool    │
                        │   selection loop)    │
                        └─────────┬────────────┘
                                  │
              ┌───────────────────┼───────────────────┐
              ▼                   ▼                   ▼
     ┌────────────────┐  ┌────────────────┐  ┌────────────────┐
     │ Retrieval (RAG)  │  │  Read-only Tools │  │  LLM Backend    │
     │ runbooks, past   │  │  device lookup,  │  │  provider-      │
     │ incidents, KB    │  │  ping/traceroute,│  │  agnostic       │
     │ (vector store)   │  │  log grep        │  │  (see below)    │
     └────────────────┘  └────────────────┘  └────────────────┘
                                  │
                                  ▼
                        ┌─────────────────────┐
                        │  Grounded answer +   │
                        │  cited sources       │
                        └─────────────────────┘

The LLM backend is provider-agnostic on purpose (src/netops_copilot/agent/llm_client.py) — it currently supports Anthropic and OpenAI-compatible APIs behind one interface, so swapping or adding a model provider is a config change, not a rewrite. This mirrors the kind of "system-on-system" flexibility needed when a team standardizes on one LLM provider today and another tomorrow.

Core capabilities

• Retrieval-augmented answers — indexes runbooks/docs into a local vector store (Chroma) and cites the source document for every claim.
• Read-only tool use — the agent can call a small set of safe, read-only tools (device facts lookup, log search) rather than only generating text; every tool call is logged.
• Evaluation harness — a scored eval set (src/netops_copilot/eval/) of realistic ops questions with expected-answer criteria, so prompt/model changes can be measured instead of eyeballed.
• Guardrails by design — no write/change actions are exposed to the agent in this repo; it is read-only by construction, not by prompt instruction alone.

Quickstart

git clone https://github.com/Chiru6006/netops-copilot.git
cd netops-copilot
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Index the sample runbooks into the local vector store
python -m netops_copilot.retrieval.index_runbooks docs/runbooks

# Ask a question
python -m netops_copilot.agent.cli "Why would BGP flap on a redundant uplink?"

Set ANTHROPIC_API_KEY or OPENAI_API_KEY depending on which backend you configure in config.yaml.

Project layout

src/netops_copilot/
  agent/        orchestration loop, LLM client abstraction, CLI
  retrieval/    document loading, chunking, embedding, vector store
  tools/        read-only tool implementations the agent can call
  eval/         scored evaluation set + runner
prompts/        versioned system/tool prompts
docs/runbooks/  sample runbook corpus used for retrieval
tests/          unit + integration tests

Status

Actively developed. See ROADMAP.md for what's built vs. planned.

License

MIT — see LICENSE.