kato

Kubernetes troubleshooting via declarative UseCase flows with AI summaries

Turn your team's runbooks into versioned CRDs. kato runs the exact checks you chose, in the order you chose, and lets an LLM do only the last mile — writing up what the evidence means.

Why kato?

"Ask an AI agent to debug my cluster" is a tempting idea and a risky one: a generic agent decides on its own what to inspect, the steps differ every time, and giving it write access to production is a non-starter. kato flips the control.

• The flow is deterministic, not the model. You author the troubleshooting steps as a UseCase CRD. Every run executes the same ordered checks. The LLM never chooses what to look at and never calls the Kubernetes API.
• Read-only by construction. The operator ships with a get/list/watch-only ClusterRole — no writes, no exec, no deletes. The worst it can do is read.
• Auditable. Every execution is persisted as a Run CRD: the inputs, every step's raw output, and the final summary. You can replay exactly what happened.
• Codify tribal knowledge. The senior engineer's "first check the events, then the previous-container logs if it restarted, then the node" becomes a reusable, reviewable, version-controlled object — not a paragraph in a wiki.
• Bring your own model. Any OpenAI-compatible endpoint, including a local Ollama model, so cluster data never has to leave your network.
• Cheap and safe on tokens. A per-step summaryFilter controls exactly which fields reach the LLM, so the model sees a curated digest — not your whole cluster.

How it works

You define a troubleshooting journey as a CRD — an ordered list of predefined checks. Calling the use case executes that flow deterministically; an LLM is used only to summarize the collected evidence.

Three CRDs:

• UseCase is the flow: inputs, ordered steps, when conditions, forEach fan-out, per-step summaryFilter, and the summary prompt.
• ModelConfig is the LLM backend (OpenAI-compatible). UseCases pick one via summary.modelConfigRef or fall back to the default.
• Run is the audit record of each execution: inputs, per-step outputs, and the summary. Create one with kubectl/GitOps, or via the REST API.

A real use case

apiVersion: kato.zufardhiyaulhaq.com/v1alpha1
kind: UseCase
metadata:
  name: pod-crashloop
spec:
  description: "Diagnose why a pod is crash looping"
  inputs:
    - name: namespace
      required: true
    - name: pod
      required: true
  steps:
    - name: status
      method: check_pod_status
      with:
        namespace: $(inputs.namespace)
        name: $(inputs.pod)
    - name: events
      method: check_events
      with:
        namespace: $(inputs.namespace)
        involvedObject: $(inputs.pod)
    - name: previous-logs
      method: check_pod_logs
      when: $(steps.status.restartCount) > 0      # only if it actually restarted
      with:
        namespace: $(inputs.namespace)
        name: $(inputs.pod)
        previous: "true"
        tailLines: "100"
    - name: node
      method: describe_node
      when: $(steps.status.nodeName) != ""
      with:
        name: $(steps.status.nodeName)
  summary:
    prompt: |
      You are a Kubernetes SRE. Based on the evidence, explain why this pod is
      crash looping and suggest a fix.

Run it:

curl -s -X POST localhost:8080/api/v1/usecases/pod-crashloop/run \
  -d '{"inputs":{"namespace":"payments","pod":"payment-api-xyz"}}' | jq

kato runs check_pod_status → check_events → (if it restarted) check_pod_logs → (if scheduled) describe_node, then hands the filtered evidence to the model for a plain-language root cause and fix. Same steps, every time.

Built-in methods

kato ships 35 read-only checks you compose into flows — pods, workloads, nodes, networking, storage, batch, config, the control plane, plus listing/fan-out and active network probes:

Area	Methods
Pods	check_pod_status, check_pod_logs, describe_pod, check_pod_resources, check_pod_usage
Workloads	check_deployment_status, describe_deployment, check_replicaset, check_daemonset_status, describe_daemonset, check_statefulset_status, describe_statefulset, check_hpa, check_pdb
Nodes	check_node_status, describe_node
Networking	check_service_endpoints, describe_service, check_ingress
Storage	check_pvc
Batch	check_job, check_cronjob
Config & events	check_configmap, check_events
Control plane	check_apiserver
Listing / fan-out	list_pods, list_failing_pods, list_nodes, list_node_pods (drive forEach across a workload's or node's pods, or the node fleet)
Active probes	probe_tcp, probe_http, probe_dns, probe_traceroute, probe_grpc, probe_tls (run from kato's pod; reachability governed by NetworkPolicy)

Full reference and every output field: docs/METHOD.md. Ready-made UseCases (general pod, deployment & node troubleshooting, cluster DNS, the Terway CNI, the istio-ingressgateway, control-plane health, and TCP/HTTP/gRPC connectivity checks) live under examples/.

API

Endpoint	Purpose
GET /api/v1/usecases	list use cases
GET /api/v1/usecases/{name}	one use case's contract
POST /api/v1/usecases/{name}/run	execute ({"inputs":{...}})
GET /api/v1/methods	built-in methods + their output fields
GET /api/v1/runs/{name}	a past run

Quickstart

helm install kato charts/kato -n kato --create-namespace \
  --set modelConfig.enabled=true \
  --set modelConfig.apiKey=$OPENAI_API_KEY

kubectl apply -f examples/usecases/pod-troubleshooting.yaml

kubectl -n kato port-forward svc/kato 8080:8080 &
curl -s -X POST localhost:8080/api/v1/usecases/pod-troubleshooting/run \
  -d '{"inputs":{"namespace":"payments","pod":"payment-api-xyz"}}' | jq

No OpenAI account? Point ModelConfig at a local Ollama model instead — see examples/modelconfig/.

Installing

To install the chart with the release name my-release:

helm repo add kato https://zufardhiyaulhaq.com/kato/charts/releases/
helm install my-kato kato/kato --values values.yaml

Values

Key	Type	Default	Description
config.gcInterval	string	"1h"
config.maxConcurrent	int	10
config.methodMaxConcurrent	int	10
config.runMaxDuration	string	"1h"
config.runReconcileConcurrency	int	2
config.runTTL	string	"168h"
config.stepTimeout	string	"30s"
image.pullPolicy	string	"IfNotPresent"
image.repository	string	"ghcr.io/zufardhiyaulhaq/kato"
image.tag	string	"0.8.0"
modelConfig.apiKey	string	""
modelConfig.baseURL	string	"https://api.openai.com/v1"
modelConfig.default	bool	true
modelConfig.enabled	bool	false
modelConfig.maxTokens	int	2048
modelConfig.model	string	"gpt-4o-mini"
modelConfig.name	string	"default"
modelConfig.temperature	string	"0"
podAnnotations	object	{}
podLabels	object	{}
replicaCount	int	1
resources.limits.cpu	string	"500m"
resources.limits.memory	string	"256Mi"
resources.requests.cpu	string	"50m"
resources.requests.memory	string	"64Mi"
service.port	int	8080
service.type	string	"ClusterIP"

see example files here

---

Autogenerated from chart metadata using helm-docs v1.14.2