Skip to content
upupabitPublic

About

Start chatting with open weights LLMs, teams channel brainstorming, create images and much more.

Topics

Resources

Security policy

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation


pAIring

Python FastAPI Self-hosted License Docker SQLAlchemy Pydantic Database Alembic Engine Cloud connectors RAG MCP OpenTelemetry Roles Cluster Tests Ruff


Important

Under active development. pAIring is updated every few days with new features and fixes. Pull often to stay current, and check the GitHub commits for what changed.


pAIring

Private AI ring is a self-hosted, production-ready platform for deploying private LLMs & image generation models across a single node, a team, a department, or an organization — role-based access, load balancing failover across servers, and rock solid UI validations to avoid configuration mistakes. Start chatting with open weights LLMs, create images and much more. Supports every existing public GGUF model out of the box, browsable from Hugging Face or your own private repository. Switch the inference engine with one setting: local llama.cpp or Matricxon, or one of a growing number of cloud connectors. Runs as a normal user (no root needed).

See where every reply spends its time: the live workflow shows each chat moving through pAIring and the Matricxon engine in real time, from one server to clusters of hundreds of nodes.

Find problems without reading logs: Workflow analysis sums up every running Matricxon server on one page and lists what needs attention (unreachable or slow nodes, queued replies, slow steps), each with what to change.

Quick start

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
cp .env.example .env   # then set a real SECRET_KEY — see README.md
.venv/bin/python run.py

Or just run bash scripts/install.sh (Linux/macOS) — same steps with extra validations.

Open http://localhost:8000 and log in with the starter account, which pAIring creates the first time it starts:

Username Password
admin admin

Change that password immediately (Settings → Account → Update password) — whether you're about to invite three friends or roll this out to a whole team. Other people's accounts are created under Admin settings → Users.

Configure it as a system service: scripts/pairing.service is a systemd unit template (Linux) — edit a couple of values and systemctl enable it.

No root needed. pAIring runs as a normal user, and the service template runs it as a dedicated pairing user. Root is only needed once, at setup, to install system packages and the systemd service. scripts/install_on_fresh_server.sh does both, installs pAIring into /opt/pairing (copied there when run from elsewhere; INSTALL_DIR=path for another folder), and runs it as a dedicated pairing user it creates (no password, no sudo; set APP_USER=name for another). scripts/install.sh runs no root step: on Linux it suggests /opt/pairing and creating that user first, and asks before going on otherwise. pAIring starts its local Matricxon as the same user, and starts it again whenever pAIring starts (MATRICXON_AUTOSTART, on by default), so both come back after a restart or a reboot.

Supports MySQL migration and multi-server deployment.

After install, open the UI and configure the essentials to get started:

  1. Go to Settings → Inference engines and choose your inference engine. llama.cpp and Matricxon are supported (Matricxon has many more options — see the Matricxon repository for details). Both can be installed directly from there: llama.cpp from its official prebuilt release (CPU by default, or a Vulkan/CUDA build). By default both use one shared models folder, so a model downloaded under either engine is stored once and is ready for both. Prefer the cloud? Set up a connector on the Connectors page and pick it here instead; switching back is just as quick.
  2. After configuring the engine, go to Settings → Models and download some models. Embedding models are needed for RAG support, an optional reranker model makes RAG answers more accurate, and vision projectors are needed for image understanding in chats and channels.
  3. Once the models are ready, go to Settings → System and set the important parameters (such as the default vision model).
  4. Run a few tests.

Production notes

In a production environment it is crucial to tune the important cache parameters. Matricxon, for example, lets you manage the persistent cache very efficiently. The hardware of a single computer/server (or several working as a cluster) plays a big role in how fast the system runs.

It is also important not to expose more models to users than the system RAM/VRAM can handle. For example, on a single computer with 32GB of RAM, expose only 2-3 models that all fit in memory. In production, models always need to stay in memory.

✨ Features

Advanced model management

  • A model management system that lets you:
    • Search, browse and download models directly from Hugging Face repositories, with advanced search and visual indexing to make choosing the right model simple.
    • Pull recommended default models right away and get started.
    • Manage installed models simply and efficiently, and get started with a few recommended models through a simple pull request.

Behavior tuning

  • Supports all the generation parameters (Temperature, Top-P, Top-K, Context window (tokens), and others).
  • Behavior tuning is per user: each user can change these values, and they affect only their own chats.

Chat

  • Real-time streaming replies, with attachments (images + documents, up to 5 + 1 combined per message).
  • Per-conversation model switching and generation parameters (temperature, top-p/k, context window, repeat penalty, reply length).
  • Stop/cancel a reply mid-stream, smart or simple auto-titling.
  • Each reply shows its speed beside its time: tokens per second and time to first token (token counts with llama.cpp and Matricxon).
  • Export a chat as Markdown from the chat header.

Real multi-user, with shared Channels

  • Every user has private conversations no one else can see or list.
  • Channels for shared, group conversations — a team standup, a student class testing models, whatever fits — with "live" (instant push).
  • Per-channel managers — trusted members who can steer that channel's model/settings and pinned notes, without needing full admin rights over the whole system.
  • Daily limits per user (AI replies and generated tokens in 24 hours), with a default for everyone and per-user overrides under Admin settings → Users. Admins have none.

Notes — persona / rules

  • Reusable, pinnable notes that shape the system prompt per conversation — who the model is and what it must follow — editable right from the chat header.

Skills

  • Task instructions chosen per chat from the 🛠️ icon: summarize, review code, analyze a threat report, write a cover letter...
  • Admins manage a shared library under Admin settings → Skills, with a gallery of 239 ready-made skills from Fabric (MIT License); users can also write private skills of their own.
  • Each channel can have its own skill set (Admin settings → Channels → Skills), changeable at any time.

Knowledge base (RAG)

  • Drop in .txt/.md/.pdf/.docx/.odt/.rtf/.html/.csv/.json and every conversation can automatically search them — a household manual or a team's internal docs, same mechanism either way
  • Private-per-user and shared/global scopes, with per-user storage quotas; answers cite which document they drew on.
  • Files added, edited or removed in the knowledge folder on the server are picked up within a minute, no restart (KNOWLEDGE_WATCH_SECONDS, default 30; 0 turns it off).
  • How it works: each document is split into short passages, and an embedding model turns every passage into a vector once, when it is added. For each question, pAIring finds the passages whose vectors are closest to the question's and adds the best few (4 by default) to the prompt.
  • Reranking (optional, Matricxon): vector search is fast but rough. With a reranker installed (Settings → Models → Reranker models, e.g. Qwen3 Reranker 0.6B), pAIring shortlists 20 passages, the reranker reads the question together with each one and scores how well it answers it, and only the best few go into the prompt: more accurate answers without a longer prompt, at the cost of a short wait. Choose the reranker or switch it off under Settings → System → Reranker; if it is off or fails, retrieval simply keeps the vector order.

Tools (MCP)

  • Admins connect MCP tool servers (streamable HTTP) under Settings → Knowledge → Configure MCP, with encrypted headers and a Test connection button, and can switch MCP off for everyone in Settings → System. Users switch Tools on per chat; the model can then call those tools (up to 5 rounds per reply) and each call is shown under the reply. Works with Matricxon (Qwen, Mistral, Llama 3.x and Gemma 4 templates) and with llama.cpp for models whose chat template supports tools.

Live dashboards

  • A real usage overview (conversations, messages, users, knowledge base).
  • An interactive D3 diagram of the app's own request pipeline, click any step for detail — genuinely useful documentation. With Matricxon it is live: running steps light up, slow ones are flagged, and each reply's data streams along the arrows, from one server to clusters of hundreds of nodes.
  • Workflow analysis: a new way to find problems. Instead of opening each server's logs, one page sums up the last minutes of every running Matricxon server and lists what needs attention, most serious first: unreachable, overloaded or slower-than-usual nodes, a long chat read from the start holding up everyone else on a node, replies waiting for a free model slot, models loading while users wait, slow first tokens, slow app steps and low prompt-cache reuse. Each finding says what to change, and tables show the nodes, models and app steps behind it.
  • Telemetry data via OpenTelemetry (latency, tokens/sec, error rate, currently-loaded models) and live CPU/RAM/disk/GPU monitoring for the host — the kind of visibility that's nice to have and, for a shared/production deployment.

Robust theme support

  • Robust themes schema, covering the whole app
  • Pick one from Settings or the header, with instant live preview.

Security, taken seriously either way

  • bcrypt password hashing, signed session cookies, login rate limiting, timing-safe unknown-username handling.
  • CSRF protection: changes made with your login must come from pAIring's own pages (an origin check, so other apps on the same host can't act as you). CSRF_TRUSTED_ORIGINS adds extra origins if a proxy needs it.
  • API docs (/docs, /redoc, /openapi.json) for admins only.
  • Role-based access (admin/user); disabling an account keeps its history intact rather than deleting anything.
  • SQLite out of the box, MySQL when you need more — switchable live.
  • Outbound HTTP proxy support with encrypted-at-rest credentials.

GPU-aware

  • Live GPUs usage shows up right in the dashboard, alongside CPU/RAM/disk.

One place to switch engines

  • llama.cpp, Matricxon and every cloud connector sit in the same Active engine picker (Settings → Inference engines): change it and the whole app follows, with no reinstall and nothing lost.

Load balancing

  • Point at extra ML engine (llama-server or Matricxon) servers (more GPUs, more machines) and this app spreads requests across them, with automatic failover if one goes down.
  • Can also run multiple copies of itself behind one URL, no reverse proxy needed.
  • Or, if you'd rather test it for enterprise use, use a reverse proxy (Nginx, HAProxy, ...) to balance across a ring of pAIring servers — that works too. All configurable directly from the UI.

Cluster management (Matricxon, BETA)

  • Manage a Matricxon cluster from the Cluster page: several nodes share models and chat caches and agree on membership through built-in Paxos consensus.
  • Add and remove nodes
  • A live, advanced interactive map shows each node's role and status, from a few nodes to hundreds. Designed for hundreds of nodes

Connectors

  • Extend the system with connectors, which add support for more inference engines, such as cloud services that offer GPU and serverless ML. pAIring is intended to run privately or locally without external services and providers (except for downloading models or outer MCP tools), but this is a much appreciated optional extra.
  • Currently supported connectors are Cloudflare Workers AI, DigitalOcean Serverless Inference, OpenRouter, Google Gemini, Groq, xAI Grok, NVIDIA NIM, Cohere, Azure OpenAI, Google Vertex AI, Amazon Bedrock and RunPod Serverless; more connectors will be added soon. All but xAI, Azure, Vertex AI, Bedrock and RunPod can be tried for free.
  • Setting one up takes an API key and a Test connection; then it is one pick away in the Active engine list.
Connector Models Free use Knowledge base
Cloudflare Workers AI Dozens: Gemma, Llama, Qwen, Mistral, GPT-OSS 10,000 neurons a day Yes
DigitalOcean Dozens, including OpenAI and Anthropic Free tokens to start Yes
OpenRouter Hundreds from many providers Free models Yes
Google Gemini Gemini and Gemma Free tier Yes
Groq Fast open models Free tier No
xAI Grok The Grok models No, billed per use No
NVIDIA NIM 100+: Llama, Nemotron, DeepSeek, Qwen Free developer credits Yes
Cohere Command and Embed Trial key Yes
Azure OpenAI Your own deployments: GPT, DeepSeek, Grok, Llama and more No, billed to your Azure subscription Yes
Google Vertex AI Gemini and Model Garden partner models No, billed to your Google Cloud project No
Amazon Bedrock GPT-OSS, Claude, Llama, Mistral, DeepSeek, Qwen and more No, billed to your AWS account No
RunPod Serverless Your own vLLM endpoint No, billed by use No

Setup, limits and privacy notes for each: pAIring connectors.

Guardrails for admin changes

  • Risky settings (switching database, external servers) get tested before they're saved, so a bad edit doesn't take the UI down.
  • If something's out of sync with what's actually running (like a hand-edited config file), you get updated.

Image generation

  • Basic text-to-image on the Images page via stable-diffusion.cpp (plain CPU or GPU), with a progress bar, Cancel and generation history. Image generation isn't our focus right now — it's deliberately basic and we'll improve it later.
  • Install it with one click from Settings → Inference engines → Image Engine (picks a Vulkan GPU or CPU build), build the newest version from source on Linux (needs git, cmake and a C++ compiler; takes a while on an old CPU; the source stays in external/stable-diffusion.cpp/src, so rebuilds only recompile what changed), or point it at remote sd-server hosts.
  • Diffusion models (SD-Turbo built in, more from Hugging Face) are managed under Settings → Models → Diffusion models.

Nothing exotic to run

  • Plain HTML/CSS/JS + Tailwind's browser build — no Node, no build step, no extra services beyond the app itself.

Efficient API — fully documented

  • The web UI is a client of pAIring's own JSON REST API — 90+ endpoints covering chat, channels, documents/RAG, notes, users, stats, telemetry, and every admin setting, so anything the UI can do, your own scripts/automation can do too.
  • Live, interactive docs at /docs (Swagger UI) and /redoc, for admins (log in first)
  • GET /health reports whether this instance can actually serve a chat request right now.

Runs on

  • Linux — All features installed.
  • macOS also works — install llama.cpp (llama-server) or Matricxon yourself and point pAIring at it. Windows isn't supported at this point.
  • x86_64 or arm64, GPU optional — CPU-only works fine, a GPU (NVIDIA or Apple Silicon) just makes replies much faster.
  • No fixed RAM requirement — depends on which model(s) you run; Models settings section shows what actually fits your hardware.



📸 Screenshots

Chat Usage dashboard
Chat Stats overview
Visual chat workflow (D3) System resource monitor
Live chat workflow System dashboard
Workflow analysis Admin skills
Workflow analysis Admin skills
Settings->models Settings->behavior
Settings models Settings behavior
Settings->account Settings->system
Settings account Settings system
Engine config Supported architectures
Engine config Supported architectures
Small Matricxon cluster Big Matricxon cluster
Matricxon cluster with 5 nodes Matricxon cluster with 500 nodes



License

GNU AGPL-3.0 — you're free to run it, modify it, and self-host it, no strings attached, for personal or internal use.

If you try it, a ⭐ is always appreciated — and issues/PRs are genuinely welcome, whoever you're running this for.

About

Start chatting with open weights LLMs, teams channel brainstorming, create images and much more.

Topics

Resources

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages