logprobs

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

Turn llama.cpp and OpenRouter models into typed choices with probability distributions

An experimental approach for AI search visibility.

Calibrated yes/no, pick-one and scale judgments from your own LLMs, read straight off the logprobs. Runs on vLLM.

Interactive Streamlit app that visualises token-level log probabilities from Azure OpenAI

Token-level entropy trajectories from LLM logprobs. Models can measure their own uncertainty — grounding it in truth requires external verification.

Jev-compatible /v1/systemone server reading typed decisions from LLM logits, benchmarked against TypeSafe's Jev on the same items via JevBench

Minimal demo of calibrated LLM-as-a-judge scores from the single-call token-logprob distribution (Wang et al. 2025; G-Eval).

An interactive client for exploring token-level log probabilities from OpenAI-compatible chat models.

A demo to show confidence probabilities in AI responses

A drop-in confidence gate for LLM agents: turn token logprobs into an accept / verify / ask-a-human routing decision, with honest per-model calibration. Local & open-model first, plus an MCP server.

MSc thesis: mechanistic interpretability of Chain-of-Thought reasoning in LLMs via SAE features and logprob signals

Stream LLM tokens with logprobs, compute per-token Shannon entropy, render a color-coded HTML uncertainty map

LLM token visualizer for decoder risk, logprobs, embeddings, branching, and consistency checks.

Measuring the computational cost of deception in LLMs through logprob analysis

Confidence-gated invoice extraction: per-field scores from constrained decoding, measured for calibration on DocILE

Analyze LLM token log-probabilities: perplexity, per-token confidence, low-confidence span detection, and a terminal heatmap. Zero deps, CLI + library.

Single-pass CES entropy risk scoring for LLM outputs with calibration, provider logprobs, CLI, and agent/RAG guardrails.

Jev-style /v1/systemone API in front of any OpenAI-compatible LLM server (one-token logprob scoring, MIT)

Renders every token a model generated with the tokens it rejected, ghosted above it by probability