Skip to content

About

Sub-150ms Web-to-AST Markdown engine & Model Context Protocol (MCP) server. Eliminates 90%+ raw HTML token bloat for LLMs, RAG, and agent context.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

6 Commits

Folders and files

Repository files navigation

⚡ TokenMarkdown (tokenmarkdown.com)

Sub-150ms Web-to-Markdown Engine & Model Context Protocol (MCP) Server
Engineered for LLM context window efficiency, RAG pipelines, and autonomous AI coding agents.

License: MIT MCP Part of Quite Good Project npm version


🧭 The Problem: Prompt Bloat & Token Degradation

When autonomous agents or RAG pipelines scrape the web using Puppeteer, Jina, or Firecrawl, they dump 25,000+ tokens of navigation trees, cookie popups, tracking scripts, and inline SVGs directly into the prompt.

  • High Cost: $0.08+ per page in Claude 3.5 Sonnet token fees.
  • Slow Latency: 4 to 8 seconds per scrape.
  • Context Poisoning: LLMs hallucinate on irrelevant footer links and cookie consent disclaimers.

TokenMarkdown compresses web pages by 95% into dense, clean Markdown in sub-150ms, preserving your context budget and boosting reasoning accuracy.


⚡ Comparison Benchmark

Feature Raw HTML / Scrapers Firecrawl / Jina TokenMarkdown.com
Average Latency 6,200ms 2,100ms ⚡ 148ms
Tokens Consumed ~28,500 tokens ~3,200 tokens 🎯 ~940 tokens (95% Savings)
Prompt Cost (Sonnet) $0.0855 / call $0.0096 / call 💰 $0.0028 / call
Zero-SDK URL Proxy ❌ No ❌ No ✅ Yes (curl tokenmarkdown.com/https://...)
Native MCP (Stdio + SSE) ❌ No Partial ✅ Yes (npx -y tokenmarkdown-mcp)

🚀 Quickstart

1. Zero-SDK Terminal / cURL Prefix Proxy

You don't even need an API key for quick testing. Prepend https://tokenmarkdown.com/ to any URL:

curl -s "https://tokenmarkdown.com/https://stripe.com/docs/api"

2. Claude Desktop & Cursor IDE Setup (MCP)

Add to your Claude Desktop configuration (claude_desktop_config.json) or Cursor MCP settings.

🎁 Free Sandbox Bonus: Pass --email you@company.com in args to instantly double your free sandbox quota from 50 to 100 requests! Zero credit card required.

{
  "mcpServers": {
    "tokenmarkdown": {
      "command": "npx",
      "args": ["-y", "@quitegoodproject/tokenmarkdown-mcp", "--email", "you@company.com"]
    }
  }
}

Or run via CLI:

npx -y @quitegoodproject/tokenmarkdown-mcp --email you@company.com

3. REST API (POST /v1/extract)

curl -X POST "https://tokenmarkdown.com/v1/extract" \
  -H "Authorization: Bearer <TOKENMARKDOWN_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://linear.app/blog/rethinking-issue-tracking",
    "include_images": false
  }'

Response:

{
  "url": "https://linear.app/blog/rethinking-issue-tracking",
  "title": "Rethinking issue tracking — Linear Blog",
  "markdown": "# Rethinking issue tracking\n\nMost issue trackers are built as databases...",
  "tokens": 820,
  "savings_percent": 96.2,
  "execution_ms": 142
}

📦 1-Click Blueprints for Agencies & Automation


🌐 Server-Side Ingestion Standard: RFC-2602 & AnswerRail

TokenMarkdown is the client-side engine for agents scraping the web. For website owners who want their own sites to be natively ingestible by AI agents without scraping latency:

  • RFC-2602 Specification: Open Inbound AI Ingestion Protocol (AIP) for context-ceiling preservation and deterministic content negotiation (Accept: text/markdown).
  • Empirical SaaS Truncation Benchmark: Research report analyzing why 84% of top SaaS applications are truncated at 25,000 tokens by Perplexity and ChatGPT Search.
  • AnswerRail (Reference Edge Gateway): Deploy RFC-2602 at your Cloudflare Anycast edge in <60s to serve sub-15ms clean Markdown, byte-0 schema tables, and Ed25519 provenance with zero origin code changes.

🏛️ developer suite & Governance

TokenMarkdown is an official developer primitive engineered by The Quite Good Project.


📄 License

MIT © 2026 The Quite Good Project.

About

Sub-150ms Web-to-AST Markdown engine & Model Context Protocol (MCP) server. Eliminates 90%+ raw HTML token bloat for LLMs, RAG, and agent context.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages