Conversation
Move Paper Intelligence to the cloud using Cloudflare's infrastructure: - **Remote MCP server** via McpAgent (Streamable HTTP transport) - search, get_paper_info, list_papers tools - Bearer token auth - **Search**: grep + RAG + hybrid (faithful port of Python logic) - Workers AI embeddings (@cf/baai/bge-small-en-v1.5) - Vectorize for vector search - D1 for markdown grep + metadata - **Web UI** at root URL - Paper browser with markdown viewer - RAG chat with streaming (Claude + OpenAI) - Model selector, token-based auth - **Container setup** for Marker PDF conversion (ready, pending CF Containers beta) - **Migration script** to upload local papers to cloud Services: D1, R2, Vectorize, Workers AI, Durable Objects
- Auto-parse paper titles from first H1 heading, display instead of filenames - User-settable aliases via PATCH /papers/:id and rename UI - Document outline panel (TOC) from parsed headers with click-to-scroll - PDF/Markdown toggle viewer with PDF served from R2 - Chat moved to right-side vertical panel - Processing status with live polling and animated indicators - D1 schema: added title and alias columns to papers table
- Re-enable Marker Container in wrangler.jsonc (Workers Paid plan activated) - Fix Vectorize deleteByIds batch size: 100, not 1000 - Clean markdown/HTML artifacts from auto-parsed titles - Migration script now uploads both markdown + PDF per paper - Re-uploaded 79 papers with PDFs for viewer toggle
- PDF viewer: fetch via JS with auth header, display via blob URL (fixes iframe 401) - Marker container: load models at import time with gunicorn --preload (eliminates per-request model loading delay on first conversion) - Increase container sleepAfter to 30m to keep models warm between uses
- Deep forest green (#004d3b) accent, warm off-white (#f2f1ed) canvas - Restrained typography with generous spacing and flat surfaces - 4px border radius on buttons/inputs, no shadows - Remove model selector — hardcode Claude Opus 4.6 - Consistent toolbar-btn class for Outline/Chat toggles - Warmer color palette throughout (green status dots, green active states)
- Replace raw HTML string with proper React SPA (Vite + TypeScript) - Tailwind CSS with Strand AI design tokens (deep green, warm off-white) - react-markdown + rehype-raw + remark-gfm for proper markdown rendering - Lucide icons for toolbar actions - Multi-file PDF upload modal with progress indicators - Fuzzy search in paper sidebar - Rename button always visible (not hidden by truncation) - Alias input auto-populated with current title - PDF default view with navpanes=0 (no thumbnail sidebar) - Static assets served via wrangler assets config - Chat hardcoded to Claude Opus 4.6
- cloud/mcp-upload.mjs: local stdio MCP server that reads PDFs from disk and POSTs them to the cloud Worker REST endpoint (multipart form) - Supports multi-file uploads via paths array - Reads API token from env or 1Password - Removed upload_paper from remote MCP server (can't handle local files)
Architecture change: single local stdio MCP server that wraps the cloud REST API, replacing the remote McpAgent on the Worker. Worker changes: - Add POST /search REST endpoint - Remove McpAgent Durable Object (deleted_classes migration v5) - Remove agents + @modelcontextprotocol/sdk Worker dependencies - Delete src/mcp.ts Local MCP server (mcp-server/): - Node.js stdio MCP server, 5 tools: search, list_papers, upload_paper, delete_paper, rename_paper - Reads PDFs from local disk for upload (no base64 size limits) - Resolves paper names by fuzzy matching name/title/alias - Auth via env var or 1Password fallback - Publishable as npm package "paper-intelligence" Cleanup: - Remove cloud/mcp-upload.mjs (superseded) - Remove paper-intelligence-cloud and paper-intelligence-upload MCP registrations
Contributor
Author
|
Superseded by cloud-storage branch approach (rclone R2 sync) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
McpAgent+ Streamable HTTP transport)Services used
D1 (metadata + grep), R2 (PDF/image storage), Vectorize (vector search), Workers AI (bge-small-en-v1.5 embeddings), Durable Objects (McpAgent)
Test plan