feat(agent_manager): add pagination and lazy loading for chat history (#116)

docs/adr/0003-cursor-pagination-and-list-response-envelope.md

@@ -0,0 +1,39 @@
+# ADR 0003 — Cursor pagination and list response envelope
+
+- **Status:** Accepted
+- **Date:** 2026-08-23
+
+## Context
+
+`GET /conversations` previously returned a flat, unbounded list of all conversations for a user. As conversation counts grow per user, loading the complete list in one request causes high latency, database scan overhead, and excessive payload sizes for clients.
+
+Additionally, simple offset pagination (`OFFSET N`) suffers from severe database performance degradation on large offsets and produces inconsistent results (skipped or duplicated rows) if conversations receive messages or are created while a user is paginating.
+
+## Decision
+
+1. **Keyset Cursor Pagination**:
+   - `GET /conversations` uses keyset pagination based on the composite ordering key `(COALESCE(last_message_at, created_at) DESC, session_id DESC)`.
+   - The active timestamp `COALESCE(last_message_at, created_at)` ensures empty conversations without messages sort predictably by their creation time alongside active threads.
+
+2. **Opaque Base64 Cursor Token**:
+   - Cursors are opaque server tokens containing base64-encoded JSON `{"t": "<iso8601_utc_timestamp>", "id": "<session_id>"}`.
+   - The pagination token codec (`encode_cursor`, `decode_cursor`) and exception `InvalidCursorError` live in `agent_manager.domain.pagination` (pure Python domain value objects and utilities with zero framework dependencies).
+   - Malformed or invalid cursor tokens raise `InvalidCursorError`, which is mapped by `as_http_error()` to `HTTP 400 Bad Request` with payload `{ "error_type": "invalid_cursor", "message": "invalid pagination cursor" }`.
+
+3. **Domain Layer Bounds**:
+   - `PageRequest` value object encapsulates pagination parameters (`limit`, `cursor`).
+   - Default page size is 20 (`DEFAULT_PAGE_LIMIT`) and maximum page limit is 100 (`MAX_PAGE_LIMIT`), enforced at domain instantiation time in `PageRequest.__post_init__`.
+
+4. **Database Indexing**:
+   - Migration `0005_add_session_pagination_index.py` and `tables.py` add an expression index `idx_conversation_sessions_user_active_session` on `(user_id, COALESCE(last_message_at, created_at), session_id)` to enable fast index range seeks for keyset pagination.
+
+## Contract changes
+
+- **BREAKING CHANGE**: `GET /conversations` response shape changed from a flat list `[ConversationSummary, ...]` to a paginated envelope object `{ "items": [ConversationSummary, ...], "next_cursor": "..." | null }`.
+- `GET /conversations` accepts optional query parameters `limit` (integer, 1..100) and `cursor` (opaque string).
+
+## Consequences
+
+- Clients fetch subsequent pages using `next_cursor` until `next_cursor` is `null`.
+- Keyset range seeks eliminate `OFFSET` database performance degradation and prevent skipped/duplicated sessions when thread activity changes mid-page.
+- Frontends deduplicate threads by `conversation_id` to handle live thread updates gracefully.

docs/api.mdx

@@ -166,6 +166,36 @@ Response:
 
 Errors: `409` the supplied `session_id` belongs to another user.
 
+### `GET /conversations`
+
+List the calling user's conversations, ordered by most recently active first, using keyset pagination.
+
+**Query Parameters:**
+- `limit` *(integer, optional, default: 20, min: 1, max: 100)* — Maximum number of conversations to return per page.
+- `cursor` *(string, optional)* — Opaque pagination cursor token obtained from `next_cursor` of the previous page.
+
+```bash
+curl "http://localhost:8100/conversations?limit=20" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Response:
+
+```json
+{
+  "items": [
+    {
+      "conversation_id": "0d5a…",
+      "title": "Order status inquiry",
+      "last_message_at": "2026-08-20T14:00:00+00:00"
+    }
+  ],
+  "next_cursor": "eyJ0IjoiMjAyNi0wOC0yMFQxNDowMDowMCswMDowMCIsImlkIjoic2Vzcy0xMiJ9"
+}
+```
+
+When `next_cursor` is `null`, no further pages remain. Cursors are opaque server tokens and must not be constructed manually by clients.
+
 ### `POST /conversations/{id}/messages`
 
 Send a message. Prior history is assembled automatically.

src/agent_manager/api/errors.py

@@ -22,12 +22,17 @@
     ConversationMessageNotFound,
     ConversationNotFound,
     ConversationTokenBudgetExceeded,
+    InvalidCursorError,
 )
 
 BUDGET_EXCEEDED_DETAIL = {
     "error_type": "context_limit_exceeded",
     "message": "This conversation has reached its context limit. Start a new chat to continue.",
 }
+INVALID_CURSOR_DETAIL = {
+    "error_type": "invalid_cursor",
+    "message": "invalid pagination cursor",
+}
 INTERNAL_ERROR_MESSAGE = "Internal server error"
 
 _HTTP_ERRORS: dict[type[Exception], tuple[int, Any]] = {
@@ -38,6 +43,7 @@
     ConversationMessageNotFound: (404, "message not found on active conversation branch"),
     ConversationBranchConflict: (409, "conversation branch changed; reload and try again"),
     ConversationLinkRefused: (403, "a visitor cannot adopt another visitor"),
+    InvalidCursorError: (400, INVALID_CURSOR_DETAIL),
 }
 
 

src/agent_manager/api/routes/conversations.py

@@ -7,7 +7,7 @@
 from collections.abc import AsyncGenerator, AsyncIterator
 from typing import cast
 
-from fastapi import APIRouter
+from fastapi import APIRouter, Query
 from fastapi.responses import StreamingResponse
 
 from agent_engine.runtime.streaming import RunStreamEvent
@@ -23,11 +23,13 @@
     CreateConversationRequest,
     CreateConversationResponse,
     MessageOut,
+    PaginatedConversationsResponse,
     SendMessageRequest,
     SendMessageResponse,
     StreamEventOut,
     TokenBudgetResponse,
 )
+from agent_manager.domain import DEFAULT_PAGE_LIMIT, MAX_PAGE_LIMIT, PageRequest
 
 router = APIRouter()
 logger = logging.getLogger(__name__)
@@ -45,17 +47,25 @@ async def create_conversation(
     return CreateConversationResponse(conversation_id=session_id, session_id=session_id)
 
 
-@router.get("/conversations", response_model=list[ConversationSummary])
-async def list_conversations(service: Service, caller: Caller) -> list[ConversationSummary]:
-    sessions = await service.list_conversations(caller)
-    return [
+@router.get("/conversations", response_model=PaginatedConversationsResponse)
+async def list_conversations(
+    service: Service,
+    caller: Caller,
+    limit: int = Query(default=DEFAULT_PAGE_LIMIT, ge=1, le=MAX_PAGE_LIMIT),
+    cursor: str | None = Query(default=None),
+) -> PaginatedConversationsResponse:
+    with as_http_error():
+        page = PageRequest(limit=limit, cursor=cursor)
+        paginated = await service.list_conversations(caller, page=page)
+    items = [
         ConversationSummary(
             conversation_id=session.session_id,
             title=session.title,
             last_message_at=session.last_message_at,
         )
-        for session in sessions
+        for session in paginated.items
     ]
+    return PaginatedConversationsResponse(items=items, next_cursor=paginated.next_cursor)
 
 
 @router.get("/conversations/{conversation_id}/messages", response_model=list[MessageOut])

src/agent_manager/api/schemas.py

@@ -47,6 +47,11 @@ class ConversationSummary(BaseModel):
     last_message_at: datetime | None = None
 
 
+class PaginatedConversationsResponse(BaseModel):
+    items: list[ConversationSummary]
+    next_cursor: str | None = None
+
+
 class MessageOut(BaseModel):
     message_id: str
     run_id: str | None = None

src/agent_manager/api/static/widget.js

@@ -52163,15 +52163,21 @@ var AgentChatClient = class {
     const data = await response.json();
     return String(data.conversation_id);
   }
-  async listConversations() {
-    const response = await this.request("/conversations");
+  async listConversations(limit = 20, cursor) {
+    const params = new URLSearchParams({ limit: String(limit) });
+    if (cursor) params.set("cursor", cursor);
+    const response = await this.request(`/conversations?${params.toString()}`);
     const data = await response.json();
-    if (!Array.isArray(data)) return [];
-    return data.map((thread) => ({
+    const rawItems = Array.isArray(data.items) ? data.items : [];
+    const items = rawItems.map((thread) => ({
       conversation_id: String(thread.conversation_id),
       title: thread.title ?? null,
       last_message_at: thread.last_message_at ?? null
     }));
+    return {
+      items,
+      next_cursor: data.next_cursor ?? null
+    };
   }
   async getMessages(conversationId) {
     const response = await this.request(`/conversations/${conversationId}/messages`);
@@ -52635,20 +52641,6 @@ function randomId() {
   return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`;
 }
 
-// src/agent_manager/api/static/widget/storage/conversationStorage.ts
-function conversationStorageKey(endpoint) {
-  return `agent-chat:${endpoint}`;
-}
-function getStoredConversationId(endpoint, storage = localStorage) {
-  return storage.getItem(conversationStorageKey(endpoint));
-}
-function setStoredConversationId(endpoint, conversationId, storage = localStorage) {
-  storage.setItem(conversationStorageKey(endpoint), conversationId);
-}
-function removeStoredConversationId(endpoint, storage = localStorage) {
-  storage.removeItem(conversationStorageKey(endpoint));
-}
-
 // src/agent_manager/api/static/widget/react/shadcnAiElements.tsx
 var import_react8 = __toESM(require_react(), 1);
 
@@ -53314,6 +53306,22 @@ function upsertTool(tools, next2) {
 
 // src/agent_manager/api/static/widget/react/useConversation.ts
 var import_react9 = __toESM(require_react(), 1);
+
+// src/agent_manager/api/static/widget/storage/conversationStorage.ts
+function conversationStorageKey(endpoint) {
+  return `agent-chat:${endpoint}`;
+}
+function getStoredConversationId(endpoint, storage = localStorage) {
+  return storage.getItem(conversationStorageKey(endpoint));
+}
+function setStoredConversationId(endpoint, conversationId, storage = localStorage) {
+  storage.setItem(conversationStorageKey(endpoint), conversationId);
+}
+function removeStoredConversationId(endpoint, storage = localStorage) {
+  storage.removeItem(conversationStorageKey(endpoint));
+}
+
+// src/agent_manager/api/static/widget/react/useConversation.ts
 var isUnusableConversation = (error) => error instanceof AgentChatHttpError && (error.status === 404 || error.status === 403);
 function useConversation(client, endpoint, onReplaced) {
   const startConversation = (0, import_react9.useCallback)(async () => {
@@ -53392,7 +53400,10 @@ function useConversation(client, endpoint, onReplaced) {
     },
     [client]
   );
-  const listThreads = (0, import_react9.useCallback)(() => client.listConversations().catch(() => []), [client]);
+  const listThreads = (0, import_react9.useCallback)(
+    (limit, cursor) => client.listConversations(limit, cursor),
+    [client]
+  );
   const switchTo = (0, import_react9.useCallback)(
     (conversationId) => setStoredConversationId(endpoint, conversationId),
     [endpoint]
@@ -53435,6 +53446,8 @@ function useConversation(client, endpoint, onReplaced) {
 
 // src/agent_manager/api/static/widget/react/AgentChatApp.tsx
 var import_jsx_runtime4 = __toESM(require_jsx_runtime(), 1);
+var THREADS_PAGE_SIZE = 20;
+var SCROLL_THRESHOLD_PX = 40;
 var DEFAULT_GREETING = "How can I help you today?";
 var GENERIC_ERROR = "Something went wrong. Please try again.";
 var COPIED_RESET_MS = 2e3;
@@ -53480,7 +53493,13 @@ function AgentChatApp({
   const canSubmit = !isExecutionActive && !budgetExceeded && !approvalBlocksComposer;
   const canStop = isExecutionActive;
   const [threads, setThreads] = (0, import_react10.useState)([]);
+  const [nextCursor, setNextCursor] = (0, import_react10.useState)(null);
+  const [loadingMoreThreads, setLoadingMoreThreads] = (0, import_react10.useState)(false);
+  const [threadsError, setThreadsError] = (0, import_react10.useState)(null);
   const [threadsOpen, setThreadsOpen] = (0, import_react10.useState)(false);
+  const hasMoreThreads = nextCursor !== null;
+  const isLoadingMoreRef = (0, import_react10.useRef)(false);
+  const threadsGenerationRef = (0, import_react10.useRef)(0);
   const launcherRef = (0, import_react10.useRef)(null);
   const inputRef = (0, import_react10.useRef)(null);
   const approvalRequestsRef = (0, import_react10.useRef)(/* @__PURE__ */ new Set());
@@ -53561,9 +53580,54 @@ function AgentChatApp({
     launcherRef.current?.focus({ preventScroll: true });
   }, [inline]);
   const openThreads = (0, import_react10.useCallback)(async () => {
-    setThreads(await conversation.listThreads());
+    threadsGenerationRef.current += 1;
+    const currentGen = threadsGenerationRef.current;
     setThreadsOpen(true);
+    setLoadingMoreThreads(true);
+    setThreadsError(null);
+    isLoadingMoreRef.current = true;
+    try {
+      const res = await conversation.listThreads(THREADS_PAGE_SIZE, null);
+      if (threadsGenerationRef.current !== currentGen) return;
+      setThreads(res.items);
+      setNextCursor(res.next_cursor);
+    } catch (err) {
+      if (threadsGenerationRef.current !== currentGen) return;
+      const msg = err instanceof AgentChatHttpError ? err.message : GENERIC_ERROR;
+      setThreadsError(msg);
+    } finally {
+      if (threadsGenerationRef.current === currentGen) {
+        setLoadingMoreThreads(false);
+        isLoadingMoreRef.current = false;
+      }
+    }
   }, [conversation]);
+  const loadMoreThreads = (0, import_react10.useCallback)(async () => {
+    if (isLoadingMoreRef.current || !nextCursor) return;
+    const currentGen = threadsGenerationRef.current;
+    isLoadingMoreRef.current = true;
+    setLoadingMoreThreads(true);
+    setThreadsError(null);
+    try {
+      const res = await conversation.listThreads(THREADS_PAGE_SIZE, nextCursor);
+      if (threadsGenerationRef.current !== currentGen) return;
+      setThreads((prev) => {
+        const existingIds = new Set(prev.map((t) => t.conversation_id));
+        const newItems = res.items.filter((t) => !existingIds.has(t.conversation_id));
+        return [...prev, ...newItems];
+      });
+      setNextCursor(res.next_cursor);
+    } catch (err) {
+      if (threadsGenerationRef.current !== currentGen) return;
+      const msg = err instanceof AgentChatHttpError ? err.message : GENERIC_ERROR;
+      setThreadsError(msg);
+    } finally {
+      if (threadsGenerationRef.current === currentGen) {
+        setLoadingMoreThreads(false);
+        isLoadingMoreRef.current = false;
+      }
+    }
+  }, [conversation, nextCursor]);
   const openThread = (0, import_react10.useCallback)(
     async (conversationId) => {
       conversation.switchTo(conversationId);
@@ -53910,8 +53974,13 @@ function AgentChatApp({
                   {
                     open: threadsOpen,
                     threads,
-                    activeId: getStoredConversationId(config.endpoint),
-                    onSelect: openThread,
+                    activeId,
+                    loadingMore: loadingMoreThreads,
+                    error: threadsError,
+                    hasMore: hasMoreThreads,
+                    onLoadMore: () => void loadMoreThreads(),
+                    onRetry: () => threads.length === 0 ? void openThreads() : void loadMoreThreads(),
+                    onSelect: (cid) => void openThread(cid),
                     onNew: startNewThread,
                     onClose: () => setThreadsOpen(false)
                   }
@@ -54160,10 +54229,21 @@ function ThreadDrawer({
   open,
   threads,
   activeId,
+  loadingMore,
+  error,
+  hasMore,
+  onLoadMore,
+  onRetry,
   onSelect,
   onNew,
   onClose
 }) {
+  const handleScroll = (e) => {
+    const { scrollTop, clientHeight, scrollHeight } = e.currentTarget;
+    if (scrollHeight - scrollTop - clientHeight < SCROLL_THRESHOLD_PX && hasMore && !loadingMore && !error) {
+      onLoadMore();
+    }
+  };
   return /* @__PURE__ */ (0, import_jsx_runtime4.jsxs)("div", { className: `thread-drawer${open ? " open" : ""}`, inert: !open, children: [
     /* @__PURE__ */ (0, import_jsx_runtime4.jsxs)("div", { className: "thread-drawer-head", children: [
       /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("span", { children: "Chats" }),
@@ -54173,7 +54253,7 @@ function ThreadDrawer({
       /* @__PURE__ */ (0, import_jsx_runtime4.jsx)(SquarePen, { "aria-hidden": true }),
       "New chat"
     ] }),
-    /* @__PURE__ */ (0, import_jsx_runtime4.jsxs)("div", { className: "thread-list", children: [
+    /* @__PURE__ */ (0, import_jsx_runtime4.jsxs)("div", { className: "thread-list", onScroll: handleScroll, children: [
       threads.map((thread) => /* @__PURE__ */ (0, import_jsx_runtime4.jsx)(
         "button",
         {
@@ -54185,7 +54265,13 @@ function ThreadDrawer({
         },
         thread.conversation_id
       )),
-      threads.length === 0 ? /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("p", { className: "thread-empty", children: "No conversations yet" }) : null
+      threads.length === 0 && !loadingMore && !error ? /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("p", { className: "thread-empty", children: "No conversations yet" }) : null,
+      error ? /* @__PURE__ */ (0, import_jsx_runtime4.jsxs)("div", { className: "thread-empty thread-error", children: [
+        /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("p", { children: error }),
+        /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("button", { className: "thread-retry-btn", onClick: onRetry, type: "button", children: "Retry" })
+      ] }) : null,
+      loadingMore ? /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("p", { className: "thread-empty", children: "Loading..." }) : null,
+      hasMore && !loadingMore && !error ? /* @__PURE__ */ (0, import_jsx_runtime4.jsx)("button", { className: "thread-load-more-btn", onClick: onLoadMore, type: "button", children: "Load more" }) : null
     ] })
   ] });
 }