feat(mcp): OpenAI app directory — domain proof + the submitted manifest - #202
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Small, well-scoped change: a single unauthenticated GET route serving a static domain-verification token, plus matching tests and CHANGELOG entries. Verified there's no global auth middleware in http-server.ts that could gate this route (auth is per-handler on /mcp, /sse, etc.), so it correctly answers before any auth check as the PR description claims. No route-path conflicts with the existing /.well-known/oauth-protected-resource* handlers. The new test file doesn't touch existing tests. No user-facing tool is added, so no routing/rendering_hint/WORKFLOWS.md entries are needed here. Committing the token in plaintext is intentional and justified — it's a public proof, not a credential, with an env-var rotation path. No blocking issues found.
The Apps submission form verifies the MCP hostname by fetching a token it issued from the origin root. Until that URL answers with the exact string, the app sits at "Domain not verified" and cannot be submitted; today it is a 404. `GET /.well-known/openai-apps-challenge` serves it as bare text, unauthenticated. The token is a constant with an `OPENAI_APPS_CHALLENGE` env override, the same shape as `STARGATE_AUTH_SERVER` a few lines above: it is a public proof rather than a credential, so keeping it in the repo means the route works wherever the server runs with no extra deploy config, and the env var covers a re-issue without waiting on a release. The two ways this fails leave nothing to read server-side — the route answering 401 behind an auth gate, and the body gaining a trailing newline or JSON quoting so it stops matching byte for byte. Both are asserted in a new test file. No version bump or changelog: this is a hosted-server route with no effect on the published npm package. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ee93873 to
0af98a4
Compare
`chatgpt-app-submission.json` is the file uploaded to the Apps submission form. Keeping it in the repo means the claims we made about ourselves — every tool's readOnly/openWorld/destructive hints and the one-sentence justification for each — are reviewable and diffable rather than living only in OpenAI's dashboard. It describes the gated surface, `https://mcp.leadbay.app/chatgpt/mcp`: 58 tools, verified against a live tools/list on that endpoint rather than against a checkout. The commerce pair and the retired agent-memory tools are absent by construction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
milstan
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d65a8631fe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "readOnlyHint": true, | ||
| "openWorldHint": true, | ||
| "destructiveHint": false |
There was a problem hiding this comment.
When leadbay_prepare_outreach is called with enrich: true, it invokes enrichContacts.execute, writes revealed contact details, and consumes non-refundable account credits. Declaring the entire tool read-only and non-destructive misleads OpenAI and MCP hosts about a paid side effect, potentially allowing it to be selected or executed without the safeguards expected for credit-consuming writes; mark it mutating/destructive in both this submission and the server tool annotation, or split enrichment into a separate write tool.
Useful? React with 👍 / 👎.
| }, | ||
| "leadbay_pull_followups": { | ||
| "annotations": { | ||
| "readOnlyHint": true, |
There was a problem hiding this comment.
When callers supply set_filter (or a resolved city), leadbay_pull_followups persists a Monitor filter through POST /monitor/filter, changing the scope of later Monitor reads and the user's saved view across sessions. Advertising this as read-only is therefore inaccurate and can let an apparently harmless lookup silently alter subsequent workflows; the submission and runtime annotation should use readOnlyHint: false for this combined read/write tool.
Useful? React with 👍 / 👎.
| }, | ||
| "leadbay_list_mappable_fields": { | ||
| "annotations": { | ||
| "readOnlyHint": true, |
There was a problem hiding this comment.
When for_records is provided, leadbay_list_mappable_fields uploads a CSV with POST /imports and leaves a durable import record on the account, as the adjacent justification acknowledges. Labeling that path read-only can cause hosts to call or retry it as a lookup, creating unexpected duplicate preview records; mark the tool mutating in both declarations or move the preview upload into a separate write tool.
Useful? React with 👍 / 👎.
| }, | ||
| { | ||
| "description": "Look a company up by name and summarise what the account already knows about it.", | ||
| "user_prompt": "Tell me what we know about the top company in that batch.", |
There was a problem hiding this comment.
When submission test cases are evaluated independently, this prompt contains no company name even though leadbay_research_lead_by_name_fuzzy requires a non-empty companyName, so the expected tool cannot be invoked without inventing an argument. Even if the cases were run sequentially, the preceding lead batch provides a UUID and naturally routes to leadbay_research_lead_by_id, not the declared fuzzy lookup; use an explicit company name or domain in this prompt so the test reliably exercises the stated tool.
Useful? React with 👍 / 👎.
| "app_info": { | ||
| "display_name": "Leadbay", | ||
| "subtitle": "Work your B2B lead pipeline", | ||
| "description": "Leadbay is a B2B prospecting workspace that sources and scores companies against a seller's ideal customer profile. This app connects ChatGPT to the user's own Leadbay account so they can work that pipeline in plain language instead of clicking through the web app.\n\nTypical sessions: pull the day's fresh, AI-scored prospects and skim the best ones; research a single company in depth (qualification signals, contacts, recent web findings, past notes and activity); draft a personalised outreach email or call opener for a named contact; plan which accounts to visit in a city before a trip; import a list of companies and have Leadbay qualify them; and log what was actually sent so the pipeline stays honest.\n\nIt also covers the account admin a seller needs along the way: managing lenses (saved audiences), campaigns, contacts, CRM custom fields, and the qualification questions every lead is scored against.\n\nThe app reads and writes only the signed-in user's own Leadbay workspace, over an OAuth token the user grants. It does not send email or messages on the user's behalf — it drafts outreach and records the outcome the user reports.", |
There was a problem hiding this comment.
When users invoke leadbay_send_feedback or leadbay_report_friction, the app sends user-authored content to Sentry or PostHog, respectively, as the tool justifications later in this same submission acknowledge. The absolute claim that the app reads and writes only the user's Leadbay workspace is therefore false and omits externally processed data from the top-level disclosure; qualify this statement and explicitly disclose these consent-gated telemetry destinations.
Useful? React with 👍 / 👎.
…y sinks Two review findings on the submitted manifest. The fuzzy-lookup case read "the top company in that batch", which carries no company name — `leadbay_research_lead_by_name_fuzzy` requires a non-empty `companyName`, so a reviewer running the case on its own has nothing to pass. Run in sequence it is worse: the preceding batch hands over a UUID, which routes to `leadbay_research_lead_by_id` instead. Now names CEGEDIM, which is also the company the demo recording uses, so the manifest, the video and the reviewer account agree. The description claimed the app "reads and writes only the signed-in user's own Leadbay workspace". Not true as an absolute: `leadbay_send_feedback` delivers to Sentry and `leadbay_report_friction` emits a PostHog event, as those tools' own justifications in this same file say. Both are consent-gated and carry no lead or contact data, but a top-level disclosure should not have to be corrected by reading further down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
[Claude]: Went through the five Codex findings. Two are fixed in a832408; three need a product call. FixedSupply a company name to the fuzzy-lookup test (P2) — correct, and worse than stated. The prompt said "the top company in that batch", which gives Disclose telemetry writes in the app description (P2) — correct. "Reads and writes only the signed-in user's own Leadbay workspace" was false as an absolute: Not fixed — needs a decision, not a patchThe other three are the same finding three times: a tool declared
Codex is right on the facts. All three were found in the source audit before this manifest was generated and were surfaced with the same recommendation; the call was to submit as declared, because the annotations live in Two things make this worth re-opening rather than closing:
Not acting on these unilaterally. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8324088f4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "app_info": { | ||
| "display_name": "Leadbay", | ||
| "subtitle": "Work your B2B lead pipeline", | ||
| "description": "Leadbay is a B2B prospecting workspace that sources and scores companies against a seller's ideal customer profile. This app connects ChatGPT to the user's own Leadbay account so they can work that pipeline in plain language instead of clicking through the web app.\n\nTypical sessions: pull the day's fresh, AI-scored prospects and skim the best ones; research a single company in depth (qualification signals, contacts, recent web findings, past notes and activity); draft a personalised outreach email or call opener for a named contact; plan which accounts to visit in a city before a trip; import a list of companies and have Leadbay qualify them; and log what was actually sent so the pipeline stays honest.\n\nIt also covers the account admin a seller needs along the way: managing lenses (saved audiences), campaigns, contacts, CRM custom fields, and the qualification questions every lead is scored against.\n\nLead and contact data is read and written only in the signed-in user's own Leadbay workspace, over an OAuth token the user grants. Two consent-gated tools are the exception and send nothing else: \"send feedback\" delivers a message the user wrote to the Leadbay team's Sentry inbox, and \"report a problem\" emits one product-analytics event to PostHog carrying the user's own words. Both fire only when the user asks or accepts an offer, and neither carries lead or contact data. The app does not send email or messages on the user's behalf — it drafts outreach and records the outcome the user reports.", |
There was a problem hiding this comment.
The top-level description says lead and contact data stays in the user's Leadbay workspace and identifies only the two telemetry tools as exceptions, but leadbay_enrich_contacts explicitly sends a contact to Leadbay's enrichment provider (also reachable through leadbay_prepare_outreach with enrich: true). For users who request enrichment, contact data is therefore processed outside the workspace, so this disclosure is incomplete and can mislead both users and the app reviewer; identify enrichment-provider processing as another exception.
Useful? React with 👍 / 👎.
| }, | ||
| "leadbay_followups_map": { | ||
| "annotations": { | ||
| "readOnlyHint": true, |
There was a problem hiding this comment.
When set_filter, city, or city_id is supplied, this tool delegates directly to pullFollowups.execute, which persists the effective filter with POST /monitor/filter. This separate tool declaration therefore still advertises a durable saved-view update as read-only, even if the already-reported leadbay_pull_followups declaration is corrected; mark this alias mutating as well.
Useful? React with 👍 / 👎.
| }, | ||
| "leadbay_tour_plan": { | ||
| "annotations": { | ||
| "readOnlyHint": true, |
There was a problem hiding this comment.
For the normal valid-city path, tourPlan.execute passes the city to pullFollowups.execute; resolving that city creates an effective Monitor filter and persists it through POST /monitor/filter. Thus the submitted Paris test case changes the user's saved Monitor view across sessions despite this declaration claiming the itinerary is read-only; advertise the tool as mutating.
Useful? React with 👍 / 👎.
| }, | ||
| "leadbay_scan_portfolio_signals": { | ||
| "annotations": { | ||
| "readOnlyHint": true, |
There was a problem hiding this comment.
When a scan is scoped with set_filter, city, or city_id rather than explicit leadIds, scanPortfolioSignals.execute stores the resulting filter via POST /monitor/filter before scanning. That overwrites the user's durable Monitor filter and affects later Monitor reads, so this tool's independent declaration must not remain read-only.
Useful? React with 👍 / 👎.
| }, | ||
| { | ||
| "description": "Draft outreach for a named lead. Verifies the app drafts but never sends.", | ||
| "user_prompt": "Draft me a personalised outreach email for that company.", |
There was a problem hiding this comment.
When submission cases run independently, “that company” provides neither a company identity nor the required leadId for leadbay_prepare_outreach. The expected tool cannot be invoked without relying on the preceding CEGEDIM case or first calling a lookup tool not listed in tools_triggered; include a concrete lead UUID, or name the company and declare the required lookup step.
Useful? React with 👍 / 👎.
| "annotations": { | ||
| "readOnlyHint": false, | ||
| "openWorldHint": true, | ||
| "destructiveHint": false |
There was a problem hiding this comment.
A successful POST /lenses/{id}/extra_refill consumes the account's finite daily LENS_EXTRA_REFILL quota and queues an asynchronous refill; that quota cannot be restored by undoing the added leads. This is the same kind of non-refundable resource consumption for which enrichment is marked destructive, so leaving destructiveHint false can suppress safeguards around an irreversible quota spend.
Useful? React with 👍 / 👎.
| "user_prompt": "Tell me what we know about CEGEDIM.", | ||
| "file_attachment_urls": null, | ||
| "tools_triggered": "leadbay_research_lead_by_name_fuzzy", | ||
| "expected_output": "A single company card for CEGEDIM: qualification signals and score, contacts with titles, recent web findings, and prior notes or activity if any exist.", |
There was a problem hiding this comment.
leadbay_research_lead_by_name_fuzzy ultimately returns the researchLeadById shape, whose engagement section contains only notes_count and whose recent timeline contains activity type/date pairs; it never fetches note bodies. When a matched company has notes, this expected output therefore asks the sole declared tool to present information it does not return; either add leadbay_account_history to the case or limit the expectation to note counts and recent activity.
Useful? React with 👍 / 👎.
| "description": "Plan in-person visits for a trip to a city, mixing existing accounts with new prospects.", | ||
| "user_prompt": "I'm visiting Paris next week — which leads should I go and see?", | ||
| "file_attachment_urls": null, | ||
| "tools_triggered": "leadbay_tour_plan", |
There was a problem hiding this comment.
This prompt specifies a city but neither which cohort to include nor whether missing contacts should be enriched. The leadbay_plan_tour_in_city prompt and leadbay_tour_plan description require asking those scope questions and waiting before invoking the tool, so a single-turn submission case that declares leadbay_tour_plan as immediately triggered and expects a completed itinerary conflicts with the app's own routing contract; provide the scope choices in the test prompt or make the expected first response the clarification.
Useful? React with 👍 / 👎.
| "user_prompt": "I'm visiting Paris next week — which leads should I go and see?", | ||
| "file_attachment_urls": null, | ||
| "tools_triggered": "leadbay_tour_plan", | ||
| "expected_output": "A city-scoped visit list mixing existing customers, already-qualified leads and fresh prospects, each with a mode badge, a one-line reason to go, and the contact to ask for. If the account holds no known accounts in that city, the app says so rather than padding the list.", |
There was a problem hiding this comment.
The tour implementation labels a Monitor lead as ★ Customer whenever it has any epilogue status or prior prospecting/Monitor action. A prospect that received one unanswered email satisfies that predicate but is not an existing customer, so this expected output encourages the app to overstate account relationships; call the bucket “known/previously worked accounts” unless an actual customer-status field establishes the relationship.
Useful? React with 👍 / 👎.
| "destructiveHint": false | ||
| }, | ||
| "justifications": { | ||
| "read_only_justification": "Combines Leadbay Monitor follow-up reads with fresh wishlist reads to assemble a city itinerary.", |
There was a problem hiding this comment.
When an ambiguous city is resolved by re-calling with only city_id, the Monitor half is correctly scoped through pullFollowups, but the Discover half is filtered with cityMatches(lead, params.city). Because params.city is then undefined, cityMatches accepts every wishlist lead, so the claimed city itinerary mixes in unrelated Discover leads from across the lens; retain the resolved city text or filter Discover leads by the selected location ID.
Useful? React with 👍 / 👎.
1 participant
Two commits, both for the OpenAI Apps submission.
1.
GET /.well-known/openai-apps-challenge— domain proofThe submission form verifies the MCP hostname by fetching a token it issued from the origin root and comparing the body byte for byte. Until that URL answers, the app sits at Domain not verified and cannot be submitted. Today it is a 404:
The route returns the bare token as
text/plain, unauthenticated,Cache-Control: no-store. The token ships as a constant with anOPENAI_APPS_CHALLENGEenv override — the same shape asSTARGATE_AUTH_SERVERtwenty lines above it. It is a public proof, not a credential: OpenAI fetches it anonymously and so may anyone. Keeping it in the repo means the route works wherever the server runs with no extra deploy config; the env var covers a re-issue without waiting on a release.It answers before any auth gate, which it does for free because the MCP routes gate inside their own handlers rather than in middleware.
Tests — new file
packages/mcp/test/unit/openai-apps-challenge.test.ts, covering the two ways this fails silently with nothing to read server-side: the route answering 401 behind an auth gate, and the body gaining a trailing newline or JSON quoting so it stops matching.2.
chatgpt-app-submission.json— the manifest we uploadedThe file the Apps form consumes. In the repo it makes the claims we made about ourselves reviewable and diffable — every tool's
readOnlyHint/openWorldHint/destructiveHintand the one-sentence justification for each — rather than living only in OpenAI's dashboard.It describes the gated surface
https://mcp.leadbay.app/chatgpt/mcp: 58 tools, verified against a livetools/liston that endpoint rather than against a checkout. The commerce pair and the retired agent-memory tools are absent by construction. No secrets — the reviewer credentials go in the form, not the file.Notes
pnpm -r typecheckandpnpm -r testgreen — 847 tests, 123 files.leadbay/frontend#1247: our consent screen rejects OpenAI's OAuthstate, so the handshake never completes and domain verification cannot even be attempted until that lands.🤖 Generated with Claude Code