Repository navigation
ADFA-6223 | Fix AI agent refusing general questions and migrate prompts to YAML - #114
Conversation
…s [ADFA-6223]
Stop refusing general and off-domain questions, and keep working through multi-part requests.
- Move the system prompts, agent loop and tool wording for ai-core and the three backends into YAML; backends take their prompt config as a constructor argument, and ai-core runs its activation checks through PromptConfigChecks.
- Make web search, fetch_url and the device clock always available, and remove the Web search switch.
- End tool runs through respond, fix add_dependency paths, and cite only real sources. Check code-bearing answers in a second review pass that verifies API claims against the evidence.
- ToolCallExtractor ignores bare {"tool":...} JSON inside code fences, closed or not.
- Web search: a GroundingRedirect on a non-HTTP link or unchecked failure is cited as given rather than failing the search; OpenAI throws on a 2xx reply carrying an error object, and 402 and insufficient_quota classify as BillingRequired. OpenAI read timeout is 180s, reported as TimedOut.
- EXAMPLE_FILE_STEM falls back to the full name for dotfiles.
- Use plugin-api's shared prompt engine, YAML loader, store and pane helpers (ADFA-6281); each plugin keeps only its config type, PromptConfigParser and sharedPromptConfig store. snakeyaml moves to testImplementation.
Daniel-ADFA
left a comment
There was a problem hiding this comment.
Reviewed at 62b7656. Depends on appdevforall/CodeOnTheGo#2087, reviewed separately.
CRITICAL
- AndroidManifest.xml:60 - five AI plugins floor at 26.39 or lower but need appdevforall/CodeOnTheGo#2087's unreleased classes
IMPORTANT
- build.gradle.kts:85 - libs/plugin-api.jar has no ai.* classes, so none of the plugins compile
- ToolCallExtractor.kt:207 - a fenced tool call no longer runs and is shown as the answer
- FetchUrlHandler.kt:70 - redirects reach loopback and LAN hosts without a new approval
- ChatViewModel.kt:988 - web_search is forced on backends that cannot search
MINOR
- AndroidManifest.xml:5 - the PR description omits the new internet access and features
NITPICK - 1 inline, not listed
No prior review rounds. The repo has no written approve/request-changes rule (CLAUDE.md, plugin-review skill), so the default grading applied.
Daniel-ADFA
left a comment
There was a problem hiding this comment.
Requesting changes for the CRITICAL and four IMPORTANT findings in the review above: floor the five AI plugins at the first CodeOnTheGo release that contains appdevforall/CodeOnTheGo#2087 and hold publishing until it ships, refresh libs/ so they compile, keep fenced tool calls running, stop fetch_url following cross-host redirects without approval, and force web_search only on backends that can search.
Floor AI plugins at 26.41, run fenced tool calls, stop fetch_url at cross-host redirects, offer web_search only where the backend can search.
…ADFA-6223] Drops the preload/release copies from AI-Core, Gemini, OpenAI and Local.
Daniel-ADFA
left a comment
There was a problem hiding this comment.
Re-review at 5c72a13. Depends on appdevforall/CodeOnTheGo#2087, reviewed separately.
IMPORTANT
- FetchUrlHandler.kt:35 - Always Allow on fetch_url approves every later host for the session
- PromptVariables.kt:224 - the web_access text tells backends without web_search to call it
- ToolResultsPrompt.kt:66 - fetch_url results are cut to 4,000 characters
- build.gradle.kts:85 (existing thread) - libs/plugin-api.jar still lacks the #2087 API
MINOR
- ToolCallExtractor.kt:56 - a stray inline ``` blanks a later bare call (plausible)
NITPICK - 1 inline, not listed
Previous round, checked against the code at 5c72a13:
- CRITICAL AndroidManifest.xml:60 - fixed: all five plugins floor at 26.41.
- IMPORTANT build.gradle.kts:85 - not fixed yet (plan agreed); reopened.
- IMPORTANT ToolCallExtractor.kt:211 - fixed: a reply that is one fenced block and nothing else runs as a call.
- IMPORTANT FetchUrlHandler.kt:70 - fixed: redirects are followed within one host; a cross-host redirect comes back as a URL.
- IMPORTANT ChatViewModel.kt:988 - fixed:
web_searchis offered and forced only whenWebSearchBackend.canSearchWeb()says so. - MINOR AndroidManifest.xml:5 - fixed: the description lists each added capability with QA steps.
- NITPICK GeminiPlugin.kt:187 - fixed: the plugins load through
PromptConfigStore.reload.
The repo has no written approve/request-changes rule (CLAUDE.md, plugin-review skill), so the default grading applied.
Daniel-ADFA
left a comment
There was a problem hiding this comment.
Requesting changes for the IMPORTANT findings in the re-review above: stop fetch_url from being session-approved, render the web_search wording only for backends that offer it, raise the fetch_url result limit, and refresh libs/ once appdevforall/CodeOnTheGo#2087 merges so the plugins compile.
Refs: ADFA-6223
Move read_app_logs/read_ide_logs wording into tool_descriptions.yml and point LogWindowCalculator at ToolResultsPrompt.DEFAULT_CHAR_LIMIT. Refs: ADFA-6223
Description
This PR resolves an issue where the AI agent incorrectly refused general or non-Android requests by overhauling how system prompts are defined and processed.
assets/prompts/for the Gemini, OpenAI, Local, and AI-Core plugins.systemInstructionfield rather than fabricating a user turn.Details
agent.yml,rules.yml,layout.yml) parsed viaPromptConfigLoaderandPromptConfigParser.After.mov
Added capabilities (and how to test them)
AI-Core now declares
INTERNETandnetwork.access, for the two web tools below.web_searchtool. A one-off search through the active backend's own provider search (Gemini Google Search grounding, OpenAI Responsesweb_search). Offered only when the backend reportsWebSearchBackend.canSearchWeb().web_searchcall and the answer cites sources. Point OpenAI at Ollama/LM Studio, or use Local, and ask again: noweb_searchis offered or attempted.VerificationPolicy). The first turn must callweb_searchwhen the message holds a code fence, two or more code-like lines, a currency question ("latest", "deprecated"), or a review request with files attached.JsonFeatureand ask "Is this correct?" The first turn searches, and the answer flagsJsonFeatureas removed. On a backend that cannot search, nothing is forced.fetch_urltool. Reads one http(s) page or raw file. Asks for approval before every fetch and follows redirects only within the same host. A cross-host redirect comes back as a URL, so following it needs a new call and a new approval.AgentLoopTest; on device, multi-part requests ("read X, then change Y") should finish every part.{"tool":…}block runs as a call. Fenced JSON inside prose stays an example.ToolCallExtractorTest.Merge and publish order
ai.prompt,ai.ui,PromptConfigStore.reload,LlmInferenceService.WebSearchBackend).plugin.min_ide_versionat 26.41. Do not publish before 26.41 ships.libs/with Update libs from CodeOnTheGo before merging this PR. The committedlibs/plugin-api.jarlacks these classes, so the AI plugins do not compile against it yet.Ticket
ADFA-6223
Observation
The YAML configurations are strictly validated upon plugin activation. Any missing, unknown, or duplicate keys will be rejected with the specific file name logged, falling back to the ai-core default prompt if rendering fails.