Skip to content

fix(js/plugins/compat-oai): omit visualDetailLevel from OpenAI request bodies - #6330

Open
cpruijsen wants to merge 1 commit into
genkit-ai:mainfrom
cpruijsen:fix/issue-4592
Open

cpruijsen wants to merge 1 commit into
genkit-ai:mainfrom
cpruijsen:fix/issue-4592

Conversation

@cpruijsen

Copy link
Copy Markdown

toOpenAIRequestBody() now omits visualDetailLevel in the existing config destructure, the same way apiKey, maxOutputTokens, topK, and version are already omitted. Image detail mapping is unchanged.

config.visualDetailLevel is applied to image_url.detail in toOpenAIMessages(), but toOpenAIRequestBody() did not pull it out of request.config before spreading restOfConfig into the Chat Completions body. OpenAI then returned 400 Unknown parameter: 'visualDetailLevel'.

Fixes #4592.

Checklist

  • PR title follows https://www.conventionalcommits.org/en/v1.0.0/
  • PR description contains enough context, bug link, code samples for new APIs
  • Tested (unit tested)
  • Docs/readmes updated (README already documents visualDetailLevel; no doc change)

Decision

Strip visualDetailLevel in the existing destructure list. Alternative: also add it to ChatCompletionCommonConfigSchema, or share a denylist with the image/audio/translate converters.

Stripping in the destructure matches toOpenAIRequestBody()'s current pattern and the Go adapter (go/plugins/compat_oai/generate.go already deletes visualDetailLevel). Putting it on the Zod schema would not stop the leak because common config is .passthrough(). Can switch to the schema or shared-denylist approach if that is preferred.

Test plan

  • Regression: toOpenAIRequestBody('gpt-4o', { config: { visualDetailLevel: 'high' }, ...image... }) has no top-level visualDetailLevel and still sets image_url.detail to 'high'.
  • Test fails without the source change (Received value: "high" on not.toHaveProperty('visualDetailLevel')) and passes with it.
  • @genkit-ai/compat-oai unit tests (7 suites, 102 tests).

…t bodies

visualDetailLevel is applied to image_url.detail, but was then spread into the Chat Completions body via restOfConfig and rejected as an unknown parameter.
@google-cla

google-cla Bot commented Sep 10, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request extracts visualDetailLevel from the request configuration to prevent it from leaking into the OpenAI request body, and adds a test to verify this behavior. The feedback recommends adding visualDetailLevel to ChatCompletionCommonConfigSchema to provide schema validation, type safety, and autocompletion, while also utilizing the otherwise unused VisualDetailLevelSchema.

version: modelVersion,
tools: toolsFromConfig,
apiKey,
visualDetailLevel,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While destructuring visualDetailLevel here correctly prevents it from leaking into the OpenAI request body, visualDetailLevel is currently missing from ChatCompletionCommonConfigSchema (defined on line 79). As a result, users do not get autocompletion, type safety, or schema validation for this option. Additionally, VisualDetailLevelSchema (defined on line 70) remains unused dead code.\n\nTo resolve this, consider adding visualDetailLevel to ChatCompletionCommonConfigSchema in js/plugins/compat-oai/src/model.ts:\n\ntypescript\nexport const ChatCompletionCommonConfigSchema =\n GenerationCommonConfigSchema.extend({\n temperature: z.number().min(0).max(2).optional(),\n frequencyPenalty: z.number().min(-2).max(2).optional(),\n logProbs: z.boolean().optional(),\n presencePenalty: z.number().min(-2).max(2).optional(),\n topLogProbs: z.number().int().min(0).max(20).optional(),\n visualDetailLevel: VisualDetailLevelSchema,\n });\n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[JS] [compat-oai] visualDetailLevel leaks into OpenAI API request body causing "Unknown parameter" error

1 participant