Conversation
…t bodies visualDetailLevel is applied to image_url.detail, but was then spread into the Chat Completions body via restOfConfig and rejected as an unknown parameter.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request extracts visualDetailLevel from the request configuration to prevent it from leaking into the OpenAI request body, and adds a test to verify this behavior. The feedback recommends adding visualDetailLevel to ChatCompletionCommonConfigSchema to provide schema validation, type safety, and autocompletion, while also utilizing the otherwise unused VisualDetailLevelSchema.
| version: modelVersion, | ||
| tools: toolsFromConfig, | ||
| apiKey, | ||
| visualDetailLevel, |
There was a problem hiding this comment.
While destructuring visualDetailLevel here correctly prevents it from leaking into the OpenAI request body, visualDetailLevel is currently missing from ChatCompletionCommonConfigSchema (defined on line 79). As a result, users do not get autocompletion, type safety, or schema validation for this option. Additionally, VisualDetailLevelSchema (defined on line 70) remains unused dead code.\n\nTo resolve this, consider adding visualDetailLevel to ChatCompletionCommonConfigSchema in js/plugins/compat-oai/src/model.ts:\n\ntypescript\nexport const ChatCompletionCommonConfigSchema =\n GenerationCommonConfigSchema.extend({\n temperature: z.number().min(0).max(2).optional(),\n frequencyPenalty: z.number().min(-2).max(2).optional(),\n logProbs: z.boolean().optional(),\n presencePenalty: z.number().min(-2).max(2).optional(),\n topLogProbs: z.number().int().min(0).max(20).optional(),\n visualDetailLevel: VisualDetailLevelSchema,\n });\n
toOpenAIRequestBody()now omitsvisualDetailLevelin the existing config destructure, the same wayapiKey,maxOutputTokens,topK, andversionare already omitted. Imagedetailmapping is unchanged.config.visualDetailLevelis applied toimage_url.detailintoOpenAIMessages(), buttoOpenAIRequestBody()did not pull it out ofrequest.configbefore spreadingrestOfConfiginto the Chat Completions body. OpenAI then returned400 Unknown parameter: 'visualDetailLevel'.Fixes #4592.
Checklist
visualDetailLevel; no doc change)Decision
Strip
visualDetailLevelin the existing destructure list. Alternative: also add it toChatCompletionCommonConfigSchema, or share a denylist with the image/audio/translate converters.Stripping in the destructure matches
toOpenAIRequestBody()'s current pattern and the Go adapter (go/plugins/compat_oai/generate.goalready deletesvisualDetailLevel). Putting it on the Zod schema would not stop the leak because common config is.passthrough(). Can switch to the schema or shared-denylist approach if that is preferred.Test plan
toOpenAIRequestBody('gpt-4o', { config: { visualDetailLevel: 'high' }, ...image... })has no top-levelvisualDetailLeveland still setsimage_url.detailto'high'.Received value: "high"onnot.toHaveProperty('visualDetailLevel')) and passes with it.@genkit-ai/compat-oaiunit tests (7 suites, 102 tests).