Conversation
Extend the socket-free --replay path (previously chat-only) to drive the chat-completions loglikelihood scorer, so perplexity can be validated without an HTTP surface. - ReplayRequest carries exactly one of `request` (chat) or `loglikelihood` (a CompletionRequest); the decoder rejects neither/both. - ReplayResult gains tokens / token_logprobs / text_offset, omitted for chat lines via the synthesized optional encoding. - Extract a socket-free runLoglikelihood(req:state:) core from the HTTP handler; ReplayRunner branches on the body type and keeps the sequential-only guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the socket-free
--replaypath (previously chat-only) so it can also drive the chat-completions loglikelihood scorer behind/v1/completions. This lets teacher-forced perplexity be validated by replaying a JSONL file, with no HTTP surface.Details
ReplayRequestnow carries exactly one of two bodies per line:request(a chat completion) orloglikelihood(a scoring request). A custom decoder rejects a line that has neither or both, keeping each record unambiguous.ReplayResultgains optionaltokens/token_logprobs/text_offset. These are present only for loglikelihood lines and are omitted for chat lines via the synthesized optional encoding.runLoglikelihood(req:state:)core, shared by the HTTP handler and--replay.ReplayRunnerbranches on the body type and keeps the sequential-only guard (loglikelihood scoring requires--variant coreai-sequential).Additive change; chat replay behavior is unchanged.