Skip to content

fix: strip every Astra directive from streamed chat deltas, not only metrics - #494

Merged
thomasluizon merged 3 commits into
mainfrom
fix/ticket-357-strip-all-directives
Aug 24, 2026
Merged

fix: strip every Astra directive from streamed chat deltas, not only metrics#494
thomasluizon merged 3 commits into
mainfrom
fix/ticket-357-strip-all-directives

Conversation

@thomasluizon

@thomasluizon thomasluizon commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Ticket: #357 in the ticket repository, https://github.com/thomasluizon/orbit-tickets/issues/357

Summary

  • Strip habit, goal, and metrics directives from streamed chat deltas with one case insensitive filter and one directive list.
  • Preserve partial directive buffering across chunk boundaries.
  • Preserve final message extraction and card construction.

Testing

  • dotnet build Orbit.slnx --no-restore -v minimal: passed with 0 errors.
  • Focused application streaming tests: 8 passed.
  • Analyzer, domain, and application suites passed.
  • GitHub Build, C# analysis, all mutation suites, guards, security scans, CodeQL, and Pullfrog approval passed.
  • GitHub Unit Tests fails in Run Infrastructure Tests, and SonarCloud Analysis fails in Test with Coverage. Both steps report the same 9 failures in unchanged AiIntentServiceStreamingTests, with 2,159 other infrastructure tests passing. No files in Orbit.Infrastructure or its tests differ from origin/main.

Closes #357

@thomasluizon

Copy link
Copy Markdown
Owner Author

Approach\n\nI will define the complete supported directive list once in the existing stream filter path, generalize its matching and partial suffix retention across that list, and keep matching case insensitive.\n\nThe implementation will land in src/Orbit.Application/Chat/Commands/ProcessUserChatCommand.Ai.cs. Behavior coverage will land in tests/Orbit.Application.Tests/Commands/Chat/ProcessUserChatCommandHandlerTests.cs for each directive, a split habit token, uppercase input, mixed directives with both final cards, and unchanged ordinary deltas.\n\nThis keeps one filter and one directive list. I rejected separate filters per card because that recreates the divergence behind this defect. I also rejected changing final extraction or card builders because they already provide the required final event behavior and are outside #357.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

This PR contains no implementation, so the advertised directive leak remains unchanged and must be fixed before merge.

Reviewed changes Compared the sole commit with main and traced the existing chat streaming and final card extraction paths.

  • Tracked changes: The commit 81371f1 changes no files.
  • Current directive path: The existing stream filter only suppresses [[orbit:metrics]]; final response processing separately extracts habit, goal, and metrics directives.
  • Verification: No focused tests were added for habit, goal, multiple, or chunk-split directives.

⚠️ The advertised fix is absent

The branch is identical to main, so it still allows habit and goal directive text to reach streamed chat deltas. Merging this PR would close without changing the reported behavior.

Technical details
# Implement directive filtering before merge

## Affected sites
- `src/Orbit.Application/Chat/Commands/ProcessUserChatCommand.Ai.cs:159-225`: `MetricsDirectiveStreamFilter` only recognizes `MetricsCardBuilder.Directive`.
- `src/Orbit.Application/Chat/Commands/ProcessUserChatCommand.cs:215-245`: final response processing already extracts habit, goal, and metrics directives and must continue doing so.
- `tests/Orbit.Application.Tests/Commands/Chat/ProcessUserChatCommandHandlerTests.cs:973-1005`: existing stream coverage only exercises a metrics directive split across chunks.

## Required outcome
- Suppress every supported Astra response directive from all emitted chat deltas, including directives split across chunks, while preserving final card extraction.
- Add focused unit coverage that fails under the current metrics-only filter.

Pullfrog  | Fix it ➔View workflow run | Using GPT Sol𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes Reviewed the directive filtering implementation and regression coverage added since the prior Pullfrog review.

  • Generalized stream filtering: Replaced the metrics-only filter with case-insensitive removal of every currently supported habit, goal, and metrics directive.
  • Preserved streaming semantics: Retained partial directive prefixes across chunk boundaries while preserving resets, ordinary deltas, multiple directive removal, and final flushing.
  • Expanded regression coverage: Added exact assertions for every directive, mixed directives, split chunks, uppercase tokens, unchanged ordinary chunks, and final card construction.

Pullfrog  | View workflow run | Using GPT Sol𝕏

@thomasluizon

Copy link
Copy Markdown
Owner Author

The red CI is not this pull request. main is already red.

Verified out of band before blaming the diff.

Unit Tests and SonarCloud Analysis fail here because nine tests in Orbit.Infrastructure.Tests.AI.AiIntentServiceStreamingTests already fail on main at df9740a7, with none of this branch's commits present. Reproduced locally on a clean checkout:

dotnet test tests/Orbit.Infrastructure.Tests --filter "FullyQualifiedName~AiIntentServiceStreamingTests"
Failed: 9, Passed: 7, Total: 16

This branch changes two files, both in the Application layer: ProcessUserChatCommand.Ai.cs and ProcessUserChatCommandHandlerTests.cs. The failing tests are Infrastructure and cover the OpenAI client's streaming parse, which this diff does not touch.

Root cause, isolated: #479 bumped OpenAI from 2.12.0 to 2.13.0. Pinning it back on main, changing nothing else, turns the same run green:

OpenAI version on main result
2.13.0 (current) Failed 9, Passed 7
2.12.0 Failed 0, Passed 16

The pin was reverted; the checkout is clean and still on 2.13.0.

Filed as thomasluizon/orbit-tickets#364, which blocks every open pull request against this repository, not just this one.

Where that leaves this pull request

Delivery verified: clean tree, 2 commits, pushed, head matches, ticket linked, 2 files, +187/-28. The verdict from verify-delivery.mjs is CI_FAILING, and the cause is inherited.

It should not merge while main is red, because the standing rule is that every required check passes. #364 lands first, this branch takes main, and its checks re-run clean.

@thomasluizon thomasluizon changed the title Strip all Astra directives from chat deltas (#357) fix: strip every Astra directive from streamed chat deltas, not only metrics Aug 24, 2026
@sonarqubecloud

Copy link
Copy Markdown

@thomasluizon
thomasluizon merged commit ce27f3e into main Aug 24, 2026
22 checks passed
@thomasluizon
thomasluizon deleted the fix/ticket-357-strip-all-directives branch August 24, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant