feat(ccwidgets): Implementing Real Time Assistance feature within cc widgets - #722
feat(ccwidgets): Implementing Real Time Assistance feature within cc widgets#722Kesari3008 wants to merge 10 commits into
Conversation
…x#668) Co-authored-by: Shreyas Sharma <shreyassharma9912@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ed response feature
|
This pull request is automatically being deployed by Amplify Hosting (learn more). |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65e4090194
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1,70 @@ | |||
| { | |||
| "name": "@webex/cc-ai-assistant", | |||
There was a problem hiding this comment.
This introduces a new workspace package/widget, but the change contains no ai-assistant module spec or standing-doc/router update, so the new public surface cannot be routed or validated by the repo's SDD docs. Please add the module spec/standing doc updates in the same change as the code.
AGENTS.md reference: AGENTS.md:L68-L68
Useful? React with 👍 / 👎.
| ); | ||
| } | ||
|
|
||
| if (status === 'listening' || hasFiredInitialRequest) { |
There was a problem hiding this comment.
When the first getRealTimeAssistance call rejects, the hook has already set hasFiredInitialRequest to true and then sets status to error; this branch runs before the error branch, so the panel keeps showing “Listening for information” and never exposes the retry/error message. Please check status === 'error' before the hasFiredInitialRequest listening fallback.
Useful? React with 👍 / 👎.
| oninput={(e: CustomEvent<{value: string}> & {target: HTMLInputElement}) => | ||
| onChange(e.detail?.value ?? e.target?.value ?? '') | ||
| } |
There was a problem hiding this comment.
For the Momentum React Input, the repo's existing usage is the camel-cased onInput; this lowercase oninput prop is not handled by the React wrapper, so typing in the “Add context” field never calls onChange and the controlled value stays empty. That blocks agents from submitting extra context after starting a suggestion session.
Useful? React with 👍 / 👎.
| onClose: 'function', | ||
| onClearChat: 'function', | ||
| onFullScreenToggle: 'function', | ||
| onSuggestionReceived: 'function', |
There was a problem hiding this comment.
For Web Component consumers this is the only arrival callback exposed, but the AI assistant hook only reads and invokes onRealTimeAssistReceived, never onSuggestionReceived. As a result <widget-cc-ai-assistant onSuggestionReceived=...> will not fire when a suggestion arrives; please either expose onRealTimeAssistReceived here or bridge the deprecated prop to the new callback.
Useful? React with 👍 / 👎.
| <label key={widget}> | ||
| <input | ||
| type="checkbox" | ||
| name={widget} | ||
| checked={selectedWidgets[widget]} | ||
| onChange={handleCheckboxChange} | ||
| data-testid={`samples:widget-${widget}`} | ||
| /> |
There was a problem hiding this comment.
Each widget now renders this second label in addition to the original one above it, producing duplicate checkboxes with the same name and data-testid. In WebRTC-locked mode the duplicate also omits the disabled guard, so users can still toggle widgets that the first checkbox correctly disables, and tests querying by test id become ambiguous.
Useful? React with 👍 / 👎.
Shreyas281299
left a comment
Shreyas281299
left a comment
There was a problem hiding this comment.
There are no UTs and SnapshotTesting for new components in cc-components.
| /> | ||
| | ||
| {formatWidgetName(widget)} | ||
| {widget === 'outdialCall' && ( |
There was a problem hiding this comment.
I think this is redundant
| @@ -1,5 +1,16 @@ | |||
| import {Profile} from './store.types'; | |||
|
|
|||
| export const AI_FEATURE_SUGGESTED_RESPONSES_KEY = 'isSuggestedResponsesEnabled'; | |||
There was a problem hiding this comment.
This should be in the constants file
| export const AI_FEATURE_SUGGESTED_RESPONSES_KEY = 'isSuggestedResponsesEnabled'; | ||
|
|
||
| const getValueAtPath = (obj: unknown, path: string): unknown => { | ||
| return path.split('.').reduce<unknown>((acc, segment) => { | ||
| if (acc && typeof acc === 'object' && segment in (acc as Record<string, unknown>)) { | ||
| return (acc as Record<string, unknown>)[segment]; | ||
| } | ||
| return undefined; | ||
| }, obj); | ||
| }; |
There was a problem hiding this comment.
Do we not have correct types for this? we are typecasting and using unknown alot
| const TASK_MULTI_LOGIN_HYDRATE = 'task:multiLoginHydrate'; | ||
| // Mirrored from the SDK's CC_TASK_EVENTS — importing the runtime const breaks | ||
| // our jest transform setup. | ||
| const SUGGESTED_RESPONSE_EVENT = 'SUGGESTED_RESPONSE'; |
There was a problem hiding this comment.
These shoudl be in the constants file
There was a problem hiding this comment.
Moved. I have moved MULTI_LOGIN_HYDRATE as well which is not part of this PR
| task.on(TASK_EVENTS.TASK_PARTICIPANT_LEFT_FAILED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_CONFERENCE_TRANSFERRED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_CONFERENCE_TRANSFER_FAILED, this.refreshTaskList); | ||
| task.on(TASK_EVENTS.TASK_POST_CALL_ACTIVITY, this.refreshTaskList); |
There was a problem hiding this comment.
Why are we removing this?
There was a problem hiding this comment.
It was already present, please check before task.on(TASK_EVENTS.TASK_CONFERENCE_ESTABLISHING, this.refreshTaskList);
| const getRealTimeAssistance = jest.fn().mockResolvedValue({}); | ||
| return { | ||
| __esModule: true, | ||
| default: { | ||
| cc: { | ||
| apiAIAssistant: { | ||
| getRealTimeAssistance, | ||
| }, | ||
| }, | ||
| clearRealTimeAssist, | ||
| }, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
we should be using mocks from test-fixtures. And use the complete mock cc object
| import {useAiAssistant} from '../src/helper'; | ||
| import store from '@webex/cc-store'; | ||
|
|
||
| jest.mock('@webex/cc-store', () => { |
There was a problem hiding this comment.
[nitpick ]If we can avoid mocking store that would be better.
| onFullScreenToggle, | ||
| onRealTimeAssistReceived, | ||
| }: UseAiAssistantInput) => { | ||
| const [chrome, setChrome] = useState<AIAssistantChromeState>('closed'); |
There was a problem hiding this comment.
Is this chrome googleChrome?? or something new?
| onRealTimeAssistReceived, | ||
| }: UseAiAssistantInput) => { | ||
| const [chrome, setChrome] = useState<AIAssistantChromeState>('closed'); | ||
| const [isFullScreen, setIsFullScreen] = useState(false); |
There was a problem hiding this comment.
Why are we worrying about fullscreen?
There was a problem hiding this comment.
It is supported in Agent Desktop as well
| const REAL_TIME_ASSIST_FLAG = 'isSuggestedResponsesEnabled'; | ||
| const GREETING_TEXT = "I'm here to help! I'll keep listening and suggest responses as the conversation evolves."; | ||
|
|
||
| interface UseAiAssistantInput extends IAIAssistantProps { | ||
| interactionId?: string; | ||
| agentId: string; | ||
| isFeatureEnabled: boolean; | ||
| realTimeAssist: RealTimeAssistPayload[]; | ||
| } | ||
|
|
||
| type UserMessage = {id: string; text: string; sentAt: number}; |
There was a problem hiding this comment.
we can move these to a types/constants file
github-actions
Bot
removed
the
validated
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 710b6684f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const [pendingRequest, setPendingRequest] = useState(false); | ||
| // ADD_SUGGESTIONS_EXTRA_CONTEXT only makes sense after a GET_SUGGESTIONS has fired. | ||
| const [hasFiredInitialRequest, setHasFiredInitialRequest] = useState(false); | ||
| const [userMessages, setUserMessages] = useState<UserMessage[]>([]); |
There was a problem hiding this comment.
When the mounted widget moves from one task to another, interactionId changes but this hook retains userMessages, contextDraft, hasFiredInitialRequest, and the request/error state from the previous task. Consequently an agent opening the assistant for the next customer can see context entered for the prior interaction and can submit additional context without starting the new session; reset the local session state whenever interactionId changes.
Useful? React with 👍 / 👎.
|
|
||
| const hasEntries = chatEntries.length > 0; | ||
|
|
||
| if (hasEntries) { |
There was a problem hiding this comment.
When any request rejects after hasFiredInitialRequest becomes true, chatEntries already contains the seeded greeting, so this hasEntries branch returns before the error UI below can run. This means even the first failed request still shows only the greeting, and failed follow-up context requests likewise provide no error or retry control. Although the earlier status-ordering issue was changed, the fresh greeting-backed hasEntries path still hides the error.
Useful? React with 👍 / 👎.
| contextDraft, | ||
| hasFiredInitialRequest, | ||
| chatEntries, | ||
| clearChat, |
There was a problem hiding this comment.
The hook returns clearChat, but the top-level component never consumes it and neither the header nor body exposes a control that invokes it. Therefore React and Web Component consumers cannot trigger a reset, onClearChat is a dead public callback, and accumulated suggestions/context can only disappear when the task is removed; pass this action into the presentation layer and wire it to a clear/reset control.
Useful? React with 👍 / 👎.
| </div> | ||
| ); | ||
| })} | ||
| {hasFiredInitialRequest && status !== 'error' ? ( |
There was a problem hiding this comment.
hasFiredInitialRequest && status !== 'error' keeps “Listening for information” visible when status === 'ready'. Please consider showing this only when status === 'listening' (or while pendingRequest is true).
|
|
||
| const hasEntries = chatEntries.length > 0; | ||
|
|
||
| if (hasEntries) { |
There was a problem hiding this comment.
Once the greeting is added, hasEntries stays true, so the error UI never shows. Check status === 'error' before the hasEntries branch, or show the error inside the chat view.
| ); | ||
| return; | ||
| } | ||
| setRequestStatus('listening'); |
There was a problem hiding this comment.
Double-clicking “Get Suggestions” or sending context quickly can start multiple getRealTimeAssistance calls because there’s no in-flight guard. Fix: add if (pendingRequest) return; before setRequestStatus('listening'), or disable Get Suggestions / Send while status === 'listening'.
| node.scrollTop = node.scrollHeight; | ||
| }, [chatEntries.length, status]); | ||
|
|
||
| const contextDisabled = !isFeatureEnabled || !hasFiredInitialRequest; |
There was a problem hiding this comment.
Context input stays enabled while a request is in flight. Consider disabling the input and Send button when status === 'listening'.
| this.store.acceptedCampaignIds = new Set(); | ||
| this.realtimeTranscriptionListeners = {}; | ||
| this.setLastConsultDestination(null); | ||
| this.setLastConsultDestination(null); |
There was a problem hiding this comment.
Duplicate line — please remove one.
| }, 1500); | ||
| } | ||
| } catch (err) { | ||
| console.warn('[AIAssistant] copy to clipboard failed', err); |
There was a problem hiding this comment.
Clipboard copy failure uses console.warn. Per repo logging conventions, please use store.logger (with module/method metadata) instead of console in widget code.
| (event: AIAssistantFeedbackEvent, assist: RealTimeAssistPayload) => { | ||
| const api = store.cc?.apiAIAssistant; | ||
| const adaptiveCardId = assist?.data?.adaptiveCardId; | ||
| if (!interactionId || !agentId || !adaptiveCardId || !api?.sendRealTimeAssistanceUserAction) return; |
There was a problem hiding this comment.
If adaptiveCardId is missing from the payload, the handler returns immediately and the agent gets no indication that like/dislike/copy was not sent. Please log a warning via store.logger or show a small inline error so failed feedback isn’t silent.
| onFeedbackRef.current?.({type: 'copy', actionId}); | ||
| return; | ||
| } | ||
| if (label.includes('dislike')) { |
There was a problem hiding this comment.
Like/dislike icons toggle before sendRealTimeAssistanceUserAction completes. If the SDK call fails, the UI still shows liked/disliked. Consider updating icons only after success, or revert on failure.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3cea779598
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (taskId && !this.realTimeAssistListeners[taskId]) { | ||
| const listener = (payload: RealTimeAssistPayload) => this.handleRealTimeAssist(taskId, payload); | ||
| this.realTimeAssistListeners[taskId] = listener; | ||
| task.on(SUGGESTED_RESPONSE_EVENT, listener); |
There was a problem hiding this comment.
When TASK_HYDRATE, TASK_MERGED, or multi-login hydration supplies a new task object for an interaction already present in realTimeAssistListeners, this ID-only guard skips attaching SUGGESTED_RESPONSE to the replacement object even though registerTaskEventListeners attaches the other handlers to it. Suggestions emitted by the active replacement task are therefore silently lost, and removal attempts to detach the callback from the wrong object; track the bound task identity and rebind when it changes.
Useful? React with 👍 / 👎.
| if (len <= lastSeenCountRef.current) return; | ||
| lastSeenCountRef.current = len; | ||
|
|
||
| const latest = realTimeAssist[len - 1]; |
There was a problem hiding this comment.
When multiple assistance events are appended before React commits a render, len can increase by more than one, but this effect advances lastSeenCountRef directly to the new length and invokes onRealTimeAssistReceived only for the final element. The callbacks for the other newly arrived payloads are permanently lost; process every item between the previous count and len before advancing the cursor.
Useful? React with 👍 / 👎.
5 participants
COMPLETES #https://jira-eng-sjc12.cisco.com/jira/browse/CAI-8037
This pull request addresses
Implementing Real Time Assistant Widget within existing cc widgets
by making the following changes
Added new widget: AI Assiatnt under which real time assistance feature is implemented
Change Type
The following scenarios were tested
https://app.vidcast.io/share/7db25f78-68f5-42a5-94cc-360b4e0aebc3
The GAI Coding Policy And Copyright Annotation Best Practices
Checklist before merging
Make sure to have followed the contributing guidelines before submitting.