Migrate e2e tests from cavy to react-native-cavynext, add web e2e support - #893
Merged
Conversation
Replace all cavy imports with react-native-cavynext, update test scripts to use the cavynext CLI, and point dev dependencies at the local cavynext workspace packages. Update workflows and gitignore to look for cavynext_results.md. Add a placeholder player-specific support package at e2e/packages/react-native-theoplayer-cavynext for THEOplayer-aware test helpers. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Remove the empty react-native-theoplayer-cavynext stub and replace the local file:../../ paths with a git submodule at e2e/packages/react-native-cavynext. The e2e package.json now points file: deps into the submodule, and package-lock.json was regenerated to match. After cloning, run `git submodule update --init` and build the cavynext packages (`npm run build` in the submodule) so lib/ output exists. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the git submodule under e2e/packages with GitHub branch references in e2e/package.json: - react-native-cavynext -> github:tvanlaerhoven/react-native-cavynext#core-dist - react-native-cavynext-cli -> github:tvanlaerhoven/react-native-cavynext#cli-dist Regenerate package-lock.json and verify `npm install` succeeds; the cavynext package lib/ and cavynext CLI bin are now installed from the dist branches. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Picks up expect/component matchers, by.* selectors, richer interactions, lifecycle hooks and the CLI --json flag. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Specs now use the cavynext expect API instead of the hand-rolled one in utils/Actions (which is removed), drop the manual START/END log boilerplate the runner already provides, use spec.platform() instead of Platform.OS, and assert on received event counts in the ad tests. Web support: a webpack config and index.html under e2e/web, index.web.js / index.test.web.js entry points following the cavynext run-web convention, a web-aware libraryLocation player config, a theoplayer devDependency for the web SDK, and test:e2e:web / build:web scripts. The web bundle builds cleanly with the test app as entry. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- The player under test is now registered in cavynext's TestHookStore by TestableTHEOplayerView (unregistered synchronously on unmount via a layout effect) instead of living in a module-level global. Specs get it through getTestPlayer(spec), which rides cavynext's component lookup. - waitForPlayerEvents rewritten: all listeners (including the ERROR and AD_EVENT watchers) are removed however the wait ends, the timeout is cleared, rejections are real Errors (PlayerEventTimeoutError) carrying a player state snapshot, and unrelated events of an expected type no longer break in-order matching. - New seekTo(player, time) helper; preparePlayerWithSource takes spec. - Platform-specific suites use spec.describeIf: presentation mode tests skip on web (fullscreen needs a user gesture), rendering target tests run on Android only. - Web run: full suite green (28 examples, 0 failures, 12 skipped). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
react-native-cavynext and react-native-cavynext-cli 0.1.0 are on npm now; the github dist-branch references are gone. Full web e2e run green against the npm packages (28 examples, 0 failures, 12 skipped). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Based on develop lockfile with cavy removed and the cavynext npm packages added incrementally, because a from-scratch npm install fails on the react-native override reference (pre-existing on develop). Full web e2e run green on RN 0.86.
New e2e job in pr_web.yml running the suite headless in Chrome, with the markdown summary appended to the job summary. run_web_e2e.sh adds the headless/no-sandbox flags when CI is set. Verified locally in headless mode: 28 examples, 0 failures, 12 skipped. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Nothing imports it; its autolinked native module registers battery and brightness observers that emit through RCTDeviceEventEmitter after the runtime tears down, producing the "Module has not been registered as callable" error at the end of an iOS test run. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The workflows cat cavynext_results.md into the job summary, but the cavynext CLI only writes it when --markdown is passed; the web job already did. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
…dules Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
cavynext 0.2.0 adds describe.each / it.each with label interpolation; the specs now use it instead of wrapping describes in a forEach. The TestSources builder is flattened into plain plainSources(platform) / adSources(platform) functions, making platform-based source selection explicit at the spec level. Web run green (28 examples, 0 failures, 12 skipped). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Install the four analytics connector packages and re-enable Connector.spec in the test set. The comscore web SDK references node builtins behind runtime guards, so webpack resolves os/http/https to nothing. react-native-device-info returns as an explicit devDependency: the adobe connector peer-requires it, and without the native module the app crashes on Android. Verified: web 32 examples / 0 failures / 12 skipped; Android (release) 32 examples / 0 failures. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
NielsenAppSDK-XC, depended on by the nielsen analytics connector, is distributed through Nielsen's own CocoaPods specs repo rather than the CDN. Regenerate the stale, pre-rebase Podfile.lock while at it.
pr_web had no paths filter and ran on every PR update; pr_android and pr_ios triggered on each other's native subtrees under e2e. Web now has a filter, and each platform excludes the other's e2e native folder.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
On a loaded CI machine IMA's ad request can be lost before it is even sent: it runs in a WKWebView whose web content process the runner sometimes takes longer to launch than IMA waits for. Testing the pre-roll once instead of once per source removes the extra ad sessions, and a second attempt covers the remaining flakiness. Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
Co-Authored-By: tom.vanlaerhoven <tom.vanlaerhoven@dolby.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the forked
cavy/cavy-clipackages in the e2e app withreact-native-cavynext/react-native-cavynext-cli(published on npm), modernises the specs, and adds web (react-native-web) e2e support.Test framework migration
cavy/cavy-cli(github forks) →react-native-cavynext/react-native-cavynext-cli^0.1.0from npmexpect(the hand-rolledexpectinutils/Actions.tsis removed)spec.describeIf(...): presentation-mode tests skip on web (fullscreen needs a user gesture), rendering-target tests run on Android only; skips show up in the reportPlayer test layer (
e2e/src/utils)TestHookStorebyTestableTHEOplayerViewinstead of a module-level global; specs obtain it withgetTestPlayer(spec). It is unregistered synchronously on unmount, so a test can never grab the previous, about-to-be-destroyed playergetTestPlayergives a freshly created player a settle time before a spec sets a source. The runner re-mounts the view before every test and cavynextawaits that re-mount, so without it a source lands on a player that is milliseconds old while its predecessor is still being released natively — which made iOS/tvOS fail withMEDIA_AVPLAYER_ERROR(5003) on the mp4 source. Forked cavy hid this behind the 1s poll interval of its player lookup:const player = await spec.findComponent(PLAYER_HOOK_ID); +await new Promise((resolve) => setTimeout(resolve, PLAYER_SETTLE_TIME)); return player;waitForPlayerEventsrewritten: listeners and the timeout are always cleaned up, rejections are realErrors carrying a player-state snapshot, and unrelated events no longer break in-order matchingseekTo(player, time)helper<StrictMode>: its double-invoked mounts created and destroyed a second native player per test, and made React DevTools log every line twiceWeb e2e support
npm run test:e2e:webruns the suite in Chrome viacavynext run-web+ webpack dev server (e2e/web/), launched with a dedicated user-data-dir and throttling disabled so background timers/playback keep runningmutedAutoplay: 'all'on web (browser autoplay policy) and transmux workers copied from the repo root'stheoplayerpackage so SDK and worker versions matchLibrary
THEOplayerAdapterclears its cachedpausedstate onplay, not only onplaying, soplayer.pausedno longer staystruefor the wholeplay→playinggapCI
pr_ios.yml/pr_android.ymlupdated for thecavynextCLI andcavynext_results.mdNielsenTVAppApiagain, without which the tvOS app fails to link (ld: framework 'NielsenAppApi' not found)Test results
Notes
npm installfails on the$react-nativeoverride reference (pre-existing on develop, worth a separate fix)e2e/ios/Podfile.lockwas regenerated withoute2e/scripts/patch_ios_podspec.shhaving run first, so it lacks the IMA/Chromecast/THEOads/Millicast integrations. CI patches the podspec and runspod update, so only a plain localpod installis affectedLink to Devin session: https://dolby.devinenterprise.com/sessions/69e1051f15d743c690a56c0a7bd9018b
Requested by: @tvanlaerhoven