Repository navigation
Description
Several test files construct synthetic [PSCustomObject] or class instances to test behavior that should instead be validated against real API responses. Fabricated data can drift from what the API actually returns, hiding bugs rather than catching them.
Request
Review and replace mocked data patterns in the test suite with real API calls wherever the test is verifying API-related behavior. Tests that validate pure local logic (byte conversion, crypto signatures, file output) are acceptable as unit tests and do not need API calls.
Acceptance criteria
- Tests that validate API response structures call the actual API instead of fabricating response objects
- Pure logic unit tests (formatters, crypto, file I/O) are left alone or explicitly documented as intentional unit tests
- No silent regressions introduced by the migration
Technical decisions
Scope: Only tests that fabricate API-like response objects need conversion. Tests for pure local logic (no API involved) are acceptable as-is.
Test categorization:
| File | Test | Pattern | Action |
|---|---|---|---|
tests/GitHub.Tests.ps1 |
GitHubCustomProperty (3 tests) |
Fabricated [PSCustomObject] simulating API response |
Fixed in #577 — replaced with integration test in Repositories.Tests.ps1 |
tests/Permissions.Tests.ps1 |
GitHubPermission Class |
Direct class instantiation from fabricated hash | Review — consider validating against actual permission objects from Get-GitHubContext |
tests/Apps.Tests.ps1 |
[GitHubPermission]::NewPermissionList() |
Class factory method generating static catalog | Acceptable — this is testing the static catalog itself, not an API response |
tests/GitHubFormatter.Tests.ps1 |
FormatFileSize conversion tests |
Synthetic byte values for conversion logic | Acceptable — pure math/formatting logic with no API equivalent |
tests/GitHub.Tests.ps1 |
Set-GitHubOutput - Object |
Fabricated complex object for output serialization | Acceptable — tests local file-writing logic, not an API call |
tests/GitHub.Tests.ps1 |
Test-GitHubWebhookSignature |
Fabricated request object with headers | Acceptable — tests local HMAC validation, not an API call |
Remaining work: The GitHubPermission class test in Permissions.Tests.ps1 constructs a permission object from a hashtable. This should verify that permissions returned by actual API contexts match the expected structure.
Implementation plan
- Remove
GitHubCustomPropertymocked tests fromGitHub.Tests.ps1(done in 🪲 [Fix]: Multi-select custom properties no longer lose individual values #577) - Add
Get-GitHubRepositoryCustomPropertyintegration test inRepositories.Tests.ps1(done in 🪲 [Fix]: Multi-select custom properties no longer lose individual values #577) - Review
GitHubPermissionclass test inPermissions.Tests.ps1— determine if it should validate against real permission objects from an authenticated context - Add explicit comments to acceptable unit tests documenting why they use synthetic data (e.g.,
# Unit test: validates local HMAC logic, no API call)
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
MariusStorhaug commentedon May 2, 2026
Heads up: when this work changes how tests provision, share, or tear down infrastructure, update
.github/instructions/tests.instructions.mdin the same PR. That file is the single source of truth for test conventions and is auto-loaded by Copilot for every edit undertests/**, so contributors and the agent stay aligned.