Skip to content

Align test definitions with the Testing Guidelines structure and the published API #34

Description

@hdamker

Problem description

The code/Test_definitions/ folder contains six .feature files that fall into two groups with different structure and different referenced API models, measured against the CAMARA API Testing Guidelines (r3.4) and the published code/API_definitions/multi-point-vpn.yaml.

Group A — multi-point-vpn.feature (aligned)

This file is consistent with the API definition and the Testing Guidelines:

  • Filename follows the single-feature-file convention (api-name), Testing Guidelines "Location of feature files".
  • Background section present, referencing a valid access token and the required scope.
  • Scenario tags follow the @<feature_identifier>_<number>_<detail> format (e.g. @createNetwork_01_success, Testing Guidelines "scenario identifier").
  • Scenarios reference the actual paths (/network, /network/{serviceId}, /assessment), operationIds, schemas (NetworkCreate, NetworkUpdate, AssessmentRequest), and CAMARA error codes (INVALID_ARGUMENT, UNAUTHENTICATED), with success, 400, 401 (and a 404 for GET) scenarios for all five operations.

Group B — creation.feature, modification.feature, query.feature, deletion.feature, assessment.feature

These five files do not follow the Testing Guidelines structure and reference an API model that is not present in multi-point-vpn.yaml:

  • Filenames use action words rather than the <api-name>-<operationId>.feature pattern recommended for per-operation splits (Testing Guidelines "Location of feature files").
  • Scenario tags are free-form without the numeric identifier component, e.g. @VPN_from_assessment_provisioning, @Scale_bandwidth_1000, @Get_VPN_ID, @Delete_non-existent_VPN.
  • Paths differ from the API: /vpns, /vpns/{vpnId}, /assessments (the API defines /network, /network/{serviceId}, /assessment).
  • Identifier differs: vpnId (the API uses serviceId).
  • They reference concepts not defined in the API: topology "hub-and-spoke", x-idempotency-key, lifecycle status values (provisioning, active, reconfiguring, terminating, deleted), assessmentId, availabilityLevel, estimatedFees, tenantId, and list/pagination queries (GET /vpns?status=active&tenantId=...).
  • Scopes used are vpn:read / vpn:write (not defined in the API and not in the CAMARA <api-name>:... scope convention).
  • Error references are lower-case (e.g. error.code "invalid_sla_tier"), where the API uses upper-case CAMARA codes.

Expected action

  • Decide the feature-file granularity (single file vs. one file per operation) and apply the Testing Guidelines filename and scenario-identifier conventions consistently.
  • Reconcile the two groups: the Group B scenarios reference an API model that does not match the published definition. Either rework them against the actual API (paths, identifiers, schemas, scopes, error codes) or remove them, keeping multi-point-vpn.feature as the aligned base.
  • Ensure each operation's mandatory scenarios (success and the documented error responses) are covered once, without conflicting duplicates.

Additional context

The scenarios themselves provide useful coverage of the intended operations; the issue is structural consistency and alignment with the published API definition. Line/section references are by filename within code/Test_definitions/.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions