Test Methodology and Practice Roadmap¶

1. Problem Statement¶

The Continuous Architecture Platform produces solution designs, API contracts, and implementation guidance — but the delivery practice currently has no defined testing standards. The arc42 templates reference coverage targets and flag test gaps as risks, yet no enforcement mechanism, methodology, or tooling exists. As the platform grows and more services evolve through solution designs, untested changes compound risk across service boundaries.

2. Goals¶

1. Full coverage — Every service meets defined minimum thresholds for unit, branch, and integration test coverage
2. Automated regression — Every PR runs the full regression suite; no merge without green tests
3. Contract testing — Cross-service API boundaries are validated automatically against OpenAPI specs
4. Test-first culture — TDD and/or BDD adopted as standard practice, not an afterthought
5. Traceability — User story acceptance criteria map directly to executable test scenarios

3. Testing Pyramid¶

         ┌─────────┐
         │  E2E    │   Few — critical happy paths only
         ├─────────┤
        │ Contract  │   Per cross-service boundary
        ├───────────┤
       │ Integration │   Service + database + dependencies
       ├─────────────┤
      │    Unit       │   Fast, isolated, high volume
      └───────────────┘

4. Methodology Options¶

4.1 TDD (Test-Driven Development)¶

Write tests before implementation code. Red-green-refactor cycle.

4.2 BDD (Behavior-Driven Development)¶

Write human-readable scenarios (Given/When/Then) derived from acceptance criteria.

4.3 Recommended Hybrid Approach¶

5. Coverage Standards¶

Coverage Tooling¶

6. Contract Testing Strategy¶

Every entry in architecture/metadata/cross-service-calls.yaml represents a contract boundary that requires a contract test.

Consumer Service ──── Pact Contract ──── Provider Service
       │                                        │
       └── Consumer test generates pact ──┐     │
                                          │     │
                                   Pact Broker   │
                                          │     │
                        Provider test verifies ──┘

OpenAPI-Driven Contracts¶

NovaTrek already maintains 19 OpenAPI specs in architecture/specs/. These specs can drive contract test generation:

1. Provider verification — validate that service implementation matches its OpenAPI spec
2. Consumer contract — validate that consumer expectations align with the published spec
3. Breaking change detection — diff OpenAPI specs on PR to flag backward-incompatible changes

7. Integration with Delivery Practice¶

Solution Design Template Changes¶

Add a test plan section to 3.solution/g.guidance/:

3.solution/
  g.guidance/
    implementation-guidance.md   (existing)
    test-plan.md                 (new — required)

Test plan contents: - Which test layers are affected (unit, integration, contract, acceptance) - New test scenarios required - Existing tests that need updating - BDD scenarios derived from user stories (Gherkin format) - Contract test additions for new cross-service integrations

PR Review Checklist Additions¶

• [ ] Test plan included in guidance folder
• [ ] Coverage thresholds met for affected services
• [ ] Contract tests added for new cross-service integrations
• [ ] BDD scenarios correspond to acceptance criteria in user stories
• [ ] No test gaps flagged by mutation testing

CI Pipeline Additions¶

8. Phased Rollout¶

Phase A: Standards and Tooling (Foundation)¶

Phase B: Contract Testing (Cross-Service Safety)¶

Phase C: BDD Adoption (Acceptance Testing)¶

Phase D: Coverage Enforcement (Quality Gates)¶

9. Success Criteria¶

Six months after adoption:

1. Every service has >= 80% line coverage with automated enforcement
2. Every cross-service integration point has a contract test
3. Every solution design includes a test plan in the guidance folder
4. User story acceptance criteria are executable as BDD scenarios
5. No production regression escapes that would have been caught by the defined test layers
6. Mutation testing identifies dead code and weak assertions across all services

10. Related Documents¶