Integration Testing
Integration Test Scope
Summary
An integration test verifies interactions among components across expected success paths, representative failure conditions, and applicable feature flag states.
Standards
- An integration test MUST verify that two or more components, services, or systems function correctly together.
std-qe-integration-test-scope-01 - Integration test coverage MUST include the expected success path and representative failure conditions.
std-qe-integration-test-scope-02 - Integration testing MUST NOT be substituted for exhaustive coverage of a single component's internal logic, which remains the responsibility of unit-level testing.
std-qe-integration-test-scope-03 - Where a capability under test is controlled by a feature flag and its integration behaviour differs between states, integration test coverage SHOULD include both states.
std-qe-integration-test-scope-04
Related Standards
Implements These Principles
Test Environments & Data
Summary
Each test run uses a fresh real instance for a team-operated dependency or a local emulator periodically verified against its managed cloud service, with repeatably provisioned and reset test data.
Standards
- An integration test MUST NOT depend on a persistent instance of a dependency, whether self-hosted or cloud-hosted, that exists independently of that test suite run.
std-qe-test-environments-data-01 - A dependency the team can run itself, such as a database or message broker, MUST run as a real instance created fresh for each test suite run, for example a container started, seeded with test data, and torn down once that run completes.
std-qe-test-environments-data-02 - A managed cloud-provider service the team does not run itself, such as a message queue or object store, MUST be simulated in routine testing with a local emulator.
std-qe-test-environments-data-03 - A local emulator simulating a cloud-provider service MUST be kept aligned with that service's current behaviour, so it cannot mask a real integration defect.
std-qe-test-environments-data-04 - A local emulator's alignment with the cloud-provider service SHOULD be verified periodically against a temporary real resource.
std-qe-test-environments-data-05 - Test data supporting an integration test SHOULD be provisioned and reset in a repeatable, automated manner, so test outcomes remain consistent across test suite runs.
std-qe-test-environments-data-06
Related Standards
Implements These Principles
External Dependency Simulation
Summary
A dependency outside the team's control is simulated for routine testing, and periodically checked against its real behaviour.
Standards
- Routine, repeated test execution MUST simulate a dependency outside the team's control using a test double or service virtualisation, to avoid the flakiness, cost, and rate limits of calling it directly.
std-qe-external-dependency-simulation-01 - An external dependency simulation MUST be kept aligned with the dependency's current behaviour, so it cannot mask a real integration defect.
std-qe-external-dependency-simulation-02 - An external dependency simulation's alignment SHOULD be verified periodically against the real dependency.
std-qe-external-dependency-simulation-03
Implements These Principles
Contract Testing
Summary
An implementation is tested against its published contract, which is periodically verified through a consumer-authored or consumer-agreed contract test or against real consumers.
Standards
- An API or event/message implementation MUST be tested for conformance against its published, machine-readable contract, so its behaviour cannot silently diverge from what it documents.
std-qe-contract-testing-01 - A published contract SHOULD be verified periodically, such as a scheduled check or before a contract change is released, using a consumer-driven contract test or, where a shared environment makes it practical, directly against real consumers.
std-qe-contract-testing-02 - A consumer-driven contract test MUST be authored or agreed by the consumer it represents, so it reflects how that consumer genuinely uses the contract.
std-qe-contract-testing-03
Related Standards
Implements These Principles
Pipeline Execution
Summary
Integration tests run automatically on every merge request and merge after faster checks, with execution time monitored to preserve feedback speed.
Standards
- Integration tests MUST be automated and executed as part of the delivery pipeline.
std-qe-pipeline-execution-01 - Integration tests MAY be omitted from a push to a branch but MUST run on a merge request and again after it merges to a protected branch, so a branch's development loop stays fast.
std-qe-pipeline-execution-02 - Integration tests SHOULD run after faster, cheaper pipeline checks, such as secrets scanning or unit tests, so a fast-failing issue is caught before the slower integration test stage runs.
std-qe-pipeline-execution-03 - An integration test's execution time SHOULD be monitored, and a sustained increase investigated, so it does not erode the pipeline's overall feedback speed.
std-qe-pipeline-execution-04
Related Standards
Implements These Principles
Integration Test Failures
Summary
A failing integration test blocks progression, and a quarantined test remains tracked until it is remediated.
Standards
- A failing integration test MUST block a change from merging or progressing to the next stage of the delivery pipeline, until the failure is resolved.
std-qe-integration-test-failures-01 - A failing or flaky integration test MUST NOT be silently commented out or disabled to obtain a passing result.
std-qe-integration-test-failures-02 - An integration test deliberately quarantined, whether due to flakiness or another reason, MUST be tracked through remediation.
std-qe-integration-test-failures-03
Related Standards
Implements These Principles
Failure Diagnosis
Summary
An integration test failure identifies exactly which integration point or dependency is responsible, with enough detail to diagnose it.
Standards
- An integration test failure MUST identify the responsible integration point, dependency, or component.
std-qe-failure-diagnosis-01 - An integration test SHOULD capture diagnostic detail at the point of failure, such as the request, response, dependency state, and a correlation or trace ID, so the failure can be diagnosed without reproducing it locally.
std-qe-failure-diagnosis-02
Implements These Principles
Integration Test Maintenance
Summary
Integration test code meets the same coding and review standard as production code, and is updated or removed as behaviour changes.
Standards
- Integration test code MUST be held to the same coding, formatting, and review standards as the production code it verifies.
std-qe-integration-test-maintenance-01 - Integration test code MUST receive substantive review with the same rigor as the production code it accompanies.
std-qe-integration-test-maintenance-02 - An integration test that no longer verifies current behaviour MUST be updated or removed.
std-qe-integration-test-maintenance-03