Contract Testing vs Integration Testing for Third-Party APIs

Teams that depend on APIs routinely ship code that passes every test they own and still breaks in production. The reason is not neglect. It is that the failures come from two fundamentally different sources, and no single testing strategy catches both.

The first failure mode is drift in the contract between services. You can rename a field, change a status code, or add a required parameter, and downstream consumers break while the provider's own tests still pass. Within two hours a reconciliation job spanning three downstream services began failing silently, and no one noticed until finance reported a mismatch the next morning.

The second failure mode is behavioral. Business logic spread across services behaves incorrectly at runtime: race conditions, transaction rollbacks, timing issues, discount logic that breaks only when a coupon and a loyalty tier stack in a specific order. No contract document can express any of this. The tests report green because they are checking whether two specific deployed versions tolerate each other that day, not whether the contract is still honored.

What contract testing actually checks, and what it deliberately ignores

Contract testing verifies that two services agree on the shape of their communication without either running the other's actual code. It catches breaking API changes with speed and precision. It has zero opinion on whether the business logic behind that API is correct, and that boundary is the hinge on which the entire argument turns.

The dominant model is consumer-driven. The consumer writes tests that exercise its API client against a mock, recording the exact requests it sends and the response fields it reads. The contract document captures only what the consumer actually requires. A consumer that reads only id and email generates a contract requiring those two fields and nothing else, which means the provider can add, rename, or remove fields no consumer contract references without breaking anything. The provider then runs verification against that contract using its own test suite, in isolation, with no consumer code involved. If the provider's real response does not match, verification fails in the provider's build before anything ships.

Pact is the most widely adopted consumer-driven contract testing framework, with official libraries for Node.js, Java, Python, Go,.NET, Ruby, and more than ten languages total, including Rust, PHP, Swift, and Scala. The workflow moves from consumer test to contract file generation, then contract publishing to a Pact Broker, then provider verification. Provider-driven contracts work in the opposite direction: the provider publishes an API specification, typically an OpenAPI document, and consumers validate that their client code conforms to that spec as it evolves. Organizations often use OpenAPI-based provider-driven contracts for third-party integrations and Pact-based consumer-driven contracts for internal service-to-service communication.

What contract testing cannot do is worth stating precisely. It has no view on whether that number is correct. It tests single interactions, not full flows, so race conditions, rollbacks, and multi-step business logic live entirely outside its scope. Contract tests run in isolation against mocks or stubs, not real production-like data or live dependencies. Those stubs often lag the real API or lack the stateful behavior that cumulative calls depend on. But even that does not turn a contract test into an integration test.

What integration testing actually checks, and what it costs

Integration testing is the only approach that can find bugs living in the interaction itself rather than the interface. It exists to cover the business logic, timing behavior, and real end-to-end flows that contracts cannot express. A real integration test creates an order through the API, confirms it lands in the database with the right status, confirms the payment service returns a charge ID, confirms the notification service sends a confirmation email, and confirms the order status updates correctly. The whole chain runs as a single verifiable scenario. Race conditions, transaction rollbacks, and timing issues are only catchable here. So is the discount logic that breaks when a coupon and a loyalty tier stack in a particular order, because a contract has no concept of what a coupon is.

The costs scale badly with architecture size. The environment must be provisioned, seeded with data, and kept synchronized with recent code from multiple teams. A failure in the shared environment can block every team at once, and diagnosing which service introduced the breakage means examining logs across multiple systems. Tests run in minutes rather than seconds. They are environment-fragile: a test passes locally and fails in CI because staging runs a different service version. At fifteen or more they begin consuming whole engineering afternoons.

Integration testing requires real or realistic infrastructure, which for third-party APIs means a staging environment, live test credentials, and actual database state. That translates into burning rate limits and managing secrets. The tests still occupy the middle layer of the testing pyramid, slower than unit tests and more complex to set up than contract checks, but their failures mean something different: the shape held, and the behavior did not.

The specific failure modes each approach cannot catch on its own

The blind spots are structural, not incidental. Using one approach as a substitute for the other guarantees that a category of failures goes undetected. Contract testing misses behavioral correctness by design. It cannot simulate asynchronous choreography: a payment intent initiated synchronously followed by a webhook delivered seconds later that fulfills the order. For third-party APIs the gap is more acute: the provider cannot run Pact verification at all, so a consumer contract validates that the team's client code correctly implements expected API usage but cannot verify that the third-party service actually behaves that way at runtime.

Integration testing misses schema drift that does not break the current deployed pair of versions. Two compatible versions get along in staging while the contract underneath has already drifted, and no integration test checks whether the contract is still honored in the abstract. A renamed field, an altered data type, a modified response structure can pass functional testing and still break downstream applications, because integration tests check what came out of a specific run rather than whether the agreed interface remains intact. Integration tests also erode independent deployability. They force coordination, or they run against stale versions and report false confidence. With a contract test each side can verify against the latest published contracts without blocking on the other team's deployment schedule.

How the two strategies fit into a layered CI/CD pipeline

Positioning contract tests and integration tests at different pipeline stages is not a compromise. It is the design that lets each one do its best work without paying the cost of the other. Contract tests belong at the pre-commit or pull-request stage, fast enough to complete before a developer moves on. Integration testing produces no equivalent portable, queryable compatibility record. Catching breaking changes this early reduces dependence on fragile end-to-end testing and lets teams release independently with confidence.

Integration tests run at pull request or post-merge. That sequencing means failures at this stage point to behavioral problems rather than interface drift. Full regression follows post-merge, with synthetic monitors post-deployment covering the end-to-end flows no earlier stage can reach.

Two operational hazards deserve explicit attention. Second, never let a mock reach a production-like environment. Scope it to the test job by URL rather than by a global configuration someone can forget to switch.

Why third-party APIs break the standard contract testing model, and what to do about it

Third-party APIs are the hardest case for both strategies because the provider cannot run Pact verification and the team cannot control what the live API does between releases. Drift and behavioral divergence are ongoing operational risks, not one-time setup problems. The provider-side half of the Pact workflow requires the provider to run verification against consumer contracts using its own test suite. Stripe, GitHub, and Slack will not do this. The limitation is unavoidable: a consumer contract against a third-party API confirms the team's assumptions but cannot confirm that the third party still honors them after a provider-side change.

The drift risk is documented by the providers themselves. So integration tests against PayPal's sandbox are not a reliable proxy for production behavior. For Slack, an automated drift check issued September 28, 2026 against the slack-web-api-client repository carried forward a previously identified gap versus the Java SDK, including blocks.validate missing from api-client.ts and request types, flagged as low confidence and possibly experimental.

Drift monitoring tools fill part of this gap. API Drift Sentinel monitors OpenAPI descriptions and developer docs from external providers, catches upstream API changes before they break production, extracts deprecation deadlines, maps changes to owners and internal services, and runs proof checks. These tools catch schema-level changes. They do not simulate behavioral fidelity or replace an integration test against a running environment. Drift monitoring is the contract-testing equivalent for third-party APIs, and it is an early warning layer, not a substitute for the integration suite.

Why stateful simulation is what makes integration testing tractable for third-party APIs

The reason integration testing against third-party APIs is slow, fragile, and expensive in practice is not that integration testing is conceptually flawed. It is that the live API is the wrong environment to run those tests against. A stateful simulator that remembers the cumulative effect of sequential calls is the correct stand-in. A stateless stub that returns the same order on every GET /orders/42 cannot express the flow that actually matters: create an order, fetch it, cancel it, fetch it again and see a different status. Live sandboxes have the state but bring rate limits, live credentials, no offline operation, and drift from production as Stripe and PayPal's own documentation confirms.

A stateful simulator provides what neither mocks nor live sandboxes do. It runs locally or in CI without touching rate limits or live credentials, so it removes two of the three major reasons teams avoid running integration tests frequently. And it supports fault injection, latency, 429s, 503s, as a first-class concern, so resilience testing happens in the same environment as functional integration testing rather than requiring a separate infrastructure chaos tool.

Rystic builds stateful simulators for Slack, GitHub, Stripe, Kalshi, PayPal, Lob, Okta, Linear, Resend, and others, each continuously verified against the real API before every release. For teams building reinforcement learning environments or evaluating AI agents, those same simulators are deterministic and horizontally scalable across containers, properties that make them right for CI pipelines and for agent evaluation at scale. The test design ideal, stateless by default and stateful by design, is directly supported: most tests run against the simulator's stateless surface, while complex multi-step workflows use isolated, seeded state without serial-mode coordination overhead.

Fault injection and resilience testing as integration testing's third dimension

Integration testing against third-party APIs is incomplete if it only covers the happy path. The failure modes that matter most, rate limit bursts, asynchronous event disorder, duplicate webhook delivery, are not edge cases. They are documented behaviors of the APIs themselves.

PayPal requires an HTTP 2xx acknowledgment for each webhook and retries failed deliveries up to 25 times over three days. Tools that operate at the HTTP, OS, or network layer can drop packets or add latency, but they cannot reach individual response body fields, which is exactly where silent semantic corruption occurs. For AI agents calling external APIs, the most damaging failures return a perfectly normal HTTP 200 with a semantically corrupted payload, and infrastructure chaos tools cannot inject that.

Fault injection belongs in the same stateful simulator environment as functional integration testing. These are not enhancements to the testing strategy. They are the third dimension of it, and without them the integration suite is testing only the paths that work.

How to decide which gaps each strategy fills in your pipeline

The question is not whether to use contract testing or integration testing. It is which failure modes are currently undetected, and which approach closes that specific gap first. Contract testing is the urgent gap when integration failures surface only when two services run together in staging, by which point the breakage is expensive to diagnose and fix. It is urgent when field-level API changes, renamed keys, altered data types, removed fields, reach production undetected. It is urgent when a provider team cannot run consumer contracts against their own build, in their own pipeline, before a consumer ever sees the change.

Integration testing is the urgent gap when the team has no way to run a realistic multi-step API workflow, create, update, cancel, re-fetch, in CI without hitting the live API or standing up a shared environment. It is urgent when fault scenarios, rate limits, webhook disorder, partial failures, go untested because the live sandbox does not support injecting them reliably. It is urgent when tests against the current mock pass but the mock has not been verified against the real API recently, because the simulator's behavior may have drifted from the provider's.

For most teams integrating multiple third-party APIs, both gaps are present simultaneously. Drift monitoring tools such as DriftLock and API Drift Sentinel provide an ongoing early-warning layer between the two, so they catch provider-side changes before the next integration test run surfaces them as failures. Catching both failure modes means testing at two different levels, which requires infrastructure that supports fast, isolated contract checks and realistic stateful end-to-end flows without the overhead of live APIs. Contract, functional, and API automation belong in one unified pipeline. Engineers who know what each one cannot do are the ones who build pipelines where failures are caught at the cheapest possible moment.