Lifting a Stateful REST API Out of Production and Into Localhost for Integration Tests

Integration testing against stateful REST APIs requires a local simulator that applies writes, preserves state across requests, and enforces the real API’s transition rules. Recording responses is not enough to test create-then-read or mutate-then-delete flows. Moving those flows to localhost makes them safe and reproducible only when cumulative state and continuous fidelity verification replace live dependencies.

Why testing stateful API flows against live endpoints is structurally unsafe

A sequence test against a live API changes the system it tests. Creating a payment or posting a message to a real Slack channel leaves real records and side effects that cannot be discarded like test output. Those effects make unrestricted execution on every commit unsafe. A passing assertion does not undo the operation.

Live endpoints also impose quotas and require credentials. A CI pipeline that calls a third-party API on every PR will eventually encounter throttling or a ban. This interrupts the sequence under test. A create operation may succeed while a subsequent read is throttled, so the result reflects the available rate-limit budget.

Drizz’s testing guide lists non-deterministic data as a main reason to use controlled doubles in place of live calls. When records change, the same test code can give different results from one run to the next. Shared staging compounds the instability because a deploy by one team during another team’s run can cause false failures.

Why replacing a live API with a stateless mock doesn't solve stateful sequence tests

Stateless mocks are appropriate for unit tests and contract assertions. They return controlled responses without a live service. But a multi-step integration test asks a different question, namely whether an earlier operation changes what a later operation observes. A mock without memory cannot establish that relationship.

The diagnostic sequence is simple and revealing. Create a resource, read it, mutate it, and read it again. A stateless mock returns whatever response was configured for the final read, typically the pre-mutation shape. A stateful simulator returns the changed record because the mutation actually updated its store. Consistency between separately configured responses proves only that the fixtures agree.

A stateless mock cannot itself reject a second payment capture, fire a webhook because state changed, or include a newly created resource in a subsequent list response. Drizz’s guide also identifies drift as the biggest risk of API mocking. The mock keeps returning what it was told to return after the real API changes.

How can you do integration testing against stateful APIs locally?

A local simulator that behaves like the real service needs state that persists across requests, rules that enforce its state-machine, realistic seed data, and ongoing checks of its fidelity. These requirements are jointly necessary. Every write must affect later reads, lists, and relational lookups. The simulator must also reject invalid operations, including capturing an already-captured payment or updating a deleted resource, according to the real API’s rules.

Tests for pagination, filters, and relationships need records at startup rather than an empty environment that requires manual preparation. Verification then establishes whether the resulting behavior still matches the external service. Karate Labs’ mock documentation states the risk directly. “If nothing verifies the mock against the real service, your suite is green against a service that no longer exists”.

Fault injection belongs in the same environment. 429 responses and 503 responses that can be configured, along with injected latency, let integration tests check how the application handles failures as well as successful sequences. This capability complements the four requirements. Even when a simulator produces failures that look real, it must keep writes to test cumulative state.

How cumulative state works structurally inside a local simulator

A stateful simulator maintains a live store shared by its request handlers. The router sits above that store. A POST handler validates the body, generates an ID, writes the initial record, and returns the created resource. A read by ID consults the store and returns 404 when the record is absent. List handlers return what is now in the store, including records created since startup.

Mutations operate on those same stored records. PATCH or another update method applies and persists a change, while DELETE removes the record so a later read returns 404. Above the store, a state-machine layer checks whether each transition is allowed. For a payment lifecycle of pending, captured, and refunded, capturing a pending payment succeeds, but capturing an already-captured payment returns the appropriate 4xx response. The current status determines which transition is allowed.

Relationships impose another constraint on those writes. A subscription referencing a nonexistent customer should fail rather than leave an invalid relationship in the store. Karate’s documentation shows this shared-state pattern with variables defined in a Background block that last as long as the server runs. Conditional logic in scenario blocks reads and writes those variables, and the example shows a working CRUD double in a dozen lines. Shared mutable state is the essential mechanism regardless of implementation size.

Wiring the local simulator into a development workflow

For local adoption, the practical target is a single command that starts the simulator with seed data. Packaging it as a Docker image gives developers the same startup environment without per-machine setup state. The image starts the service, and startup loads the seed file. Lists then contain records, and relational lookups have valid anchors. Manual preparation is no longer a prerequisite for each session.

The application-side change is base URL substitution. The SDK or HTTP client points to the simulator on localhost instead of the production endpoint, while its request logic stays the same. Real API credentials should stay out of the local configuration. The simulator instead accepts any token or a configured test token, keeping production keys out of this workflow.

Incremental adoption can use proxy mode for unimplemented endpoints. Karate’s documentation describes karate.proceed() as forwarding traffic while allowing selected endpoints to remain under local control. That allows one API surface to move at a time rather than requiring a complete replacement immediately. But forwarded requests still reach the real service. Proxy mode is therefore a migration mechanism toward the credential-free, isolated workflow provided by fully simulated test sequences.

Lifting the simulator into CI as an ephemeral per-job service

In CI, the simulator should become a disposable service for each job. A Docker Compose service definition can declare the simulator as a named service with a health check. The test job waits for that check to pass before running. This prevents a startup race from appearing as an integration failure. The service is torn down when the job ends.

The job injects the simulator’s base URL through an environment variable that the application already reads. No application code needs to change between local and CI execution. Each service starts with fresh seed data. This isolation keeps records that one PR creates or deletes confined to its own test environment, eliminating cross-run contamination.

Integration tests that use the simulator should run as a gated stage after unit tests pass. For parallel sharding, each shard can receive its own simulator container, with state held inside the container. A subset of runs, nightly or on release branches, should inject latency, rate-limit responses, and service-unavailable errors. Those runs exercise retry and backoff behavior before it reaches production.

How fidelity verification prevents the simulator from becoming the new source of drift

A stateful implementation can still reproduce the wrong behavior. Without ongoing comparison against the real API, it becomes a slow-moving mock that begins accurately and then diverges. Fidelity verification therefore needs two layers. At release time, spec-to-spec diffing compares the implemented OpenAPI specification with the real API’s current specification to detect field additions, removals, and type changes. Automated conformance tests against the live API then check behavior that the specification alone cannot establish.

The verification pipeline begins with golden request-response pairs captured from a real API account. Simulator responses are compared with those fixtures at every nesting level, and automated conformance tests assert structural parity. The checks cover key sets, value types, the shapes of empty collections, and the side effects of mutations. A machine-readable coverage map tracks verification by endpoint. The map shows which endpoints have yet to be verified, even when the suite is green.

Mutation over-serialization occurs when the simulator returns a full object but the real API returns only a sparse subset. A null-versus-absent mismatch occurs when the simulator includes a key with a null value that the real API omits entirely. Neither difference is resolved by making a fixture match the test’s expectation. The verification pipeline must run before every simulator release because fidelity is an ongoing constraint.

Applying the same stateful simulator approach to AI agents that call external APIs

AI agents face the same sequencing problem as conventional integration tests. Their writes must affect later reads, and the actions they choose must be allowed as resource state changes. When agents run on their own, a wrong sequence has greater consequences. Evaluating agents against live services can trigger irreversible changes, consume rate-limit budget, and produce conditions that cannot be reproduced for the next evaluation.

ClawsBench, published in April 2026, has stateful services built to test agents. Its five high-fidelity mock services cover Gmail, Calendar, Docs, Drive, and Slack. Each is a standalone REST API backed by SQLite, so agents can be tested in isolation without real user accounts. Its Slack mock implements 45 endpoints covering channels, messages, reactions, and users, with behavior validated against golden fixtures. This applies the same state-and-fidelity model to an agent benchmark.

AWS ToolSimulator within Strands Evals expresses the shared-state requirement through share_state_id. Tools that interact with the same backend use that identifier so writes become visible to subsequent reads. This is essential for multi-turn workflows such as booking, cart management, or database updates. The evaluation must follow the consequences of the agent’s actions.

Choosing or building a simulator for a specific third-party API

The choice between an existing simulator and a custom implementation depends first on the verification story. An existing simulator with published conformance data is less risky than a homegrown mock that has no verification pipeline. Endpoint availability alone is not sufficient evidence. The relevant criteria are shared state, valid transitions, relational consistency, seed data, and demonstrated agreement with the real API.

Stripe’s documentation states that its APIs are rate-limited and that its frontend interfaces have security measures preventing automated testing. It recommends simulation using mock data. For a create-customer, create-subscription, read-subscription sequence, that simulation needs state. The subscription read must reflect the earlier creation and its customer relationship.

ClawsBench’s Slack implementation includes 45 endpoints and 57 golden fixtures, illustrating the verification work behind that surface. GitHub has community tooling at the npm level but is explicitly excluded from the benchmark’s mock services. Coverage in an agent benchmark and availability in developer tooling are therefore distinct questions.

For an API without an existing verified simulator, the build path is to capture golden fixtures, implement stateful handlers against them, write conformance tests, and automate the pipeline on every simulator change. That work establishes both the local behavior and the evidence supporting it. The decisive acceptance test is causal. A later read must reflect the earlier write because the simulator applied it under the real API’s rules.