Run tomorrow’s code against today’s traffic.

You can ask a replacement implementation what it would have done with a real request before allowing it to answer a real visitor.

Consider a WordPress directory whose search engine is being rewritten. The existing implementation returns the public response. A candidate receives an equivalent input and produces a result for comparison. If the candidate disagrees, the visitor still receives the established answer while the engineering team gets evidence about the difference.

This is shadow execution. Its value comes from exposing assumptions that a neat fixture dataset missed: an unusual combination of filters, an old profile with incomplete metadata, or a permission rule that exists in one code path but not the other. Its difficulty is arranging a fair comparison without doubling the application’s side effects or quietly overwhelming its resources.

Choose a boundary you can actually isolate

A search repository with an explicit input and output is a useful starting point. A callback that reads global state, sends email, updates a session and renders HTML is not. Extract a boundary that can be exercised independently before attempting to shadow it.

Define the input contract: normalised filters, pagination cursor, locale, visibility context, relevant configuration version and a stable notion of “now”. Define the output contract separately from the implementation’s internal objects. A database row and a domain object can describe the same teacher without having the same PHP representation.

GitHub’s Scientist library is a useful reference for control-versus-candidate experiments and custom result comparison. The WordPress design here uses the underlying idea; it does not require the Ruby library or claim that arbitrary plugin callbacks are safe to execute twice.

Compare meaning rather than serialised objects

Suppose both implementations find the same twenty teachers, but the candidate places a suspended teacher first. Set equality alone would call that a match. Conversely, a changed internal property order can make serialised objects look different while their public behaviour is identical.

Compare dimensions explicitly: membership of the result set, ordering, pagination boundary, total-count semantics and permission decisions. Normalise fields only when the product contract says the difference is irrelevant. Removing every troublesome field until a comparison passes defeats the purpose.

The following pure function compares a small, illustrative result contract. Each result must provide an ordered list of unique integer IDs, an exact integer total and an allowed/denied boolean. Validate those types at the adapter boundary. If either implementation returns approximate totals or duplicate IDs, define a different contract rather than silently coercing them.

function enjin_compare_search(array $control, array $candidate): array {
    $control_set = $control['ids'];
    $candidate_set = $candidate['ids'];
    sort($control_set, SORT_NUMERIC);
    sort($candidate_set, SORT_NUMERIC);

    return [
        'same_members' => $control_set === $candidate_set,
        'same_order'   => $control['ids'] === $candidate['ids'],
        'same_total'   => $control['total'] === $candidate['total'],
        'same_access'  => $control['allowed'] === $candidate['allowed'],
    ];
}

A mismatch is an observation, not a verdict. The control may contain the bug you are trying to remove. Classify differences against the intended rule and retain a reviewed expectation for deliberate changes. “Matches production” and “is correct” are different claims.

Make both executions observe equivalent facts

If a teacher updates their location between the two queries, a difference may be caused by data movement rather than code. Running the candidate asynchronously widens that window. Capturing the same request parameters does not capture the same world.

There are several possible contracts. Compare both implementations against the same immutable fixture or versioned snapshot. Pass a resolved, read-only domain snapshot to both functions. Or accept that live comparisons are observational, record the relevant versions and classify changes between reads as inconclusive. Select one deliberately.

A shared database transaction is not a universal snapshot solution. Its isolation behaviour is engine-dependent, remote services do not join it, and cached objects may represent different versions from the SQL reads. Long-lived snapshots can also impose operational costs. The experiment should document exactly which state it holds constant.

Choose where the extra work is allowed to run

An in-process candidate shares the PHP worker and its memory budget. Even if you intend to return the control result, a fatal error or exhausted worker can still affect the request. Running a candidate after the response has been flushed does not make the worker free or guarantee that its work will finish.

For more substantial experiments, enqueue a sanitised comparison envelope into a bounded queue and execute it in an isolated worker. That changes the state-consistency problem, so the envelope must reference a reproducible snapshot or explicitly record its observational limits.

Apply a sampling budget, queue-age limit, execution deadline and kill switch. Use stable sampling based on a suitable request key when reproducibility matters; do not persist raw account identifiers just to choose a sample. Drop optional experiments before allowing them to compete with production work. Count those drops so “no mismatches” cannot conceal “nothing ran”.

Prevent writes at the boundary, not by convention

A database rollback does not unsend an email, retract a webhook or erase a remote payment. It also does not rewind a cache mutation outside the transaction. A candidate described as “read-only” may still trigger hooks that have those effects.

Use read-only database credentials where feasible, fake external adapters and network restrictions in the execution environment. Supply a recording implementation for side effects when you want to compare intended behaviour. Treat attempted writes as an experiment failure that needs investigation.

Do not load two versions of the same WordPress plugin into one PHP process: global functions, classes and hooks can collide. A whole-plugin upgrade belongs in an isolated application environment with controlled data and outbound effects. Replay then becomes a separate form of testing, with its own fixtures, configuration and version inventory.

Keep useful evidence without collecting a second customer database

A comparison record can contain experiment version, input shape, relevant state versions, mismatch category and execution outcome. Store only the smallest payload needed to reproduce a failure. Search text, account state and hidden results may contain information that does not belong in general application logs.

Restrict access, set a retention period and redact before data leaves the request boundary. Where an identifier is required for correlation, choose an appropriately scoped pseudonym and remember that pseudonymised data still needs protection. Sanitise error messages too; database exceptions can include values from the query.

Use the experiment to earn a controlled cutover

Before enabling the candidate, decide which mismatches are unacceptable and what coverage is required. An access-control disagreement deserves a different response from an approved change to alphabetical tie-breaking. Inspect rare journeys deliberately rather than hoping random traffic will include them.

Progress from fixtures to sampled observation, then to a small controlled rollout if the evidence supports it. Keep the old implementation available only while rollback remains compatible with the current data model. For schema changes, pair the experiment with an explicit migration and rollback plan.

The outcome is a better basis for a release decision: known behaviours, understood differences and a tested recovery path. A shadow experiment supplies evidence. It does not grant permission to skip the engineering judgment that interprets it.

Code examples illustrate the architecture and identify application-specific interfaces. Adapt them to your environment and test the failure paths before deployment. Sources are linked alongside the relevant discussion.