Set a Sales Experiment Stopping Rule

The first two replies arrive for variant A, and the team wants to stop the test. A week later, a different experiment has no clear result, so the team wants to extend it until something looks convincing. Both choices move the finish line in response to the outcome.

Write a stopping rule before starting. It should explain when ordinary observation ends, what requires an immediate pause, and what happens if the available evidence remains inconclusive. It must never require continuing harmful or unauthorized activity for the sake of a clean dataset.

Separate four different reasons to stop

Normal completion means the planned observation period and analysis conditions have been met. A safety stop protects people or prevents an unauthorized action. A data-integrity pause means the records are too unreliable to interpret. A capacity stop means the team cannot responsibly continue the approved process.

These outcomes are not interchangeable. An early safety stop does not establish that the other variant won. A logging failure does not establish that no one responded. A capacity limit should not be disguised as a successful test endpoint.

The Microsoft Research: Patterns of Trustworthy Experimentation, During-Experiment Stage distinguishes monitoring from trustworthy interpretation of early results. Microsoft describes monitoring during experiments while accounting for repeated testing and early peeking in statistical methods. Its large-scale product experiments are not a ready-made sample-size rule for a small sales pilot.

Complete a fictional stopping card

An invented team plans a limited comparison within communications already approved for the intended audience and channel. Its card reads:

• Purpose: learn whether the proposed opening is understandable and produces the predefined substantive reply, not maximize message volume.

• Entry cutoff: no new assignments after Friday in the stated week; no automatic extension.

• Observation rule: each assigned case is followed through the same predeclared fourteen-calendar-day window, with unsent and restricted cases retained separately.

• Normal analysis: after the last included case reaches its observation cutoff and the ledger reconciles.

• Immediate stop: a known restriction is not applied, an unauthorized commitment appears, or the intended audience differs from the approved scope.

• Integrity pause: assignment identifiers or actual sent versions cannot be reconciled.

• Inconclusive result: retain the existing approach or design a new, separately documented test; do not declare equivalence from absence of a clear difference.

The dates and duration are illustrative planning choices, not universal recommendations. A different question may need a different observation window or no external communication at all.

Work through three possible endings

On day three, A has two substantive replies and B has none. The team records the observation but does not declare a winner. Other cases have not completed their windows, and the planned design does not support opportunistic early significance claims.

On day five, a reviewer discovers that a stop request was omitted from a pending task. The affected contact action is stopped immediately. The team checks the scope of the handling problem before deciding what else must pause. It does not wait for the normal analysis date, and it does not send a replacement message to restore balance.

At the planned end, several outcomes cannot be reconciled because a logging step failed. The result is not automatically negative or positive. The report identifies the missing evidence and limits interpretation. Recover only what can be established through authorized existing records; do not contact recipients merely to complete an experiment spreadsheet.

Decide who can pause and who can interpret

Give the operator clear authority to stop the affected action when a defined boundary fails. Assign a separate owner to assess scope, repair records, and approve any resumption. Nobody should need permission to keep a known unsuitable message unsent.

For formal statistical conclusions, obtain a design that accounts for the intended monitoring and stopping behavior. A predeclared date alone does not supply adequate sample size, randomization, reliable measurement, or a valid analysis method. Do not borrow a significance label from software without understanding those assumptions.

Preserve the ending, even when it disappoints

Save the planned rule, actual stop time, reason, evidence available then, and decision made. If a later test changes the endpoint, create a new version and explain why rather than rewriting the original plan.

A good stopping rule protects both recipients and the quality of the conclusion. Sometimes its most useful result is a clear statement that the team learned too little to justify changing course. That is more valuable than a winning variant manufactured by choosing a convenient moment to stop.

Continue with Measure Sales Navigator Prospecting with a Small, Honest Funnel or Compare Two Target Segments with a Fair Sales Experiment.