Separate Assigned and Exposed Experiment Records

An account assigned to message B has not necessarily received message B. The draft may be rejected, a contact restriction may appear, or the account owner may already be speaking with the company. If the report quietly removes these cases, the comparison can become a contest between the easiest records left in each group.

Preserve the original assignment and describe what happened afterward. This is an accounting discipline for a bounded, appropriately authorized experiment, not permission to send a message merely because a row was selected.

Name the state that your evidence actually supports

Use separate fields for assigned variant, approved version, send status, delivery evidence, and outcome. A draft saved in a queue is not a send. A send confirmation is not proof the intended person read it. An automatic response is not necessarily a substantive business reply.

Define exposure for the question you are studying. If you can observe only a confirmed send, call the measure confirmed sends rather than messages read. Avoid introducing invasive tracking just to make the dashboard look more complete. Unknown attention is a legitimate limitation.

The relevant principle in NIST Engineering Statistics Handbook: Completely randomized designs is that treatment levels are assigned to experimental units. Assignment is therefore a design fact worth preserving even when later execution differs. The ledger below does not by itself establish a causal result.

A fictional assignment ledger

Suppose an internal pilot assigns twenty-four eligible account records equally between variants A and B. Each account has one proposed communication, subject to the same final checks. All counts are illustrative.

For A, twelve records were assigned. Nine reached confirmed send, two were held after newly discovered contact restrictions, and one remained unsent because the owner had already opened a relevant conversation. For B, twelve were assigned. Six reached confirmed send, four needed evidence corrections, one was held for a restriction, and one remained unsent during an owner absence.

Three A accounts and three B accounts later produced the defined substantive reply within the observation window. The report contains both views:

• Assigned-account view: A has three observed qualifying replies among twelve assignments; B also has three among twelve.

• Confirmed-send view: A has three among nine confirmed sends; B has three among six.

• Execution view: A has three unsent records; B has six, with reasons retained.

The confirmed-send percentages differ, but the people who reached sending were selected by different intervening events. That view should not be presented as proof that B's wording caused a better response. B's evidence-correction burden is itself an important process finding.

Preserve restrictions without rewriting history

The two A restrictions and one B restriction must stop the relevant contact action. Do not send to equalize the groups. Keep the assignment as historical context and record why execution stopped, without storing unnecessary personal detail.

Likewise, do not move an unsent B record into A after seeing which wording seems promising. A legitimate reassignment requires a documented rule and creates a protocol deviation. The original assignment should remain available for review rather than being overwritten.

If a person is accidentally sent the wrong variant, record assigned B, actually sent A, and the incident reason. Do not make the ledger look consistent by changing the assigned field. Which analysis is appropriate depends on the experiment design and assumptions; seek statistical expertise for consequential causal claims.

Reconcile before interpreting

Check that every assigned record appears exactly once in the final status accounting. Confirm that send records map to the right account and version. Investigate duplicate identifiers, unmatched events, and a group with unexpectedly many missing logs.

Separate unfinished observation from a completed window without the target event. Preserve the observation cutoff and the event definition. Otherwise, one group may receive extra time to generate a reply while appearing to share the same denominator.

End the report with what the ledger supports: the actual allocation, execution barriers, observed outcomes, and unresolved measurement gaps. In the fictional pilot, the useful next step is to understand why B required more evidence correction, not to announce a winning message or bypass the checks that prevented unsuitable sends.

Continue with Measure Sales Navigator Prospecting with a Small, Honest Funnel or Compare Two Target Segments with a Fair Sales Experiment.