Monadic Testing vs. Sequential Monadic: Examples & Sample Size
A fictional AB/BA comparison shows how a 52% versus 50% result can hide order differences, and why paired analysis needs linked answers.
Monadic testing gives each buyer one complete offer, keeping the first reaction separate from exposure to other concepts. Sequential monadic testing asks each person about several offers, one at a time. The fictional A/B results below show why a two-point pooled lead may be less useful than the difference between first and second position.
Sample size follows the comparison
Plan valid responses per concept and per decision-critical group. If two independent concepts each require n valid answers, the total is 2n before allowance for exclusions; dividing a fixed total among more concepts lowers each cell’s precision. Set the effect worth detecting, expected response rate, comparison method and uncertainty requirement with the analyst before fixing n. A sequential design reuses respondents, so its paired analysis and order checks need a different plan. A count of modeled profiles does not inherit a human-sample margin of error.
Exposure starts with the assignment
The worked example uses two unnamed product offers, A and B, for eligible category buyers. The team needs to decide which offer deserves further development and whether reactions collected in sequence can support that decision. AB and BA identify the exposure order; A and B identify the offers.
Design | Details |
|---|---|
Monadic | Example allocation: Half assigned A, half B Record: One concept and one evaluation per person |
Sequential | Example allocation: Half assigned AB, half BA Record: Position and evaluation for both concepts |
For independent appeal, randomize eligible people to A or B and show one card. Keep the buyer rule, presentation and question block comparable. For a sequence, assign AB or BA and preserve position for both. Plan any critical audience balance within the allocation rather than adjusting mid-fieldwork.
Save planned assignment separately from actual display. A person assigned A who receives B because of a routing error has not supplied the intended exposure. A record of both lets the analyst diagnose the issue instead of relying on the group label.
The unit is the person for independent cells. In a sequential study, two concept evaluations remain linked to one person. Preserving that link is necessary to distinguish independent first-position comparisons from paired concept comparisons.
Question wording and product versions stay fixed
Use the same question set after each card: unaided comprehension, relevance, concerns, intent at the displayed price and alternatives. Keep the outcome wording, timeframe and response labels identical. Decide which intent answers count as positive before reading the results, and keep the full distribution.
Pilot the cards and block as one experience. A missing fact should be clarified consistently in the study version. Giving extra explanation only to a confused second card changes the comparison unless that presentation is deliberately the intervention.
Qualtrics’ concept program guidance documents single and sequential setups. Agree whether the main result should reflect a first reaction or an evaluation made after another concept.
Read the AB/BA comparison before naming a winner
Sequence | Details |
|---|---|
AB | First concept: positive: A: 30 of 50 (60%) Second concept: positive: B: 21 of 50 (42%) |
BA | First concept: positive: B: 29 of 50 (58%) Second concept: positive: A: 22 of 50 (44%) |
Our four aggregate records and copyable CSV contain counts for 100 people, 50 in each sequence. The same predeclared positive-intent outcome applies throughout.
All four aggregate records
Each row has evidence_type=fictional_teaching_data. The n is the concept-position base; positive is the positive-intent count; positive_share is that count divided by the row’s base.
Sequence / concept / position | Aggregate values |
|---|---|
AB / A / 1 | n: 50 positive: 30 positive_share: 0.60 |
AB / B / 2 | n: 50 positive: 21 positive_share: 0.42 |
BA / B / 1 | n: 50 positive: 29 positive_share: 0.58 |
BA / A / 2 | n: 50 positive: 22 positive_share: 0.44 |
Copy the complete concept-order CSV
sequence,concept,position,n,positive,positive_share,evidence_type
AB,A,1,50,30,0.60,fictional_teaching_data
AB,B,2,50,21,0.42,fictional_teaching_data
BA,B,1,50,29,0.58,fictional_teaching_data
BA,A,2,50,22,0.44,fictional_teaching_data
The position pattern is larger than the pooled lead
Positive intent by concept and exposure position
Concept | Details |
|---|---|
A | First: 60% Second: 44% Change: −16 points |
B | First: 58% Second: 42% Change: −16 points |
Pooled: A 52% · B 50% · 100 people total
Aggregate counts: 50 AB and 50 BA, 100 people total. First/second rates compare sequence groups, not reconstructed person-level transitions.
A has 30 positive first-position answers and 22 second-position answers, pooling to 52 of 100. B has 29 and 21, pooling to 50. Yet A’s rates are 60% and 44%; B’s are 58% and 42%. Both concepts’ second-position shares are 16 percentage points lower than their first-position shares.
For a decision about independent appeal, that position pattern needs attention before the pooled two-point lead. Fatigue, anchoring and comparison context are possible explanations; the aggregate records cannot identify which occurred. They also cannot reconstruct how an individual changed their answer.
The follow-up depends on the uncertainty
For this comparison, do not develop A solely because 52% exceeds 50%. Inspect survey length, instructions and first-exposure understanding. If the decision still depends on independent appeal, run a focused one-concept-per-person follow-up on the remaining viable offers.
A declared first-exposure analysis can be useful. It compares A-first AB people with B-first BA people, rather than paired answers from one person. Choose it for the research purpose and design, not retrospectively because it favors the team’s preferred concept.
If one concept is confusing only when it is shown second, check both its wording and what the first card may have taught people. A concept issue and an exposure issue can coexist. The next study should resolve the material uncertainty, rather than repeat the entire screen without a new question.
A stopped block is missing evidence
If someone completes A and stops before B, keep A and record the missing B exposure or outcome. Do not code it as negative intent. Uneven loss by sequence can create a later-position pattern as well as change the achieved comparison.
Report assigned, exposed and valid-answer counts by sequence and position. Separate a display failure from a skipped optional item and an actual nonpositive response. The reason determines what evidence is available.
Plan the primary analysis and any check limited to people who answered both concepts before fieldwork. That complete-pair base may differ from the first-exposure base. Show the change instead of suggesting that the same group supplied every summary.
Paired comparisons need joint outcomes
Save linked outcomes with the identifiers needed to retrieve each answer and display.
participant_id / sequence | Details |
|---|---|
P01 / AB | concept / position: A / 1 response_id, exposure_id, stimulus_version and positive_intent: P01-QA; P01-E1; A-v1; positive_intent=1 |
P01 / AB | concept / position: B / 2 response_id, exposure_id, stimulus_version and positive_intent: P01-QB; P01-E2; B-v1; positive_intent=0 |
P02 / BA | concept / position: B / 1 response_id, exposure_id, stimulus_version and positive_intent: P02-QB; P02-E1; B-v1; positive_intent=1 |
P02 / BA | concept / position: A / 2 response_id, exposure_id, stimulus_version and positive_intent: P02-QA; P02-E2; A-v1; positive_intent=1 |
These four example rows show the record structure; they do not reconstruct the 100-person aggregate records. Participant ID links outcomes, response ID identifies the answer, exposure ID identifies the display and position records order. Here 1 means definitely/probably would buy, while 0 is another valid intent response. Missing and not applicable stay separate.
For a paired comparison, retain positive to both, A only, B only and neither. The A-only and B-only states determine the within-person disagreement. The two pooled margins do not reveal those joint states.
A mathematical bound makes that missing information concrete. With 52 A-positive and 50 B-positive among 100, the number positive to both could range from 2 to 50. At the first endpoint, 50 are A-only and 48 B-only; at the second, 2 are A-only and none B-only. Both joint arrangements match the same margins. These are possible tables, not reconstructed study outcomes.
The total discordant count therefore could be 98 or 2 at those extremes, despite the identical two-point marginal difference. A paired uncertainty calculation depends on that information. The aggregate records support descriptive totals and position patterns, while leaving that inference unavailable.
These bounds show why the real linked outcomes are necessary; they aren't alternate results to choose from. Preserve the actual links. For a modeled study, keep the run identifier, how the run was built and its relationship to other outputs. A model-run identifier is distinct from a human participant.
The recommendation depends on what people saw
A useful report states which concept to progress or revise, the supporting independent or sequential outcome, and the exposure issue that affects the decision. It should let another reviewer retrieve the card, block, base and follow-up.
For human studies, sample structure and related responses affect uncertainty. For modeled responses, assignment helps you inspect sensitivity; it doesn't create a human sample. Confirm how the audience was built and validated for the task.
In this A/B comparison, an independent follow-up is justified if the development decision still rests on first reactions. It should resolve that uncertainty, rather than rescue A's pooled lead. Our concept examples guide shows what the resulting evidence should change.
Further reading
If the next investment depends on first impressions, discuss an independent concept comparison with Vensia and agree how the modeled audience and exposure should be checked.
