Maze alternatives: preserve the meaning of your task results
The same ten attempts produce 87.5% or 50% depending on how help and stopped tasks are counted. Use the delivery-address example to test a Maze replacement.
UserTesting, Lookback and Userlytics are alternatives to Maze for recorded task review, live observation and prototype-URL tasks with screeners. When migrating a delivery-address task, your team needs the same attempts to receive the same outcome codes. Reproduce that measure before attributing a changed score to a redesigned interface.
Why 87.5% can describe a 50% unassisted result
In this illustrative set of ten eligible attempts, five succeed without help, two finish with assistance, one reaches the wrong end state and two stop. A report that counts help as success and omits stopped tasks shows 7/8, or 87.5%. The unassisted measure among all eligible attempts is 5/10, or 50%.
A dashboard change can look like a usability change
The same 10 eligible attempts
Outcome category | Count |
|---|---|
Unassisted success | 5 |
Assisted completion | 2 |
Wrong end state | 1 |
Stopped | 2 |
Old report: 7 / 8 = 87.5%
Counts help as success; omits stopped tasks
Unassisted rule: 5 / 10 = 50%
Retains all eligible attempts
Nothing about the interface or participants changed. Two reporting decisions changed the numerator and denominator. Before evaluating a platform migration, define the target measure and make both outputs reproduce it. The difference above demonstrates a measurement problem; it supplies no evidence that one vendor improves usability.
Rule to align | Preserved rule and bridge error |
|---|---|
Success state | What to preserve: A defined address change for the specified delivery Mistake the bridge exposes: Reaching another screen becomes success |
Assistance | What to preserve: Whether and when the moderator supplied help Mistake the bridge exposes: Assisted completion becomes unassisted success |
Task denominator | What to preserve: All eligible attempts, with exclusions justified Mistake the bridge exposes: Stopped tasks disappear |
Timing | What to preserve: The same start/stop points and treatment of incompletion Mistake the bridge exposes: A successful-task average looks like everyone finished quickly |
Specify changing the next delivery address
Use the neutral prompt: “Your next delivery should go to a different address. Show how you would change it for that delivery.” Do not name the control when finding it is part of the task. Define success as the intended address being saved for the specified delivery, not a different screen merely being reached.
Set assistance, timing and stopping rules before the pilot. Keep a wrong end state distinct from a stopped attempt; each suggests a different diagnostic question. Preserve eligible incompletions in the task denominator unless a prespecified eligibility or technical exclusion applies.
Field | Definition |
|---|---|
Participant ID | One identifier per recruited person |
Stimulus version | Prototype or site version and device |
Task result | Unassisted success / assisted completion / wrong end state / stopped |
Start and stop rule | Explicit points for any timing measure |
Path evidence | Recording or interaction trace |
Failure reason | Researcher code supported by the evidence |
Participant explanation | Their stated account, kept separate from the code |
Structured metrics and recordings have different uses
Maze’s prototype page documents completion, misclick, duration and mission-path outputs. A recording can support manual coding, but it does not automatically supply the same structured metrics. Determine whether your team needs a repeatable tracking measure, a detailed explanation, or both.
Option | Best-fit job and consequential limit |
|---|---|
Concrete advantage / best-fit job: Screen/audio video and clickstream data let you examine task behavior beside think-aloud explanations Limitation that changes the choice: Confirm the exact trace/export fields and coding; advertised interaction evidence does not establish Maze-equivalent scoring | |
Concrete advantage / best-fit job: Live observation with exportable recordings and transcripts supports researcher coding of an episode Limitation that changes the choice: Session exports do not promise automated mission-path metrics; include manual coding work in the migration cost | |
Concrete advantage / best-fit job: Prototype URLs and behavioral screeners support a specified stimulus and audience Limitation that changes the choice: Verify prototype permissions, device behavior and record-level results; broad support does not prove your particular version |
Confirm the trace fields, exports and prototype behavior on the plan your team would retain. Use the pilot’s recovered records to decide whether that configuration can replace the workflow.
The first pilot keeps the prototype fixed
Check links, overlays, back navigation and mobile presentation on the exact version. Retrieve an unassisted completion, assisted completion and incomplete attempt from the candidate. Have another researcher recover the codes using your rules. Confirm that the old and new reports agree on the eligible base.
Hold task, stimulus, audience definition and coding steady for the first bridge. Changing the interface at the same time makes a score movement difficult to attribute. Preserve the version and export mapping beside the migration result, then retest product changes separately.
Manual coding changes the operating cost
A small program may accept coding failures from recordings. Frequent prototype tracking may require record-level exports and consistent automated measures. If assistance or stopped attempts disappear, recover them through a defined coding process or choose a different route. A headline success percentage cannot reconstruct lost evidence.
Timing needs an explicit scope too: a duration among successful tasks does not describe how long everyone attempted the task. Keep start, stop and incomplete-task treatment attached to the summary. Include recruitment, session review and coding in the operating cost.
The delivery explanation needs its own study
Our managed research service can compare how modeled buyers understand the delivery promise. Human prototype evidence tests whether people can save the intended address for the specified delivery. Keep the two studies’ sources and conclusions distinct.
Reproduce the measure before crediting the design
For the ten-attempt case, fix the rule and reproduce 5/10 in both workflows. Accept a replacement when it preserves the task outcomes and supporting episode at a workload the team can operate. Then redesign the control or route and test the new interface under the same rule. That sequence lets the next score answer a product question.
Further reading
Book a demo to compare explanations of which delivery an address change affects.
