Respondents describe a problem before seeing a prototype, then react positively to the new screen. Those responses answer different questions and should not be combined into a single before-and-after proof of demand.
For founders comparing study rounds, preserve what changed in the audience, prompts and product exposure. A clearer prototype can improve feedback without establishing that the underlying problem became more common or valuable.
What does a before-and-after customer study compare?
A pre-prototype study can explore current behaviour and unaided problem accounts. A later study can examine how people understand or use a proposed workflow. The evidence types are related but distinct.
If different participants take part, the comparison also includes audience differences. If the same people return, they bring prior exposure and expectations. Neither arrangement is automatically wrong, but its limits should remain visible.
Create a study-round comparison record
Use a fictional unresolved-enquiry product. The proposed record below helps the team interpret changes without inventing causality.
| Comparison field | What to retain |
|---|---|
| Research question | What each round intended to resolve |
| Audience and recruitment | Who participated and how they were found |
| Prompt version | Wording and order of questions |
| Product exposure | What participants saw before answering |
| Task evidence | What they actually attempted or described |
| Remaining uncertainty | What the round could not establish |
Do not compare a general satisfaction answer from one round with task completion in another as though they were the same metric.
Separate concept appeal from task evidence
A participant saying the prototype looks useful is different from completing a relevant task with it. Observe where they need help, misunderstand status or fall back to their current process.
Avoid coaching people through the critical step and then recording independent success. If assistance is part of the test, label it and preserve what the person could do without help.
The neutral question guide helps keep solution framing visible. The assumption ledger separates a positive reaction from an established business claim.
Use the comparison to narrow the next build
If the prototype clarifies the task but exposes a confusing ownership step, that may justify a focused revision. It does not require adding every feature mentioned during the session.
My StackFlight case study describes a product for collecting source responses. This comparison method is proposed and does not claim a measured conversion or demand result from its use.
Write the decision with its evidence and limits: which workflow improved, what remained difficult and what the next experiment should resolve. Preserve both rounds so later reviewers can distinguish learning from a change in measurement.
Does more positive feedback prove the prototype created demand?
Not by itself. Inspect task behaviour, recruitment, framing and the actual commitment evidence.
Should we use the same participants again?
Choose based on the question and record prior exposure; returning and new participants each have different interpretation limits.
What should we compare first?
Align research questions, audience and evidence types before a prototype study review.