Free field notes / Updated 2026-10-21
Set Success Thresholds Before Testing an Idea
Write pass and fail numbers before your idea test begins. Use clear denominators, deadlines, and next actions to avoid moving the goalposts.
A success threshold states what must happen, among which people, and by what deadline for a test to justify your next step. Write it before collecting results, including what counts as a commitment and what you will do if the threshold is missed. This prevents an encouraging conversation from quietly replacing the evidence you originally needed.
Start with the decision the test will support
A threshold is useful only in relation to a decision. "Get ten signups" says little unless you know what ten signups would justify.
Suppose you are considering spending US$80 on materials for a small workshop. Before buying them, you could test whether four suitable people reserve places at the stated US$20 price. Four reservations might justify contacting them to confirm the date and collecting payment for a workshop you can deliver.
It would not justify renting a permanent studio. The evidence and the proposed commitment are too far apart.
Write the next investment first: "If this passes, I will spend two evenings preparing one pilot." Then ask what evidence would make that investment reasonable. Keep regulated ideas outside this process until qualified guidance establishes what is appropriate to test.
Write a complete threshold sentence
Use five parts:
- Audience: who is eligible to count.
- Exposure: what they must actually see or experience.
- Action: the observable behavior you will count.
- Quantity: how many actions out of how many opportunities.
- Deadline: when counting stops.
A hypothetical sentence is: "By Sunday at 6 p.m., at least four of twelve local hobbyists who see the date, location, scope, and US$20 price reserve a specific workshop place without paying yet."
That sentence closes several loopholes. A visitor who never sees the price is not the same exposure. A person asking about a different date is not a reservation for this workshop. Someone saying "great idea" is not a booking.
Also specify how you handle cancellations and duplicates. Count one person once. If someone cancels before the deadline, record the cancellation and use the rule you wrote in advance.
Choose evidence that matches the assumption
Do not use an easy-to-measure event just because it produces a larger number. If your assumption concerns payment, page views are several steps removed from the question.
| Test type | Hypothetical threshold | Decision it might support |
|---|---|---|
| Problem interviews | 3 of 8 people describe a paid workaround used in the last 3 months | Prepare a relevant offer |
| Landing page | 10 of 80 suitable visitors request a sample after seeing the price | Run sample follow-up conversations |
| Manual service | 3 of 10 suitable prospects book a US$40 pilot | Deliver a small pilot batch |
| Delivery feasibility | Complete 3 sample jobs within 45 minutes each at the agreed quality | Offer a limited paid trial |
| Follow-up | 4 of 12 signups answer a specific use-case question within 3 days | Recruit participants for a pilot |
These are illustrative numbers, not industry averages or universal pass rates. A different cost, audience, or next decision may require a different rule.
Avoid substituting volume for relevance. Eight interviews with people outside the intended buyer group cannot satisfy a threshold written for eight likely buyers.
Include a minimum exposure rule
A test can miss its target because the offer was rejected or because too few people encountered it. Those outcomes need different interpretations.
Suppose your rule requires three bookings from twelve offers, but only four suitable people see the full offer. Zero bookings from those four is an observation worth keeping. It is not the completed twelve-person test you planned.
Write an inconclusive condition: "If fewer than twelve suitable people see the offer by the deadline, do not judge demand from this test. Review reachability and decide whether another attempt is worthwhile."
Inconclusive should not become a permanent escape hatch. Repeated inability to reach the audience is evidence about access. If you cannot find likely buyers without disproportionate effort, that may be a reason to change the audience or stop.
Define pass, revise, and stop before starting
A simple decision table keeps the next step bounded.
| Result in a hypothetical twelve-offer test | Written response |
|---|---|
| 3 or more confirmed bookings | Deliver this small batch and measure effort |
| 1 or 2 confirmed bookings | Review objections, then test one justified change |
| 0 confirmed bookings after full exposure | Stop this version and reassess the problem |
| Fewer than 12 full offers delivered | Review access; do not claim the demand test passed |
The middle band is optional. Use it only if you can explain what it changes. "Revise" should lead to a named question, such as whether the available date excluded otherwise suitable buyers.
A passing result earns the next small step. It does not erase safety concerns, settle long-term economics, or establish repeat demand.
Keep the measurement method simple
Use one row per suitable person or one clearly defined visitor record, depending on the test. Track exposure, response, commitment, cancellation, and any relevant context.
Define ambiguous actions before they occur. Does a "booked call" require a selected calendar slot, or will a casual agreement count? Does "paid workaround" include a tool purchased for another purpose? Choose a definition consistent with the assumption and apply it across the sample.
You do not need an elaborate analytics system. A small manual log can be clearer than a dashboard if you understand exactly what each row represents. Avoid collecting private information that is unnecessary for the decision.
At the deadline, save the totals and the original offer. Keep later responses in a separate line or follow-up period rather than silently extending the test until it passes.
Change the plan openly when you learn something
Sometimes a test reveals a practical flaw early. Perhaps the booking link fails, or the proposed date conflicts with a local event. Fixing a broken test is reasonable.
Write down what changed and when. If the change affects the offer or the action, restart the relevant count or report the groups separately. Do not combine incompatible versions into one convenient success rate.
For example, if five people see a US$40 offer and seven later see a US$20 offer with a different scope, report two small groups. You did not test one offer with twelve people.
Common mistakes
- Choosing a threshold after seeing how many people responded.
- Using a percentage without reporting the count behind it.
- Counting friends who are encouraging but cannot use the offer.
- Changing the deadline because the result is uncomfortable.
- Treating an inconclusive test as either a pass or a definitive rejection.
- Setting a low threshold that cannot justify the next expense.
When you struggle to choose a metric, return to the riskiest assumption. When you struggle to act on a mixed result, use the kill or pivot decision guide.
Next step
Paste your idea into the free triage tool, then write your threshold in one sentence before contacting anyone. The 48-Hour Validation Kit can hold that rule beside your evidence and final decision.