Discovery Is an Evidence System

TL;DR

• Discovery exists to resolve uncertainty, not to produce research, documents, or prototypes.
• Choose the artefact that matches the risk: evidence for the problem, a narrative for strategic clarity, a prototype for interaction, an eval for model capability, or a pilot for behaviour and operations.
• Keep a problem queue and a short delivery queue. Every active idea needs evidence, a next decision, and a stop condition.

---

Cheap implementation did not kill discovery. It removed the excuse for vague discovery.

Teams can now turn an idea into a polished prototype before they have agreed which customer matters, what problem they observed, or what result would justify production. The artefact looks late-stage. The thinking is still early.

Discovery is the evidence system that prevents visible progress from outrunning understanding.

Manage problems, not feature requests

The product backlog should be a problem queue. Each item represents an unresolved customer or business opportunity, not a pre-approved solution waiting for capacity.

A useful problem record contains:

• The affected user and situation
• The observed pain or opportunity
• Evidence, including contradictory evidence
• The outcome that would matter
• Current solution ideas
• The largest remaining uncertainty
• The next decision and evidence required
• The condition for stopping

This structure keeps customer understanding separate from the solution currently being tested. If one idea fails, the team can decide whether the underlying problem remains worth pursuing.

Feature requests still belong in the evidence. They do not become the queue. “Add CSV export” may point to a reporting workflow, an integration gap, a compliance requirement, or one customer's preference. Discovery finds out which.

Keep the delivery queue short

The delivery queue answers a different question: what work is actively moving now?

It should be shallow. Keep only work that has an owner, a defined next state, and enough context to proceed. A two-month inventory of decomposed tasks becomes stale faster when build cycles compress.

For small changes, a problem statement and acceptance criterion may be enough. Complex or regulated work still needs deeper design, dependency mapping, and controls. Shorter implementation time does not make the surrounding system simple.

Refine when a decision is approaching. A recurring calendar event is not sufficient reason to expand the queue.

Start with the uncertainty

Do not begin discovery by asking which artefact to make. Ask which uncertainty prevents the next responsible decision.

Product work usually carries six risk classes:

AI adds uncertainty inside several classes:

• Probabilistic output quality
• Compound failure across multi-step workflows
• Data provenance and privacy
• Prompt injection and tool abuse
• Inference, review, and support costs
• Model, prompt, and context drift
• Human escalation capacity

Name the dominant risk. Trying to resolve every risk at once produces expensive tests and ambiguous results.

Choose the right discovery artefact

Implementation abundance makes artefact choice more important.

No artefact is inherently modern. A document is powerful when the uncertainty is conceptual. A prototype is powerful when the uncertainty is experiential. An eval is powerful when the uncertainty is probabilistic.

The product builder's advantage is being able to create all three and knowing which one the decision requires.

Prototypes create evidence and bias

Prototypes make abstract ideas concrete. That is their strength and their danger.

A polished prototype anchors the conversation around its current form. Stakeholders start discussing button labels, edge cases, and launch timing. The visible implementation can make an untested premise feel decided.

Label prototypes by maturity:

• Provocation: makes an idea discussable; no claim of feasibility or desirability
• Interaction test: evaluates whether people understand and can use a flow
• Capability test: connects enough real model behaviour to evaluate quality
• Production-shaped pilot: includes real integrations, controls, telemetry, and support paths for a bounded audience

State what the prototype proves and what it does not. Never let visual polish communicate maturity implicitly.

Build evals before model confidence

For an AI capability, create a small eval set before optimising prompts or selecting models.

Start with 20 to 50 representative cases:

• Common requests
• Important edge cases
• Known failure patterns
• Cases where abstention is correct
• Cases carrying material customer or business consequences

Define the rubric with domain experts and users where appropriate. Run a simple baseline. This reveals whether the core risk is model capability, context, workflow design, data quality, or an unrealistic quality bar.

The evaluation framework explains how the set grows into regression infrastructure.

Use an evidence ladder

Discovery evidence becomes stronger as it gets closer to real behaviour and consequence.

1. Stated opinion
2. Attention or expressed interest
3. Effort invested by the user
4. Repeated behaviour change
5. Economic or organisational commitment
6. Durable customer and business outcome

Do not promote evidence to a higher rung. Five enthusiastic prototype interviews are not retention. A waitlist is not willingness to migrate a workflow. A successful offline eval is not trust.

Product Signal Beats Product Hope covers how to define the threshold and abandonment condition before the result arrives.

The idea lifecycle

Every problem moves through five states. Movement is not always forward.

1. Observe

Collect evidence of a problem in the real workflow. Use customer conversations, analytics, support, sales, operations, and direct observation.

Output: a problem record with source evidence.

2. Frame

Define the affected user, context, outcome, and why this problem matters now. Separate the customer instinct from any one solution.

Output: a problem narrative and the riskiest assumptions.

3. Explore

Generate multiple interventions. Analyse proven patterns, recognisable improvements, and the genuinely new element. Choose the next artefact based on uncertainty.

Output: alternatives and a test plan.

4. Prove

Seek the minimum credible evidence required for a commitment. This may combine user behaviour, eval results, economics, and an operational pilot.

Output: a decision to invest, revise, pause, or stop.

5. Learn in production

Release progressively. Compare the result with the original belief and threshold. Feed failures, overrides, support cases, and adoption patterns back into the problem record and eval suite.

Output: stronger product judgement, not only a shipped feature.

Continuous feedback without the feedback swamp

Collecting more feedback is easy. Maintaining a decision-quality evidence base is harder.

Use multiple channels:

• Customer interviews and observation
• Product analytics and search behaviour
• Support and complaint themes
• Sales objections and lost deals
• Implementation and operations friction
• AI overrides, escalations, and failure traces

Triangulate before generalising. A large customer may expose a strategically important requirement even when the issue is rare. A high-volume support theme may reflect poor communication rather than missing capability. Context determines weight.

AI can cluster, summarise, and retrieve evidence across thousands of records. Preserve links to the underlying source. The PM must be able to move from a generated theme back to actual customers, examples, and consequences.

Lighthouse users

Lighthouse users are customers whose problems point toward the market you intend to serve. They are not simply your largest or loudest accounts.

Choose them because:

• Their workflow represents the intended future market
• They feel the problem acutely
• They will invest time in testing and feedback
• Their constraints expose risks you need to solve
• Their success would be meaningful evidence for similar customers

Co-design is useful when the customer helps you understand the problem and test the mechanism. It becomes outsourcing when one customer's desired solution controls the roadmap.

The discovery review

Review active problems weekly:

1. Which problem has the strongest new evidence?
2. Which assumption is currently limiting confidence?
3. Does the next artefact match that uncertainty?
4. Which idea should stop?
5. Are we learning from real behaviour or producing internal artefacts?
6. What moved from opinion to stronger signal?

The objective is not a full backlog. It is a clear sequence of decisions.

Anti-pattern: the polished answer to an unasked question

A PM uses an agent to build a complete prototype. It has realistic data, smooth interactions, and a persuasive demo. Leadership approves production investment.

Nobody observed the target user experiencing the problem. The prototype answered “can we make this?” so convincingly that the team skipped “should this exist?”

Discovery failed because the artefact resolved the wrong uncertainty.

Execution and Delivery begins once the next commitment is clear. Discovery continues alongside it, testing whether the commitment remains justified.