8. The ladder of validation experiments

§The question: for this validation question, what is the cheapest experiment that could answer it against a real product?

§The third whitepaper's ladder of fidelity was for before launch, testing a product that did not exist. The same rungs run after launch, testing features and directions inside a product that does. The mistake, once you have a product, is to test every idea by building it, because building is now the thing you know how to do. The ladder is the reminder that a built feature is the most expensive rung, not the default one.

§ 8.1The rungs, after launch#

§Landing page or announcement. Describe a capability you have not built, to your existing users, and measure who asks for it. Answers: is there demand? Cost: a page, an email.

§Concierge. Deliver a new value by hand to a few existing users, openly. Answers: is the value real, and what does delivering it involve? Cost: your time, for a few users.

§Wizard of Oz. A feature that looks automated with a human behind it. The user experiences the feature; you do the work manually. Answers: does the experience work before the machine exists? Cost: your labor per use.

§Fake door. A button or entry point for a feature that is not built. A user clicks; you record the click and show "coming soon" or a short survey. Answers: how many users want this enough to try it? Cost: a button.

§The real feature. Build it, ship it to a cohort, measure against a control. Answers: does it actually move the metric? Cost: the build.

§ 8.2The cheapest thing that can be wrong#

§The rule from the sprint is unchanged and more tempting to break, because now the team can build. Before spending a week building a feature, a fake door tests whether anyone wants it for the cost of a button. Before building an automated system, a Wizard of Oz tests whether the experience lands for the cost of some manual labor. The team that builds first learns the same thing later and more expensively.

§Alberto Savoia's pretotyping is this rule as a philosophy: test the "it", whether the thing should exist, before the "how", how to build it well, and do it with the flimsiest artifact that produces real behavior.

§ 8.3The ethics of the fake door#

§A fake door shows users something that is not real, and that is a small deception that has to be handled with care or it costs trust. The rules: it resolves honestly and immediately, "this is not available yet, want us to tell you when?", never a dead end or a fake success. It is used for genuine uncertainty, not to manipulate. And it is used sparingly, because a product full of buttons that go nowhere teaches users the product is unreliable. Bland and Osterwalder's library of experiments is explicit that the cheap demand tests carry a trust cost, and the cost is paid whether or not the team notices.

§ 8.4Matching rung to question#

§Demand for a feature: fake door or landing page. Whether a value is real: concierge. Whether an experience works before automating: Wizard of Oz. Whether a built feature moves the metric: the real thing with a control. A team that runs the right rung for each question learns fast and cheap. A team that builds everything learns slow and expensive, and runs out of runway with a backlog of built features nobody validated.

§ 8.5What you leave with#

§For each validation question, the cheapest rung that answers it, chosen deliberately over building. Fake doors that resolve honestly. A concierge or Wizard of Oz where the question is about value or experience rather than demand. And built features reserved for the questions only a real, measured feature can answer. The next chapter is how to know when validation is finished.