How do you validate business assumptions with existing data?

Most companies already hold data that can test their business design hypotheses before anything is built: visit frequency, order history, support logs, churn, the behavior of the people who would be the first customers. Bring it into the room, already extracted, so that assumptions are tested against numbers rather than against each other. A hypothesis that survives internal data is worth building. One that contradicts it was never a hypothesis.

What internal data should a startup use before building?

Most companies already hold data that can test their business design hypotheses before anything is built: visit frequency, order history, support logs, churn, the behavior of the people who would be the first customers. Bring it into the room, already extracted, so that assumptions are tested against numbers rather than against each other. A hypothesis that survives internal data is worth building. One that contradicts it was never a hypothesis.

How do you stress-test a business model?

Most companies already hold data that can test their business design hypotheses before anything is built: visit frequency, order history, support logs, churn, the behavior of the people who would be the first customers. Bring it into the room, already extracted, so that assumptions are tested against numbers rather than against each other. A hypothesis that survives internal data is worth building. One that contradicts it was never a hypothesis.

Against internal data · Business Design

§The hypothesis: every claim on the running list can be checked against a number the company or its partners already hold, and we have done so before deciding to build.

§Ten chapters have produced with numbers. Most of the numbers were estimates. This chapter is about replacing estimates with data that already exists, because almost every company, even a company of three, holds more evidence than it has looked at.

§ 11.1What is already there#

§For a company with any operating history, or a corporate team building inside one, the data is abundant: how often customers return, what they order, how much they spend, when they leave, what they complain about, what staff turnover costs, what a reservation that fails to find a table does to the evening's revenue. For a pure startup with no history, the data is thinner but not absent: waitlist behavior, the pricing conversations from chapter 9, public benchmarks for the category, and the internal data of any pilot partner.

§Douglas Hubbard's argument is that the value of information already held is almost always underestimated: the question is not whether you have perfect data but whether you have any data that would change a decision, and you usually do.

§ 11.2Bring it extracted#

§The rule that made this work in the case was procedural. The data came into the room already extracted, in a form the team could read against the hypotheses, not as a database someone would query if asked. When data is available on request, the request is never made, because the conversation moves faster than the query, and the team ends up testing its assumptions against each other's confidence.

§ 11.3Reading the list against the data#

§Each hypothesis on the running list is read against the closest number the data holds. Three outcomes.

§Supported. The data agrees within a range the team accepts. The hypothesis keeps its number and gains a citation.

§Contradicted. The data disagrees. The hypothesis is rewritten with the data's number, and the arithmetic downstream, unit economics, monetization, is redone with it. This is the outcome that pays for the exercise.

§Silent. The data does not speak to it. The hypothesis stays a hypothesis, marked so, with the test that will move it.

Case

The hospitality group brought visit and spend data across its brands, extracted before the session. Against the running list, three things happened. The frequency assumption behind the tier economics, how often a frequent customer actually visited across brands, was contradicted: real cross-brand frequency was lower than the team had assumed, because most regulars were regulars of one brand and did not know the others were related. The tier thresholds were rewritten on the real distribution. The incremental-visit assumption, that a recognized guest visits more, was silent: the data could not show it because nobody had ever been recognized across brands. It stayed a hypothesis, the central one, with a pilot as its test. And the turnover cost that managers had described as a pain was supported: the data on server tenure and the anecdotal cost of a departing server's knowledge lined up, which made the managers' willingness to pay the best-evidenced number on the list.

§ 11.4The outside view#

§Kahneman's outside view is the other source of numbers: what happened to comparable efforts elsewhere. Category benchmarks for conversion, retention, support load and inference cost exist for most kinds of , and a hypothesis that assumes twice the category's conversion rate should say so and say why. The Amplitude playbook's discipline of input metrics applies here in reverse: before choosing what to measure after launch, check whether the inputs you are assuming have ever been observed at those levels anywhere.

§ 11.5What has to be true#

§Added to the running list: for each hypothesis, the internal or external number it was read against and the outcome: supported, contradicted, silent. For every contradicted one, the rewritten number and the downstream arithmetic redone. For every silent one, the test that will move it. This is the last chapter that adds to the list. The next one collects it.

11. Against internal data