How do you run a usability test of a prototype with real users?

Recruiting is the critical path: start it the day you write the questions, not the day the prototype is done. In the session, spend the first stretch on the person's life before the prototype appears. Give tasks as situations, never as instructions. Let them think aloud and stay silent through the hard parts. Ask for a commitment at the end. Synthesize the same day or it did not happen. And remember what a quiet room cannot tell you about a Saturday floor.

How do you recruit people to test a startup prototype?

Recruiting is the critical path: start it the day you write the questions, not the day the prototype is done. In the session, spend the first stretch on the person's life before the prototype appears. Give tasks as situations, never as instructions. Let them think aloud and stay silent through the hard parts. Ask for a commitment at the end. Synthesize the same day or it did not happen. And remember what a quiet room cannot tell you about a Saturday floor.

What is think-aloud testing and how do you moderate it?

Recruiting is the critical path: start it the day you write the questions, not the day the prototype is done. In the session, spend the first stretch on the person's life before the prototype appears. Give tasks as situations, never as instructions. Let them think aloud and stay silent through the hard parts. Ask for a commitment at the end. Synthesize the same day or it did not happen. And remember what a quiet room cannot tell you about a Saturday floor.

How many users do you need to test a prototype?

Recruiting is the critical path: start it the day you write the questions, not the day the prototype is done. In the session, spend the first stretch on the person's life before the prototype appears. Give tasks as situations, never as instructions. Let them think aloud and stay silent through the hard parts. Ask for a commitment at the end. Synthesize the same day or it did not happen. And remember what a quiet room cannot tell you about a Saturday floor.

Testing with strangers · Product Sprint

§The question this chapter answers: where does this break in the hands of someone who owes us nothing?

§The prototype is frozen. The temptation now is to show it to the people nearest you, who will be encouraging, and to call that a test. The people nearest you are not strangers. This chapter is about the ones who are.

§ 12.1Recruiting is the critical path#

§Every part of the can be compressed except this one. A stranger who fits your segment has a life, and getting an hour of it takes days of asking. Start recruiting the day you write the questions in chapter 11, not the day the prototype is done. The team that starts recruiting after the freeze has a prototype and nobody to show it to, and fills the gap with friends.

§Where to find them: the shortlist from the expert conversations, the communities you listened to, the customers you interviewed who said you could come back, and the people they introduced you to. Offer something honest: a coffee, an early look, a say in what gets built. Expect a third of the people you ask to say yes, and ask three times as many as you need.

§How many: Jakob Nielsen's argument that five users find most of the usability problems in one design still holds for what it claims, and it claims less than people think. Five find the problems. They do not tell you whether the value is real, and they are five from one segment. Plan for five to six per segment you care about, and if you have two segments, customer and operation, plan for both.

§ 12.2Life before the prototype#

§Do not open the prototype for the first stretch of the session. Ask about the person's life in the part of it the touches: how they eat out, how they run a shift, what happened last time. This is a shortened chapter 5, and it does two things. It tells you who this person is so you can read their reactions correctly. And it puts them in the mindset of their real situation before you ask them to imagine one.

§ 12.3Situations, never instructions#

§The single most important technique in the room. Do not say "tap check-in." Say "you just sat down at your table. Show me what you would do."

§An instruction tests whether they can find a button you named. A situation tests whether the product makes sense to a person in the moment it is for. The difference is the whole point. In the field sprint, six of six customers failed the check-in when given the situation; every one of them could have found the button if told its name.

§ 12.4Think-aloud and silence#

§Ask them to narrate: what they see, what they expect to happen, what they are looking for. The Nielsen Norman Group has called this the single most valuable usability method for thirty years, and it costs nothing. The device is theirs to hold. You do not touch it.

§Then the hard part for you: stay silent when they get stuck. Every instinct says help. Do not. The pause while they search for the button is the finding. How long it lasts, what they try, what they say when they give up: that is what you came for, and a helpful word erases it. Steve Krug's advice for moderators is the right one: your job is to keep them talking and to keep yourself from talking.

§Two people per session if you can. One who speaks, one who writes and does not speak. The note-taker catches the face the moderator misses.

§ 12.5The commitment at the end#

§Close with a request that costs them something. A second conversation, with a date. An introduction to someone who has it worse. A place in the pilot. Rob Fitzpatrick's rule from chapter 5 applies with more force here, because they have now seen the thing: what they accept and what they dodge is worth more than everything they said about the screens.

§Note the face when they say yes. A yes with a date and a smile is different from a yes while reaching for a coat.

§ 12.6Same-day synthesis or it did not happen#

§One page per person, filled before you leave the building. Three verbatim quotes. Where they got stuck, for how long, what they said. What they thought the unit of value was. What they agreed to at the end, and with what face. One thing you learned that you did not want to hear. Your confidence, and why.

§By the next morning the words remain and the is gone: the pause, the face, the reach for the coat. The page is not a record. It is the decision, made while it is still true, about what you just saw.

§ 12.7What a quiet room cannot tell you#

§A prototype test is a controlled setting, and some things do not survive the control. In the field sprint, two of the six operations interviewees barely touched the screen. One refused to use the laptop at all. They were being asked to imagine a Saturday night floor with two hundred covers while sitting in a quiet room with two people watching. What the room could test was whether the concept made sense to them. What it could not test was whether a server would ever look at a screen during service. That is a different test, in a different place, and the honest synthesis says so.

§ 12.8What you leave with#

§One page per person, dated the day of the session. A first pass at the pattern: what everyone did, what nobody did, what split them. The commitments, listed, with what was accepted and what was dodged. And a note of what this setting could not test and where you would have to go to test it. The next chapter turns these pages into a position.

12. Testing with strangers