What User Research Actually Looks Like When You Do It Right
What User Research Actually Looks Like When You Do It Right
What User Research Actually Looks Like When You Do It Right
The gap between talking to users and doing user research is wider than most designers realize. Here is what rigorous, insight-generating research looks like in practice.
The gap between talking to users and doing user research is wider than most designers realize. Here is what rigorous, insight-generating research looks like in practice.
The gap between talking to users and doing user research is wider than most designers realize. Here is what rigorous, insight-generating research looks like in practice.
I have sat through dozens of presentations where a designer says “we talked to users” and then presents a single insight slide with three quotes and a persona. I understand why this happens — user research is time-consuming, recruiting is hard, analysis is harder, and stakeholders often want answers faster than good research can provide them.
But there is a real cost to this shortcut. Decisions made on thin research are not just decisions made with less certainty — they are decisions made with false certainty, which is considerably more dangerous. A designer who does no research knows they are guessing. A designer who does bad research thinks they have evidence.
So what does genuinely useful user research look like in a studio context, where time and budget are always constrained? Here is our current practice.
We start by writing our research questions before we design our research method. This sounds obvious, but it is where most research goes wrong. If you start by asking “should we do interviews or a survey?”, you are letting the method determine the question. Instead, write down the three to five things you most need to know to make a good design decision. Then choose the method that will best answer those specific questions.
For exploratory research — understanding behavior, motivations, and mental models — we use semi-structured interviews. We conduct a minimum of eight and a maximum of twelve, depending on how quickly we reach thematic saturation. The interview guide is short: five to seven open-ended questions, with probes prepared but used flexibly. We record every session with consent, and we transcribe and code every transcript. This is not quick, but it produces insights that are grounded in actual language, not designer paraphrase.
For evaluative research — testing whether a design is usable and achieves its intended goal — we use moderated usability testing. We test with five to eight participants per round. We measure task completion, time-on-task, and error frequency. But we also ask participants to think aloud, because behavioral data without verbal data leaves too many “why” questions unanswered.
For quantitative validation — understanding how widespread a behavior or problem is — we use surveys. But we only run surveys when we already understand the problem space from qualitative work. A survey you run before you understand the domain is a survey that asks the wrong questions.
The analysis phase is where most research efforts fail. Pulling themes from twelve interviews is not a matter of reading through transcripts and highlighting quotes that support your existing hypothesis. It requires affinity mapping, a process of open coding followed by axial coding, and constant pressure-testing of emerging themes against disconfirming evidence. If every participant is saying the same thing, you probably designed your question to produce a particular answer.
The output of good research is not a presentation. It is a shift in how the team understands the problem. When research is working, designers start referring to research participants by name in design critiques. They say “remember what Priya said about the checkout flow” instead of “users want.” That granularity is the sign of research that has genuinely shaped thinking rather than merely validated it.
I have sat through dozens of presentations where a designer says “we talked to users” and then presents a single insight slide with three quotes and a persona. I understand why this happens — user research is time-consuming, recruiting is hard, analysis is harder, and stakeholders often want answers faster than good research can provide them.
But there is a real cost to this shortcut. Decisions made on thin research are not just decisions made with less certainty — they are decisions made with false certainty, which is considerably more dangerous. A designer who does no research knows they are guessing. A designer who does bad research thinks they have evidence.
So what does genuinely useful user research look like in a studio context, where time and budget are always constrained? Here is our current practice.
We start by writing our research questions before we design our research method. This sounds obvious, but it is where most research goes wrong. If you start by asking “should we do interviews or a survey?”, you are letting the method determine the question. Instead, write down the three to five things you most need to know to make a good design decision. Then choose the method that will best answer those specific questions.
For exploratory research — understanding behavior, motivations, and mental models — we use semi-structured interviews. We conduct a minimum of eight and a maximum of twelve, depending on how quickly we reach thematic saturation. The interview guide is short: five to seven open-ended questions, with probes prepared but used flexibly. We record every session with consent, and we transcribe and code every transcript. This is not quick, but it produces insights that are grounded in actual language, not designer paraphrase.
For evaluative research — testing whether a design is usable and achieves its intended goal — we use moderated usability testing. We test with five to eight participants per round. We measure task completion, time-on-task, and error frequency. But we also ask participants to think aloud, because behavioral data without verbal data leaves too many “why” questions unanswered.
For quantitative validation — understanding how widespread a behavior or problem is — we use surveys. But we only run surveys when we already understand the problem space from qualitative work. A survey you run before you understand the domain is a survey that asks the wrong questions.
The analysis phase is where most research efforts fail. Pulling themes from twelve interviews is not a matter of reading through transcripts and highlighting quotes that support your existing hypothesis. It requires affinity mapping, a process of open coding followed by axial coding, and constant pressure-testing of emerging themes against disconfirming evidence. If every participant is saying the same thing, you probably designed your question to produce a particular answer.
The output of good research is not a presentation. It is a shift in how the team understands the problem. When research is working, designers start referring to research participants by name in design critiques. They say “remember what Priya said about the checkout flow” instead of “users want.” That granularity is the sign of research that has genuinely shaped thinking rather than merely validated it.
I have sat through dozens of presentations where a designer says “we talked to users” and then presents a single insight slide with three quotes and a persona. I understand why this happens — user research is time-consuming, recruiting is hard, analysis is harder, and stakeholders often want answers faster than good research can provide them.
But there is a real cost to this shortcut. Decisions made on thin research are not just decisions made with less certainty — they are decisions made with false certainty, which is considerably more dangerous. A designer who does no research knows they are guessing. A designer who does bad research thinks they have evidence.
So what does genuinely useful user research look like in a studio context, where time and budget are always constrained? Here is our current practice.
We start by writing our research questions before we design our research method. This sounds obvious, but it is where most research goes wrong. If you start by asking “should we do interviews or a survey?”, you are letting the method determine the question. Instead, write down the three to five things you most need to know to make a good design decision. Then choose the method that will best answer those specific questions.
For exploratory research — understanding behavior, motivations, and mental models — we use semi-structured interviews. We conduct a minimum of eight and a maximum of twelve, depending on how quickly we reach thematic saturation. The interview guide is short: five to seven open-ended questions, with probes prepared but used flexibly. We record every session with consent, and we transcribe and code every transcript. This is not quick, but it produces insights that are grounded in actual language, not designer paraphrase.
For evaluative research — testing whether a design is usable and achieves its intended goal — we use moderated usability testing. We test with five to eight participants per round. We measure task completion, time-on-task, and error frequency. But we also ask participants to think aloud, because behavioral data without verbal data leaves too many “why” questions unanswered.
For quantitative validation — understanding how widespread a behavior or problem is — we use surveys. But we only run surveys when we already understand the problem space from qualitative work. A survey you run before you understand the domain is a survey that asks the wrong questions.
The analysis phase is where most research efforts fail. Pulling themes from twelve interviews is not a matter of reading through transcripts and highlighting quotes that support your existing hypothesis. It requires affinity mapping, a process of open coding followed by axial coding, and constant pressure-testing of emerging themes against disconfirming evidence. If every participant is saying the same thing, you probably designed your question to produce a particular answer.
The output of good research is not a presentation. It is a shift in how the team understands the problem. When research is working, designers start referring to research participants by name in design critiques. They say “remember what Priya said about the checkout flow” instead of “users want.” That granularity is the sign of research that has genuinely shaped thinking rather than merely validated it.
