Research Tools / Design

In order to plan and select the proper UX Research method, we need to ask the right questions in the right way and document consistently to ensure we collect the right data and answer our pre-defined objectives.

Keep in mind that we work in a fast-paced environment where people want quick fixes and results, so you might feel pressured to solve a problem as quickly as possible to show results. Research is fundamental for the User Centred process, but you shouldn’t focus on the complete research process every single time.

Sometimes, the questions you want to answer are very simple and straightforward, so a simple bit of research will help you answer it. Other times, the research might have already been done exhaustively and you already have a lot of the information you need, all you need is to fill a few gaps.

Understanding and accepting these points will help you focus more on what methodology to use and how exhaustively.

What questions do I wish to answer, and why?

How can I learn that? Is there an easier or more efficient way of answering these questions?
What secondary research methods can complement and further, validate these findings?
Who will benefit from this data?
Don’t forget that the user-centered approach to product development aims to solve problems for the user while staying aligned with business strategy. Solving an internal process could be the solution.

“If I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”

Bias/Leading: “Why did you have difficulty with the navigation?”. Again, the user might not have had (or felt) a problem with the navigation, but once led in this direction
Embedded Assumption: “What are the problems you face while using our website?”. The user might not have any problems, but the question assumes that they would have had some problems.
Double-barreled questions: “Do you think the website should have more content and products”? A typical answer would be yes, but yes to what? To more content and more products, or more contents and the same products, or more products and the same content?
Close-ended questions: asking close-ended questions is not a problem in itself, can help with statistical data but will rob you of the insights that you are usually after.
Confusing questions: Confusion can be caused by incorrect grammar, usage of sophisticated words, long questions (often double-barreled), usage of industry jargon, and so on.

Checklist of activities when planning your study

Define the goal
Determine the format and setting: In the lab, infield, moderated vs unmoderated, in person vs remote?
Sample Size: Number of users (How to determine it Read more here http://bit.ly/UXMethods)
Recruit the right participants: Match your demographics or even personas
Write tasks: Explorator or Specific tasks that the user should achieve
Metrics: Use metrics like Time on tasks, satisfaction, success rate, and error rate.
Write a test plan and share it (Read more https://www.nngroup.com/articles/pm-research-plan/):

Product tested
Location, date and contact details
Purpose, method, and Goals (How to determine it Read more here http://bit.ly/UXMethods)
Sample size and User Profiles
Team & Roles
Ask the Observer to read this Guidelines https://www.nngroup.com/articles/observer-guidelines/
To Do Checklist
System, Equipment & Supplies Checklist (Incentives)
Tasks / Scenarios
Metrics / Questionnaires
Schedule
Script

Conduct a pilot study: Rehearse to polish your test plan, know the language, time, order, etc.

Target Audience

Inclusive Design

Diversity / Inclusive, make sure you test with a variety of users:

Ages
Geography
Income
Education
Gender
Tech-savviness

Methods

Based on the stage

Early Evaluation: Card sort, interviews, focus groups, photo studies
Concept Generation: Desire testing, Participation Design, Workshops, Prototyping
Test & Evaluate: Usability tests, Remote moderate, Wizard of Oz

Presence: Do you need them in the room or in their environment?

Attitude or Behavior? Attitude: What they say? / Behavior: What they do?

Sample Size

Assess these factors to determine the number of users for your study:

What’s the nature and scope of research — is it exploratory or validatory?
Who and what kind of users are you planning to study?
What’s the budget and time to finish the study?
Does your research involve presenting statistically significant numbers or inferring behavioral estimates for the problem statement?

Once you answer those questions then you can have an idea of the sample size:

Design a product from scratch and doing exploratory design research: Maximize the number of users.
Find solutions to problems of an already existing product and doing validatory research eg. usability testing, identifying new features etc.: The fewer the number of participants. When probing for usability problems with an interface, sometimes the magic number of 5 users can do the trick as explained by this NN group article.
Probing for new feature capabilities or studying the current state of the system then, sometimes one may need more than 5 users.
Traditional qualitative studies, we generally recommend 5 participants for the best return on investment.
Research involves more than one target user group, then you may need to adjust the number of participants to 2–5 per group, depending on the level of experience and attitudinal overlap between the groups.
Quantitative studies and eye-tracking require a larger sample size to obtain meaningful conclusions. Expect to increase the number of participants by at least 4 times. You may need at least 20–30 participants in each target user group.

Incentives

Contact Melanie Munn Melanie.Munn@Ferguson.com to request incentives for those customers that participate.

Method List

More about What method to use. A list of different methods and their descriptions: http://bit.ly/UXMethodsGuide