...
Understanding and accepting these points will help you focus more on what methodology to use and how exhaustively.
What questions do I wish to answer, and why?
- How can I learn that? Is there an easier or more efficient way of answering these questions?
- What secondary research methods can complement and further, validate these findings?
- Who will benefit from this data?
- Don’t forget that the user-centered approach to product development aims to solve problems for the user while staying aligned with business strategy. Solving an internal process could be the solution.
...
- Bias/Leading: “Why did you have difficulty with the navigation?”. Again, the user might not have had (or felt) a problem with the navigation, but once led in this direction
- Embedded Assumption: “What are the problems you face while using our website?”. The user might not have any problems, but the question assumes that they would have had some problems.
- Double-barreled questions: “Do you think the website should have more content and products”? A typical answer would be yes, but yes to what? To more content and more products, or more contents and the same products, or more products and the same content?
- Close-ended questions: asking close-ended questions is not a problem in itself, can help with statistical data but will rob you of the insights that you are usually after.
- Confusing questions: Confusion can be caused by incorrect grammar, usage of sophisticated words, long questions (often double-barreled), usage of industry jargon, and so on.
Checklist of activities when planning your study
- Define the goal
- Determine the format and setting: In the lab, infield, moderated vs unmoderated, in person vs remote?
- Sample Size: Number of users (How to determine it Read more here http://bit.ly/UXMethods)
- Recruit the right participants: Match your demographics or even personas
- Write tasks: Explorator or Specific tasks that the user should achieve
- Metrics: Use metrics like Time on tasks, satisfaction, success rate, and error rate.
- Write a test plan and share it (Read more https://www.nngroup.com/articles/pm-research-plan/):
- Product tested
- Location, date and contact details
- Purpose, method, and Goals (How to determine it Read more here http://bit.ly/UXMethods)
- Sample size and User Profiles
- Team & Roles
- Ask the Observer to read this Guidelines https://www.nngroup.com/articles/observer-guidelines/
- To Do Checklist
- System, Equipment & Supplies Checklist (Incentives)
- Tasks / Scenarios
- Metrics / Questionnaires
- Schedule
- Script
- Conduct a pilot study: Rehearse to polish your test plan, know the language, time, order, etc.
Target Audience
Inclusive Design
Diversity / Inclusive, make sure you test with a variety of users:
- Ages
- Geography
- Income
- Education
- Gender
- Tech-savviness
Methods
Based on the stage
- Early Evaluation: Card sort, interviews, focus groups, photo studies
- Concept Generation: Desire testing, Participation Design, Workshops, Prototyping
- Test & Evaluate: Usability tests, Remote moderate, Wizard of Oz
...
Attitude or Behavior? Attitude: What they say? / Behavior: What they do?
Sample Size
Assess these factors to determine the number of users for your study:
...
- Design a product from scratch and doing exploratory design research: Maximize the number of users.
- Find solutions to problems of an already existing product and doing validatory research eg. usability testing, identifying new features etc.: The fewer the number of participants. When probing for usability problems with an interface, sometimes the magic number of 5 users can do the trick as explained by this NN group article.
- Probing for new feature capabilities or studying the current state of the system then, sometimes one may need more than 5 users.
- Traditional qualitative studies, we generally recommend 5 participants for the best return on investment.
- Research involves more than one target user group, then you may need to adjust the number of participants to 2–5 per group, depending on the level of experience and attitudinal overlap between the groups.
- Quantitative studies and eye-tracking require a larger sample size to obtain meaningful conclusions. Expect to increase the number of participants by at least 4 times. You may need at least 20–30 participants in each target user group.
Incentives
Contact Melanie Munn Melanie.Munn@Ferguson.com to request incentives for those customers that participate.
Method List
More about What method to use. A list of different methods and their descriptions: http://bit.ly/UXMethodsGuide