Routine performnce Reduces or prevents humn error Prevents or recovers from system error Plesnt to use usble useful We wnt to improve usbility If you cnnot mesu
Работа добавлена на сайт samzan.net:
Agenda
Evaluation overview
Usability evaluation methods
Heuristic evaluation
Online lecture from Stanford HCI Group.
Scott Klemmer in Section with discussion.
In coming episodes
Web analytics
Think aloud protocol
Clients: Reports
p.s. any questions on affinity diagramming or competitive analysis ?
What is usability?
A usable product:
• Supports routine performance
• Supports non-routine performance
• Reduces or prevents human error
• Prevents or recovers from system error
• Pleasant to use
usable ≠ useful
We want to improve usability!
If you cannot measure it,
you cannot improve it.
Usability Evaluation Methods
Empirical methods $ $ $
Observation
Experimentation
Analytical methods $
Derived from physical, psychological, sociological, or design theories
Heuristics derived from experience
Empirical (testing) methods
• Contextual inquiry > done
• Web analytics > coming
• Think aloud > coming
• Remote testing
• Log analysis
• Eye tracking
• “Wizard of Oz” studies
• Surveys and questionnaires
• Diary studies
Analytical (inspection) methods
• Heuristic evaluation (UIM1 Ch2)
• The GOMS (Goals, Operators, Methods, and Selection rules) family
Keystroke-Level Model (KLM)
• Cognitive walkthrough (UIM1 Ch5)
• Pluralistic walkthrough (UIM1 Ch3)
1UIM = Usability Inspection Methods, Nielsen &
Mack
Analytical (inspection) methods
“Discount usability engineering methods” Jakob Nielsen (not so much of $$)
Usually a small team of evaluators using analytical methods to review an interface based on recognized usability principles
Heuristic evaluation
Brief the group
Evaluate individually
Aggregate issues
Apply severity ratings
Summarize findings
Step 0: Brief the group
Heuristic evaluation methodology (this)
Domain briefing
Important if evaluators are unfamiliar with the products domain
Scenario briefing
Can optionally include specific tasks or scenarios or allow evaluators to explore on their own
Step 1: Evaluate individually
Two passes
1. Inspect flow (and optional tasks/scenarios)
2. Inspect each element against heuristics
Recognized usability principles
10 Nielsen heuristics (UIM Ch2, p.30)
Nielsens Heuristics
H1: Visibility of system status
H2: Match between system and real world
H3: User control and freedom
H4: Consistency and standards
H5: Error prevention
H6: Recognition rather than recall
H7: Flexibility and efficiency of use
H8: Aesthetic and minimalist design
H9: Error recovery
H10: Help and documentation
Step 3: Apply severity ratings
4 Catastrophic
Product cannot be released
3 Major
High-priority issue
2 Minor
Good to fix when theres a lull
1 Cosmetic
Icing on the cake (these rarely get done)
0 Not a problem
I dont agree that this is a problem at all
Justification:
Frequency: Common or rare occurrence?
Impact: How bad is it? How hard to recover?
Persistence: One-time problem users can work around or unavoidable problem?
For each issue, average the rating from each evaluator
Step 4: Summarize findings: Executive summary
What are the important take-aways for people who do not read the individual Usability Aspect Reports?
Look for the forest in the trees
Consider affinity diagramming
Audience: Project managers, team leads
Heuristic evaluation advantages
“Discount usability engineering”
Low intimidation
Dont need to identify tasks, activities
Can identify obvious fixes
Can expose problems user testing doesnt
Provides a shared language for talking
about usability recommendations
Heuristic evaluation disadvantages
Un-validated
Inconsistent
False alarms -- problems unconnected with tasks
May be hard to apply to new technology
What is a competitive analysis?
A comparison between
two or more products
that meet similar user
needs.
Why perform a competitive analysis?
Understand the marketplace
Products become successful for a reason
Provide direction to management
Exploit strengths and weaknesses
Build domain knowledge
Learn best practices
Identify common patterns, language
When should you perform a competitive analysis?
Research phase
Study existing products to get the lay of the land
Evaluation phase
See how your product/prototype compares to existing products
Who is your audience?
Design team
Management
External stakeholders
What is your deliverable?
Presentation
Walkthroughs
Executive summary
What if you dont find competitors?
A: others tried and failed
B: you didnt look hard enough
C: its a blue ocean
Metrics
Feature checklist
Walkthrough
Expert review
User testing
Dependent on time, budget, access to competing products, and goals
Feature checklist
A high-level inventory of the features, content, and other attributes provided by each product
The point is not an exhaustive list but to highlight meaningful differences
Dont forget about attributes like
Price, license, availability
OS, hardware requirements
Popularity
Navigation tools, search
As you evaluate you may uncover new attributes you wish to evaluate
Walkthrough
Define tasks
For each product, perform tasks while taking screenshots (requires access!)
Assemble screenshots into a slideshow and provide captions describing user actions and system animations, feedback, and delays
Expert review
Heuristic evaluation (Nielson) holistic
Cognitive walkthrough task specific
User testing
Time on task
Think aloud protocol
Surveys, questionnaires
Step 3: Summarize products
Brief summary highlighting strengths, weaknesses, and unique features of each product
Include illuminating quotes, screenshots, and/or videos
Step 4: Recommend courses of action
Executive summary
If your product was analyzed, how did it compare?
Your audience will recall only a couple of recommendations and takeaways; make sure theyre the right ones
Executive summary
Lead with something interesting
Provide recommendations or conclusions and justify them
Use proper grammar (see Strunk & White)
If you can remove a word, do it
<10% overall length of the report
Executive summary
First we looked at twelve different chat clients and discovered that they basically do the same things but some of them have a lot of more customization options than our chat client does. Also, audio and video are popular.
Executive summary
While all chat clients offer similar core functionality, most offer more customization
options than ours. We recommend working
with users to determine whether additional
customization options would be useful. In
addition, several of the most popular chat
clients now offer audio and video
communication. We recommend considering
these features for future releases.
How might we improve the measuring cup?
It sometimes slippery
Handles get hot
Nobody complained about measuring part
But how do you measure the liquid level?
How might we improve the measuring cup? (solution)
Numbers are visible from above
Elegant solution by OXOs Alex Lee.
Still improvements: looks like medical device.
HCD Lessons:
I he asked people what they wanted, they would have said faster horses. Remember?
FOR User Interfaces:
Observing actions
Evaluating outcomes
Two Gulfs
First step to cross: The Gulf of Execution
How the user know what to do?
Second step to cross: The Gulf of Evaluation
How the user know what happened?
The making of gulfs. How easily can someone:
Determine the function of the device?
Tell what actions are possible?
Determine mapping from intention to physical movement?
Perform the action?
Tell what state the system is in? / if its in desired state?
Determine mapping from state to interpretation
To reduce gulfs, provide …
Visibility (perceived affordances or signifiers)
Feedback
Consistency (also known as standards)
Non-destructive operations (hence the importance of undo)
Discoverability: All operations can be discovered by systematic exploration of menus
Reliability. Operations should work. Period. And events should not happen suddenly.
Bu using you know look at this:
video
Direct manipulation
Immediate feedback on actions
Continuous representation of objects
Leverage metaphor
Eye to the Future: Gestures
The solution to menu creep?
Even more direct?
To learn more …
Don Norman, The Design of Everyday Things
Hutchins, Hollan, Norman, Direct manipulation interfaces, 1985
Mental Models
Key to good design:
What makes an interface learnable?
What leads to errors?
The goal: design beacons the right model
Users model develops through interaction with the system
Designers often expects users model to be the same as theirs
But often it isnt!
Conceptual model mismatch
Mismatch between designers & users conceptual models leads to …
Slow performance
Errors
Frustration
…
Mental models arise from experience, metaphor; and analogical reasoning
“A text processor is a typewriter”
We have models (beliefs) about our own behavior, of others, of objects, software …
Our models are incomplete, inconsistent, unstable in time, and often rife with superstition
Direct manipulation provides
Leverages real-world metaphors
Good idea of how each object works and how to control it.
Interface discloses how to use it.
“If technology is to provide an advantage, the correspondence to the real world must break down at some point.” - Jonathan Grudin
To learn more …
JM Carroll, JR Olson, Mental models in human-computer interaction: Research issues, 1987
Don Norman, Design of Everyday Things
James Reason, Human Error
H <-> H. How to interview people?
Possibly your most important skill
• hear about peoples experiences
• learn their meaning & feelings
• discover design opportunities
Interviewing skills a basis for more advanced methods--survey design, contextual inquiry, think-aloud, task analysis, and so forth
What and why?
An interview is a method of asking questions and listening
Uses a planned interview protocol a set of questions
Ask what you cant observe
I know you are a nursing supervisor. If I followed you through a typical day, what would I see you doing?
Types of questions
Close-ended (“forced choice”)
In what year were you born?
Are you satisfied with your hospital?
On a scale from 1 to 10, how much do you like your job?
Open-ended
What is the most enjoyable part of your job?
If I were in the emergency room of your hospital, what would I notice first?
Question options
Behavioral questions
“Can you describe a recent occasion when
a patient alert was sounded, and tell me
what you did?”
Feeling questions
Knowledge questions
“If a patient says she is in pain, what do
you look for?”
Background; ratings (leave for last, or even better, a private questionnaire)
Planning
Decide whom you will interview
Plan your interview protocol
Ethical checklist:
How will you explain your purpose?
What promises will you make?
Any risks, psychological or otherwise?
How will you achieve confidentiality and data security?
How will you get informed consent?
How will you record and take notes?
Test your protocol and revise
Starting the interview
Everything depends on trust and rapport. So what do you do first?
[Rapport occurs when two or more people feel that they are in sync or on the same wavelength because they feel similar or relate well to each other.] by Wikipedia
Good and poor follow ups
Interview with John, a student in a drug treatment program.
I. Does anything in the program stand out for you?
J. Yeah, the hot seat
I. The hot seat is where someone is the focus of attention?
J. Right
I. So what was it like? Is it different with different people?
J. Yeah, it depends
Clear language
Interview with low-income mother who has been given an iPad for a research project on energy-saving practices
I. So, how do you like your iPad?
M. My son is using it.
I. What apps is he running?
M. Im not sure.
I. What about your Internet connectivity; is your wireless working ok?
Other good question types
Illustrative questions
Some nurses hate working at night, but others like the flexibility. Whats your experience?
Role-playing questions
Suppose I were a new nurse just coming to this hospital, and I asked you what I should do to succeed. What would you tell me?
Preparatory questions
Weve been talking about your job. Now I want to ask you about how you got to be where you are today.
Encourage your interviewee
That was really helpful.
We are about halfway through the interview. Its going really well. Hows it going for you?
I realize this is a sensitive issue for you. You should decide how comfortable you are talking about it.
Embodied cognition
• The idea is that we don't just think with our minds, we also think with our bodies
• A good posture to empathize with your interviewee is. . . their posture
• Best posture to understand your interviewee is. . their posture
• Mimicry works but NOT if the other is socially incompetent or resistant
The final interview question
What should I have asked you that I didnt think to ask?
Thats all I wanted to ask you. Anything you want to add?
(or)
How did the interview go for you? Keep the recorder on!
Lessons: Strategic interviewing
Information gathering not accusatory or blame-finding
I asked Scott, oh, where did everyone live?
“How did you get into the city when they blocked it off? “Who else was with you?”
Ask some unanticipated questions (less boring & less likely to be rehearsed) and dont be afraid to probe for examples or “tell me more”
Can you show me your wines?
Would you introduce me to your manager?
Lessons: Coding
• Trick is to get the right level, not replicate what the interviewee says but not so abstract as to be useless
• Think about themes that are actionablesomething you could improve through design
• Be sure to hit feelings, goals and subgoals, interactions with others, context
Design for people
Peoples tasks, goals and values drive development
Work processes of the users within the work
Access decisions from the users point of view
Context awareness.
Talk to the actual experts.
“Users are the only one of the stakeholders (decision makers) in the process” Jh. Zimmerman
Prototyping
Tremendously valuable strategy (one of the biggest skills you will get from the class)
Rapid creation of the approximation of design idea
Getting feedback not just making artifact
Communicate with other stakeholders
Prototyping is a efficient strategy for dealing with things that are hard to predict
focus on GOALS
evolve the DESIGNS
Right prototype
Not complete (in strategic way)
Easy to change
Ready for retire
What prototypes prototype?
Feel
What it might look like?
Implementation
What it might work like?
role
What might the experience be?
“The best way to have good idea is to have a lot of ideas.” Linus Pauling
Class structure
Lab hours (2 hrs):
1st hour: group presentations, weekly milestones
2nd hour: workshops
Practice hours (1 hr):
Article discussions, case-studies
Lectures (1 hr):
First half emphasis on the main concepts
Second half class activity on the given material
Storyboards isnt about “pretty pictures”
Its about communicating ideas
Storyboards Should Convey
Setting
People involved
Environment
Task being accomplished
Sequence
What steps are involved?
What leads someone to use the app?
What task being illustrated?
Satisfaction
Whats motivates people to use this system?
What does it enable people to accomplish?
What need does the system fill?
Benefits of Storyboarding
Holistic focus: Helps emphasize how an interface accomplishes a task
Avoids commitment to a particular user interface (no buttons yet)
Helps get all stakeholders on the same page in terms of the goal
6 paper prototyping tips and tricks
Keep all your materials in one place! Small interface widgets tend to get lost or damaged easily
Work quickly and make reusable components (buttons, etc)
If something is difficult to simulate (progress indicators, right mouse menus, hyperlinks), have user ask if it is available and then verbally describe interaction.
Backgrounds (25cm x 29cm poster board) can be useful to contain
Dont be afraid to mix and match hardware and software! For instance, if size constraints are important, you might want to make a blinder using a photograph of the device that would be used and manipulate the prototype within the frame.
When appropriate, add context by including familiar operating system components.
Get Creative with Materials
Widgets: Paper, Cardboards, Transparencies
Connectors: Tape, Glue, Rubber Cement
Drawing: Pens, Pencils, Markers
… and more
More materials
Poster board, unlined index cards, a foam core are all useful depending on the size of your prototype
Removable tape or restickable glue is useful for changing components quickly
Transparency pens allow the user to input content use a sheet of transparency paper for the input field
Use wide-tipped pens and markers (think Sharpie) smaller line widths can be difficult to see
Use stacks of index cards to simulate tabbed dialog boxes.
Wizard of Oz technique
Make an interactive app without (much) code
Front end interface
(remote) wizard controls user interface
Makes sense when its faster/cheaper/easier than making real thing
Get feedback from users people
Hi-fidelity: users think its more real
Low-fidelity: more license to suggest changes
Wizard-of-Oz prototyping . . .
simulates machine behavior with human operators
Aardvark
“Why start-ups Must Pay Attention to Whats Behind The Curtain”
Venture Capital Dispatch WSJ
Making a Wizard powered prototype
Map out scenarios and application flow
What should happen in response to user behavior?
Put together interface “skeleton”
Develop “hooks” for wizard input
Where and how the wizard will provide input?
Selecting the next screen, entering text, entering a zone, recognizing speech, etc.
Remember that later youll need to replace with computer
Rehearse Wizard with a colleague
Running Wizard powered prototypes
Practice with friends first
Once youre comfortable recruit users
Two roles: facilitator and wizard
Facilitator provides tasks (paper) and takes notes
Wizard operates interface (more authentic if hidden or remote)
User feedback can be:
Think Aloud (speak freely as performing tasks)
Retrospective (best when think aloud distracts)
Heuristic evaluation (works with experts too)
Debrief users (reveal wizard if needed)
Advantages of Wizards
Fast and thus, cheaper and more iterative prototypes
Creating multiple variations is easy
More “real” than paper prototyping
Identifies bugs and problems with current design
Places the user at the center of development
Can envision challenging-to-build application
Designer learn by playing wizard
Disadvantages of Wizards
Simulations may represent otherwise imperfect tech
May simulate tech that doesnt exist yet (and may never)
Wizard require training and may be inconsistent
Playing wizard can be exhausting
Some features (and limitations) are difficult/impossible to simulate effectively
May be inappropriate in some venues (e.g. home)
Benefits of Video Prototyping
Cheap and Fast
Great communication tools
Helps achieve common ground
Ideally portable and self-explanatory
Can serve as a spec for developers
Ties interface designs to tasks
Aligns and orients interface choices
Makes sure you have a complete interface
And that theres nothing extra
What should the video show?
Like a storyboard, the whole task, including motivation and success
Establishing shots and narrative help
Draw on tasks youve observed
Illustrate important tasks your system enables
Can help scope a minimum-viable-product
Changes what design teams argue about (in a good way)
What are the steps?
Start with an outline (or your storyboards)
Fine to extemporize
Equipment
A camera. Could be a phone. Built-in laptop camera
People
a realistic location
In general, focus on message more than production values
Considerations
Can use audio or a silent movie with title cards (audio can be too wordy)
Interface can paper, mockups, code or invisible (just showing the task)
Can show both success and failure (of your interface and others)
Edit as little as possible because editing is hugely time-consuming (in camera/pause editing is most efficient)
Creating and comparing alternatives
Quantity or Quality?
“The works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work - and learning from their mistakes - the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.” (Bayles and Orland, 2001)
Benefits of sharing multiple
More individual exploration
More features sharing
More conversational turns
Better consensus
Increase in group rapport
Alignment guides the eye, reducing clutter
Avoid slight misalignments
they undermine your ability to beacon organization.
We automatically notice patterns
and deviations
So when you deviate from a pattern
do so strategically
And use visual proximity and scale
to convey semantic information
A brief word on color
Pay attention to it
Design in grayscale first
Keep luminance values from grayscale when moving to color
Grids: to learn more …
Kevin Mullet and Darrel Sano, Designing Visual Interfaces
Luke Wroblewski, Web Form Design
Jan Tschichold, The New Typography
How can you detect poor scent?
Flailing
Low confidence
Back button
Low-scent navigation
Surprising categories
Short links
Hidden navigation
Icons provide little additional information.
Icons help when …
They facilitate repeat recognition
When you know what something looks like but not what its called
Good redundant coding can help
Improving scent: multi-word links
With specific, recognizable terms
Trigger words, not “clever” terms
This also helps accessibility
Prime real estate
Above the fold
Where people expect
Where other pages put similar content
Not where ads usually
Whitespace conveys grouping
“Some space might be narrow so that other space may be wide, and some space must be emptied so that other space may be filled.”
Use size contrasts to indicate hierarchy
“Information consists of differences that make a difference.”
Three basic tools of visual design
Typography
Layout
Color
“Minute to learn and lifetime to master”
Challenges
Individual differences dwarf manipulation effects
i.e. some people read faster than others. If there is an effect, its very small.
Confound: Reading requires familiarity
Dependent Variable? Speed? Comprehension?
“Legibility, in practice, amounts simply to what one is accustomed to” Eric Gill, 1931
No Robust Evidence for Serif Hypothesis
“What initially a neat dichotomous question of serif vs sans serif has resulted in a body of research consisting of weak claims and counter-claims, and study after study with findings of no difference. Is it the case that more than one hundred years of research has marred by repeated methodological flaws, or are serifs simply a typographical red herring?”
To improve typographical eye
Use typefaces in multiple places to see its range
Display the same text with different typefaces to change its effect
Multiple ways to evaluate
Empirical assess with real users
Formal methods and formulas to calculate measures
Automated software measures
Critique: expertise and heuristic feedback
When to get design critique?
Before user testing: Dont waste users on the small stuff. Critique can identify minor issues that can be resolved before testing, allowing users to focus on big issues.
Before redesigning: Dont throw out the baby with the bathwater: critique can help you learn what works and what should change.
When you know there are problems, but you need evidence. Perhaps youve received complaints from customers or found yourself stumbling around your site. Critique can help you articulate problems and provide you with ammunition for redesign.
Before release: smooth off rough edges.
Heuristic Evaluation
Developed by Jacob Nielsen
Helps find usability problems in a design
Small set (3-5) of evaluators to examine UI
Independently check for compliance with usability principles (“heuristics”)
Different evaluators will find different problems
Evaluators only communicate afterwards:
Finding are then aggregated
Can perform on working UI or sketches
Nielsens Ten Heuristics
Visibility of System Status
Match between System and World
User Control & Freedom
Consistency & Standards
Error Prevention
Recognition Rather than Recall
Flexibility & Efficiency of Use
Aesthetic & Minimalist Design
Help Users Recognize, Diagnose, & Recover from Errors
Help and Documentation
Evaluators Process
Step through design several times
Examine details, flow and architecture
Consult list of usability principles
… and everything else that comes to mind
Which principles?
Nielsens “heuristics”
Category-specific heuristics from
E.g. Design goals, competitive analysis, existing design
User violations to redesign/fix problems
Why Multiple Evaluators?
No evaluator finds everything
Some find more than others
Heuristic Eval: Cost-effective
In one case: benefit-cost ratio of 48
Estimated benefit $500,000; cost $10,500
Value of each problem ~15K
How might we calculate this value?
In-house -> productivity; open market -> sales
Severe problems found more often
Single evaluator achieves poor results
Only finds 35% of usability problems
5 evaluators find ~75% of problems
Heuristics vs User Testing
Heuristic Evaluation often faster:
1-2 hours each evaluator
HE presets come pre-interpreted
User testing is more accurate (by def.)
Takes into account actual users and tasks
HE may miss problems & find “false positives”
Valuable to alternate methods
Find different problems
Dont waste participants
Phases of Heuristic Evaluation
1. Pre-evaluation training: give evaluators needed domain knowledge and information on the scenario
2. Evaluation: individuals evaluate and then aggregate results
3. Severity rating: determine how severe each problem is (priority). Can do first individually and the as a group.
4. Debriefing: review with design team
How-to: Heuristic Evaluation
At least two passes for each evaluator
First to get feel for flow and scope of system
Second to focus on specific elements
If system is walk-up-and-use or evaluators are domain experts, no assistance needed
Otherwise might supply evaluators with scenarios
Each evaluator produces list of problems
Explain why with reference to heuristic or other information
Be specific and list each problem separately
How-to: Heuristic Evaluation
Why separate listings for each violation?
Risk of repeating problems aspect
May not be possible to fix all problems
Where problems may be found
Single location in UI
Two or more locations that need to be compared
Problem with overall structure of UI
Something is missing
Ambiguous with early prototypes; clarify in advance
Sometimes features are implied by design docs and just havent been “implemented” relax on those.
Severity Rating
Independently estimate after review
Allocate resources to fix problems
Estimate need for more usability efforts
Severity combines
Frequency
Impact
Persistence
Severity ratings
0 dont agree that this is a usability problem
1 cosmetic problem
2 minor usability problem
3 major usability problem; important to fix
4 usability catastrophe; imperative to fix;
Severity Ratings Example
Issue: unable to edit ones weight
Severity: 2
Heuristic violated: User control and freedom
Description: when you open the app for the first time, you have to enter your weight, but you cannot update it. It could be useful if you mistyped your weight, or if one year or two after the first use of the app, your weight has changed.
Debriefing
Conduct with evaluators, observers, and development team members
Discuss general characteristics of UI
Suggest potential improvements to address major usability problems
Dev. team rates effort to fix
Brainstorm solutions
HEURISTICS
Show system status
Familiar metaphors and language
Control and freedom
Consistency
Error prevention
Recognition over recall
Flexibility and efficiency
Aesthetic & minimalistic design
Recognize, diagnose & recover errors
Help
Show system status: Time
Feedback depends on response time:
< 1s: just show outcome
~ 1s: feedback that activity is underway
>> 1s: show functional progress; time
Show: space
Feedback above is telling Im out of space in Gmail account.
Show: change
Youve changed doc, want to save?
Show: next steps
Its important for user to know what is actionable for the next. They dont see the problem, but have a solution provided already.
Answers already in the message.
Problem Solving as Representation
“Solving a problem simply means representing it so as to make the solution transparent”
- Herbert Simon, The Sciences of the Artificial
Working memory
Capacity: 2 +/- 2 (dont require your users to remember)
Getting Things Done
Task management system
Whenever something comes to mind, just write it down immediately.
When interfaces help people distribute cognition, it can . . .
Encourage experimentation
Scaffold learning and reduce errors through redundancy
Show (only) differences that matter
Convert slow calculation into fast perception
Support chunking, especially by experts
Increase efficiency
Facilitate collaboration
A good representation
. . . shows all of the relevant information, and nothing else.