| Engaged Scholars, Thoughtful Practitioners: The Interdependence of Academics and Practitioners in User-Centered Design and Usability | | BIB | 1-7 | |
| Susan M. Dray | |||
| When Links Change: How Additions and Deletions of Single Navigation Links Affect User Performance | | BIBA | HTML | PDF | 8-20 | |
| Lauren F. V. Scharff; Philip Kortum | |||
| This study examined user performance for Web sites in which a critical
navigation link may have changed over the course of two visits. These second
visits occurred either immediately after the first visit, one week later, or
three weeks later. A shortcut link to the information was either consistently
present, consistently absent, or varied between user visits. Results indicated
when a pertinent navigation link was removed users searched through more pages
and were less accurate in finding target information. When a link was added
after the first visit, only about half of the users used it; however, even
users who didn't use the added link still showed a performance improvement in
their subsequent visit, suggesting that completing a single previous search on
a site will lead to relatively long-term memories that can influence subsequent
searches.
The following were the main findings of the study: * By the second time users visit a site, their performance improves compared to their first visit (decreased page counts and search times), even after delays of up to three weeks. * Even small changes, like the addition or deletion of a single link, can have large impacts on user performance. * Presence of a shortcut link to vital information dramatically increases the probability that a user will find that information. Compared to a site without the link, users will be faster and visit fewer pages. * When a pertinent navigation link is removed, users search through more pages and are less accurate in finding target information. * When users have clear memory of a site, deletion of a link causes users to exhaustively examine the primary navigation structure; this behavior persists as the delay between visits increases for at least up to three weeks. * When a link is added after the first visit, only about half of the users will use it. The rest will continue to use the site in the way they have done in the past. * Even users who don't use an added link still show a performance improvement in their subsequent visits, suggesting that secondary paths to information should not be deleted simply because a top-level navigation link has been added. * Even if the link is removed after a user has been exposed to it, users are no less accurate than if the link had never been present. | |||
| The Combined Walkthrough: Measuring Behavioral, Affective, and Cognitive Information in Usability Testing | | BIBA | HTML | PDF | 21-33 | |
| Timo Partala; Riitta Kangaskorte | |||
| This paper describes an experiment in studying users' behavior, emotions,
and cognitive processes in single usability testing sessions using an
experimental method called the combined walkthrough. The users' behavior was
studied using task times and completion rates, and emotions were studied using
bipolar scales for experienced valence and arousal. Cognition was studied after
each task by revisiting detected usability problems together with the users and
applying an interactive method based on cognitive walkthrough to each usability
problem. An interactive media application was tested with 16 participants using
these methods. The results of the experiment showed that the developed methods
were efficient in identifying usability problems and measuring the different
aspects of interaction, which enabled the researchers to obtain a more
multifaceted view of the users' interaction with the system and the nature of
the problems encountered.
The following were the main findings of this experiment: * Behavioral, affective, and cognitive aspects of computer system usage can be cost-effectively studied together in usability testing. * The information obtained by the behavioral, affective, and cognitive measurements can contribute to a more multifaceted understanding of user interaction with the system. * Variations in the users' emotional experiences (valence and arousal) related to completing a task using an interactive system can be efficiently measured using bipolar scales. Systematic measurement of emotional experiences broadens the scope of subjective measures beyond traditional satisfaction measures. * The use of highly positive or negative media elements influences overall ratings of task-related affective experiences in interactive media applications. * Ideas underlying the cognitive walkthrough can be useful in retrospective analysis of usability problems together with the user. | |||
| How To Specify the Participant Group Size for Usability Studies: A Practitioner's Guide | | BIBA | HTML | PDF | 34-45 | |
| Ritch Macefield | |||
| Using secondary literature, this article helps practitioners to specify the
participant group size for a usability study. It is also designed to help
practitioners understand and articulate the basis, risks, and implications
associated with any specification. It is designed to be accessible to the
typical practitioner and covers those study scenarios that are common in
commercial contexts.
The following were the main findings in this article: * For a wide variety of reasons, specification of the participant group size for a usability study remains a matter of opinion and debate. * The goal for usability practitioners is to help negotiate a group size that is optimal, or at least beneficial, for the wider project in which the study is taking place. This means that practitioners should be able to articulate the basis, risks, and implications associated with any specification. * When utilizing research literature in this area, practitioners should carefully consider how well the specific studies underpinning the particular research relates to the study under consideration. Similarly, they should pay careful attention to any caveats in the advice being offered. * There is no "one size fits all" solution to the challenge here. However, for studies related to problem discovery a group size of 3-20 participants is typically valid, with 5-10 participants being a sensible baseline range. In these scenarios, the group size should typically be increased along with the study's complexity and the criticality of its context. In scenarios concerned with discovering severe ("show stopping") problems in early conceptual prototypes a group size of five participants is typically valid. For comparative studies where statically significant findings are being sought, a group size of 8-25 participants is typically valid, with 10-12 participants being a sensible baseline range. * In many scenarios, it can be beneficial to split study groups into chunks of participants within a punctuated study, whereby the results data is incrementally analyzed after each chunk. One benefit of this tactic is that a study can be terminated early if its objectives have already been met, thereby saving project resources. For example, a comparative study may be terminated early because it has already produced the statistically significant findings being sought. Another benefit of this tactic is that it promotes the iterative design processes that are fundamental to a UCD philosophy. For example, in a study of an early conceptual prototype, one chunk of participants revealed a show stopping problem. After the interface design was revised, the study continued. | |||
| Is Technology Becoming More Usable -- or Less -- and With What Consequences? | | BIB | HTML | PDF | 46-49 | |
| Daryle Gardner-Bonneau | |||
| Reliability of Self-Reported Awareness Measures Based on Eye Tracking | | BIBA | HTML | PDF | 50-64 | |
| William Albert; Donna Tedesco | |||
| Participants in a usability evaluation are often asked whether they noticed
certain elements after some level of interaction with a design. The purpose of
this study was to evaluate the reliability of self-reported awareness measures
using eye tracking data. Participants were shown 20 popular homepages for 7
seconds each and then asked afterwards if they saw 2 particular elements on
each page. The results showed that self-reported awareness measures are
reliable, but can vary depending on question structure and object type. These
findings have implications for how usability practitioners ask questions about
object awareness, and how that information is used in the design process.
Practitioner's Take Away * Usability practitioners should feel confident in collecting self-reported awareness measures from participants. * If a practitioner wants to minimize the chance of making an incorrect conclusion, they should use a continuous (5- or 7-point) scale for self-reported awareness (similar to Experiment 2). * If a practitioner wants to maximize the likelihood of confirming that a participant did or did not see an element, they should use a discrete set of questions for self reported awareness (similar to Experiment 1). * Participants are more reliable in their self-reported awareness for navigation and image elements, than functional elements, regardless of question structure. | |||
| Usability Evaluation of Randomized Keypad | | BIBA | HTML | PDF | 65-75 | |
| Young Sam Ryu; Do Hyong Koh; Brad L. Aday; Xavier A. Gutierrez; John D. Platt | |||
| In this work, the usability of a randomized numeric keypad was examined and
compared to the usability of a conventional numeric keypad. The comparison used
completion time measurements and the error rate of short (4 digit) and long
(8-digit) PINs to contrast efficiency and accuracy of the keypads. The results
showed that the average completion time with a randomized keypad is longer than
with a conventional keypad. Additionally, the number of errors with a
randomized keypad was significantly higher than with a conventional keypad,
particularly when using long PINs. Accordingly, a randomized numeric keypad is
more applicable to tasks with short (4-digit) PINs.
Practitioner's Take Away * Longer completion times should be expected when using a randomized numeric keypad versus a conventional keypad. * The number of errors with a randomized keypad was significantly higher than with a conventional keypad when users typed longer PINs. * The number of errors with a randomized keypad was not significantly higher than with a conventional keypad when users typed short PINs. * A randomized numeric keypad is better suited to applications requiring short PINs. | |||
| The Seven Habits of Highly Effective Usability People | | BIB | PDF | 76-80 | |
| Mary Beth Rettger | |||
| Plain Language Makes a Difference When People Vote | | BIBA | HTML | PDF | 81-103 | |
| Janice (Ginny) Redish; Dana Chisnell; Sharon Laskowski; Svetlana Lowry | |||
| The authors report on a study in which 45 U.S. citizens in three geographic
areas and over a range of ages and education levels voted on two ballots that
differed only in the wording and presentation of the language on the ballots.
The study sought to answer three questions: * Do voters vote more accurately on a ballot with plain language instructions than on a ballot with traditional instructions? * Do voters recognize the difference in language between the two ballots? * Do voters prefer one ballot over the other? In addition to voting on the two ballots study participants commented on pages from the two ballots indicated their preference page-by-page and overall. For this study, the answer to all three questions was "yes." Participants performed better with the plain language ballot. Their comments showed that they recognized plain language. They overwhelmingly preferred the plain language ballot. The authors also discuss issues that arose on both ballots from problems with straight-party voting, with mistaking one contest for another, and with reviewing votes. Based on the study results, the authors provide guidance on language to use on ballots. This article includes links to the two ballots, other materials used in the study, and the full report with more details. The following are key points from this study: * Language matters. Study participants voted more accurately on the ballot with plain language than on the ballot with traditional language. * Education matters. Level of education correlated with accuracy. Voters with less education made more errors. * Location, gender, age, and voting experience do not matter. None of those factors was a statistically significant correlate of accuracy. * People recognize plain language. After they voted both ballots, participants were shown pairs of pages (the A and B versions of the same ballot page) and were told, "Notice that the instructions on these pages are different. Please compare them and comment on them." Participants commented that certain words were "simpler," "more common," and "easier to understand." * People prefer plain language. Asked for an overall preference between the two ballots, 82% (37 of 45) chose Ballot B, the plain language ballot. * Straight-party voting confuses many people. Even on the plain language ballot, participants made errors related to straight-party voting. * Some voters do not have a good grasp of levels of government. Many of the errors on both ballots related to confusing U.S. Senate with State Senator and County Commission with City Council. * Usability professionals can help make ballots and other voting materials more usable through research and consulting. * Even in a summative test, usability specialists often see ways to improve the product for its next release. In the study reported here, the plain language ballot did significantly better than the ballot with traditional language. Nonetheless, after watching participants work with the ballot, we realized we could make the language even clearer. We include recommendations for an even better plain language ballot. | |||
| Response Interpolation and Scale Sensitivity: Evidence Against 5-Point Scales | | BIBA | HTML | PDF | 104-110 | |
| Kraig Finstad | |||
| A series of usability tests was run on two enterprise software applications,
followed by verbal administration of the System Usability Scale. The original
instrument with its 5-point Likert items was presented, as well as an alternate
version modified with 7-point Likert items. Participants in the 5-point scale
condition were more likely than those presented with the 7-point scale to
interpolate, i.e., attempt a response between two discrete values presented to
them. In an applied setting, this implied that electronic radio-button style
survey tools using 5-point items might not be accurately measuring participant
responses. This finding supported the conclusion that 7-point Likert items
provide a more accurate measure of a participant's true evaluation and are more
appropriate for electronically-distributed and otherwise unsupervised usability
questionnaires.
The following are the main findings of this study: * Five-point Likert scales are more likely than 7-point scales to elicit interpolations in usability inventories. * Interpolations are problematic because they cannot be mitigated within an electronic survey medium and require interpretation with facilitated surveys. * Interpolations provide evidence that 5-point Likert scales may not be sensitive enough to record a usability test participant's true evaluation of a system. * Seven-point Likert scales appear to be sensitive enough to record a more accurate evaluation of an interface while remaining relatively compact. * Seven-point Likert scales appear to be more suited to electronic distribution of usability inventories. * Practitioners can quickly test Likert items through verbal protocols by using interpolations as a metric. | |||
| A Comparison of the Usability of a Laptop, Communicator, and Handheld Computer | | BIBA | HTML | PDF | 111-123 | |
| Piia Suomalainen; Leena Korpinen; Rauno Pääkkönen | |||
| The goal of this study was to examine the usability of a laptop,
communicator, and handheld computer using test subjects and questionnaires. The
study aimed to determine how user-friendly and ergonomically correct these
devices are. The subjects (25) had 5 minutes to perform typing or calculation
tests with each device. While the subjects performed the tasks, an observer
monitored the subjects' work posture. After the tasks were completed, the
subjects completed questionnaires about the usability of each device Based on
the subjects' experiences, the handheld computer and laptop had better
ergonomic characteristics than the communicator. Subjects felt the highest
amounts of stress in their neck while working on the laptop, subjects felt
stress on their backs while working on the communicator, and they felt stress
in their eyes while working on the handheld computer. Subjects performed the
typing tasks best using the laptop. Our research suggests that companies
developing mobile devices should consider ergonomic issues and the ergonomic
differences between different types of mobile devices to further improve user
satisfaction.
The following were the main findings of this study: * Usability of laptops, communicators, and handheld computers can be determined by combining observations of subjects' work posture, tests on how well subjects complete tasks, questionnaires on subjects' perceptions of ergonomic design, and questionnaires on how subjects' physically felt while using the devices. * Most stress caused by these devices can be felt in a person's back, upper limbs, and eyes. | |||
| A Commentary of "How To Specify the Participant Group Size for Usability Studies: A Practitioner's Guide" by R. Macefield | | BIB | PDF | 124-128 | |
| Rolf Molich | |||
| Moving Towards an All-Encompassing Universal Design Approach in ICT | | BIB | PDF | 129-131 | |
| André Liem; Sarah J. Swierenga; Rama Gheerawo | |||
| Comparing Computer Versus Human Data Collection Methods for Public Usability Evaluations of a Tactile-Audio Display | | BIBA | HTML | PDF | 132-146 | |
| Maria Karam; Carmen Branje; John-Patrick Udo; Frank Russo; Deborah I. Fels | |||
| We present a public usability study that provides preliminary results on the
effectiveness of a universally designed system that conveys music and other
sounds into tactile sensations. The system was displayed at a public science
museum as part of a larger multimedia exhibit aimed at presenting a youths'
perspective on global warming and the environment. We compare two approaches to
gathering user feedback about the system in a study that we conducted to assess
user responses to the inclusion of a tactile display within the larger
audio-visual exhibit; in one version, a human researcher administered the study
and in the other version a touch screen computer was used to obtain responses.
Both approaches were used to explore the public's basic understanding of the
tactile display within the context of the larger exhibit.
The two methods yielded very similar responses from participants; however, our comparison of the two techniques revealed that there were subtle differences overall. In this paper, we compare the two study techniques for their value in providing access to public usability data for assessing universally designed interactive systems. We present both sets of results, with a cost benefit analysis of using each in the context of public usability tests for universal design. We have found that it is important to consider the following concepts when creating systems using universal design principles: * Use automated questionnaires to increase participant numbers in evaluations. * Use public usability studies to supplement lab experiments with real-world data. * Modify existing evaluation tools to extend into the public domain. * Include as wide an audience in evaluations to ensure universality of the design. * Expect to alter major features of the design to ensure target users are addressed. * Select only technology that is robust enough to withstand constant public use. * Reinforce and secure systems to ensure safety of users and the equipment. * Restrict public studies to short, succinct questions and questionnaires to maintain ease of ethics approval, and focus the study on the broader aspects of system interactions. * Ensure proper assistance is in place to accommodate users with special needs during the study. Sign language or other interpreters, or special access needs are essential to address when conducting studies on universal designs. | |||
| Online Learning: Designing for All Users | | BIBA | HTML | PDF | 147-156 | |
| Cindy Poore-Pariseau | |||
| During the fall of 2008, 4.6 million students pursued their education in
online environments in the United States (Allen & Seaman, 2009). Considering
that students with disabilities represent nearly 10% of all U.S. college
students (National Council on Disability as reported by Frieden, 2003), one can
see a need to disseminate information regarding how to best meet the needs of
this population as they look to further their education by taking advantage of
online learning opportunities. Through this paper, the reader will learn about
Universal Design for Learning (UDL), accessibility laws, how the laws affect
online education, and how instructional design can be implemented as a way to
increase access to education for college students with disabilities. Finally,
there will be an exploration of the impact accessibility laws have on
instructional design and how an increase in accessibility can improve
motivation for all segments of the population.
Note: Although the statistics cited are based on U.S. postsecondary education student populations, the ideas outlined herein can be applied beyond the U.S., as the needs of disabled students are universal. Practitioner's Take Away As the number of students choosing to complete their studies in online environments continues to grow, so will the number of disabled students enrolling in online courses continue to grow. The following points must be considered when an online course is being developed: * Recognition must be given to the fact that retrofitting accommodations in online environments is not only an arduous task, it is often impractical, necessitating training in concepts such as Universal Design for Learning to be placed at the forefront. * Importance must be placed on ensuring that instructional designers (including faculty members who design their own instruction) receive the training necessary to become familiar with disabilities and disability related laws. * Importance must be placed on ensuring that instructional designers (including faculty members who design their own instruction) receive the training necessary to develop the ability to resolve accessibility issues (or to partner in resolving such issues). * A conscious effort must be made to proactively incorporate accessibility standards into all course rooms, coursework, and course materials, so that all have comparable opportunities to contribute effectively to the educational process. * All involved parties, including society as a whole, will benefit from a product that is fully usable and accessible. | |||
| Beyond Specifications: Towards a Practical Methodology for Evaluating Web Accessibility | | BIBA | HTML | PDF | 157-171 | |
| Panayiotis Koutsabasis; Evangelos Vlachogiannis; Jenny S. Darzentas | |||
| The current set of tools and specifications for ensuring web accessibility
require expert knowledge and often have a highly technical orientation, with
the consequence that it is not very clear how, or even when, to make use of
them. In an attempt to tackle this problem, this paper reviews the types of
tools and specifications available and proposes a simple and practical
methodology for web accessibility evaluation that demonstrates how these tools
and specifications could be used. The proposed methodology proposes methods and
processes for reaching and maintaining web accessibility, and consists of the
following phases: (a) identification of user requirements and setting up of
accessibility goals, (b) web accessibility evaluation and redesign process, and
(c) establishment and follow-up of accessibility policy. Further, in order to
illustrate step (b), an example of web accessibility evaluation is described,
where the domain is contemporary scientific publishing web sites. The work
presented in this paper reports on issues that need to be considered by
human-computer interaction (HCI) researchers, interaction design practitioners,
and usability professionals for inclusive web design and are complementary to
web usability engineering.
The following are the key points of this paper: * Web accessibility concerns every user, designer, and business owner. There are many ethical, business, user, and technical arguments for designing for web accessibility. * Current web accessibility tools and specifications are technically oriented and need expert knowledge to be understood and applied. * Despite the work on web accessibility, most web sites are still inaccessible. For example (described in this paper) six out of ten e-publishing homepages pose major accessibility problems and only two out of ten are accessible. * Typical web accessibility problems found include the loss of (a) information organisation, (b) navigation, (c) visibility, and (d) user control, when users access web sites in constrained contexts (e.g., with an oral browser or in a keyboard only situation). * The paper proposes a practical methodology reaching and maintaining web accessibility, which includes (a) identification of user requirements and set up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of accessibility policy. This three step approach will help to achieve the following: * Identify particular user requirements when designing for web accessibility and conform to a level of accessibility specifications (e.g., single-A of WCAG 1.0). * Implement a fast and practical method for regular web accessibility inspection according to your accessibility goals. This paper illustrates a method that makes use of automated tools and heuristics that can be employed in accessibility inspections. * Understand the appropriate time and the value of user testing. User testing is important for web accessibility, but a basic level of accessibility should be there. Testing with disabled users also increases web developers' awareness. * Establish and follow up a web accessibility policy. This should focus on basic rules for content update of the web site that editors of the web site should follow as well as tools for checking content on the fly or in the background. This is an area for further research development and application in the field. | |||
| Intra- and Inter-Cultural Usability in Computer-Supported Collaboration | | BIBA | HTML | PDF | 172-197 | |
| Ravi Vatrapu; Dan Suthers | |||
| In this paper, we argue for an increased scope of universal design to
encompass usability and accessibility for not only users with physical
disabilities but also for users from different cultures. Towards this end, we
present an empirical evaluation of cultural usability in computer-supported
collaboration. The premise of this research is that perception and
appropriation of socio-technical affordances vary across cultures. In an
experimental study with a computer-supported collaborative learning
environment, pairs of participants from similar and different cultures
(American-American, American-Chinese, and Chinese-Chinese) appropriated
affordances and produced technological intersubjectivity. Cultural usability
was analyzed through the use of performance and satisfaction measures. The
results show a systemic variation in efficiency, effectiveness, and
satisfaction between the two cultural groups. Implications of these findings
for the research and practice of usability, in general, and cultural usability,
in particular, are discussed in this paper.
The following are the key points of this paper: * Demand characteristics, in the context of cultural usability, refer to the study of expectations, evaluator-participant relationships, and cultural norms for appraisal and attribution. Demand characteristics should be given careful consideration in the design of cultural usability evaluations and in the interpretation of the results. * For usability evaluations involving participants from social collectivist cultures with interactional concerns for deference and harmony-maintenance, it might be beneficial to use questionnaires that solicit both Likert-type ratings and open-ended comments. * Participants from highly collectivistic cultures (such as the Chinese participants of this experimental study) with a high-context communication style and a field-dependent perceptual style might offer higher overall ratings to a system despite offering lower subjective ratings for the constituent parts of the systems. * Participants from highly individualistic cultures (such as the American participants of this experimental study) with a low-context communication style and a field-independent perceptual style might offer lower overall ratings to a system despite offering higher subjective ratings for the constituent parts of the systems. * Participants from highly collectivistic cultures with a greater emphasis on deference and harmony-maintenance (the Chinese participants in the context of this study) might make a higher number of positive comments compared to negative comments during usability evaluation despite higher negative ratings on Likert-type questionnaires. * Participants from highly individualistic cultures with a lesser emphasis on deference and harmony-maintenance (the American participants in the context of this study) might make fewer positive comments and more negative comments during usability evaluation despite higher positive ratings on Likert-type questionnaires. * Efficacy of open-ended comments might be higher for inter-cultural user groups when compared to intra-cultural user groups. * Despite significant differences in objective measures of usability, there might be no significant differences in performance or achievement of the primary task. In other words, there is a possibility of a "performance preference paradox" in the usability evaluation of computer-supported collaboration systems. * It might be more productive and informative to conceive of cultural variation at the level of human-computer interaction as a variation in the perception and appropriation of action-taking possibilities and meaning-making opportunities relate to actor competencies and system capabilities and the differences in social relationships and expectations rather than homeostatic cultural dimensional models or stereotypical typologies. * Design the technological capabilities of the system taking into account the competencies of the users. Design should focus on what action-taking possibilities and meaning-making opportunities users can psychologically perceive and sociologically appropriate. Design for sustaining, transforming, and creating traditional and novel configurations of social relationships and interactions in socio-technical systems. | |||