HCI Bibliography Home | HCI Journals | About VR | Journal Info | VR Journal Volumes | Detailed Records | RefWorks | EndNote | Hide Abstracts
VR Tables of Contents: 010203040506070809101112131415161718

Virtual Reality 17

Editors:Daniel Ballin; Robert D. Macredie
Dates:2013
Volume:17
Publisher:Springer-Verlag
Standard No:ISSN 1359-4338 (print) EISSN 1434-9957 (online)
Papers:23
Links:link.springer.com | Twitter | Table of Contents
  1. VR 2013-03 Volume 17 Issue 1
  2. VR 2013-06 Volume 17 Issue 2
  3. VR 2013-09 Volume 17 Issue 3
  4. VR 2013-11 Volume 17 Issue 4

VR 2013-03 Volume 17 Issue 1

Influence of contextual objects on spatial interactions and viewpoints sharing in virtual environments BIBAKFull-Text 1-15
  Amine Chellali; Isabelle Milleville-Pennel; Cédric Dumas
Collaborative virtual environments (CVEs) are 3D spaces in which users share virtual objects, communicate, and work together. To collaborate efficiently, users must develop a common representation of their shared virtual space. In this work, we investigated spatial communication in virtual environments. In order to perform an object co-manipulation task, the users must be able to communicate and exchange spatial information, such as object position, in a virtual environment. We conducted an experiment in which we manipulated the contents of the shared virtual space to understand how users verbally construct a common spatial representation of their environment. Forty-four students participated in the experiment to assess the influence of contextual objects on spatial communication and sharing of viewpoints. The participants were asked to perform in dyads an object co-manipulation task. The results show that the presence of a contextual object such as fixed and lateralized visual landmarks in the virtual environment positively influences the way male operators collaborate to perform this task. These results allow us to provide some design recommendations for CVEs for object manipulation tasks.
Keywords: Spatial communication; Virtual environment; Collaboration; Common frame of reference; Visual landmarks
Perceiving affordances in virtual reality: influence of person and environmental properties in perception of standing on virtual grounds BIBAKFull-Text 17-28
  Tony Regia-Corte; Maud Marchal; Gabriel Cirio; Anatole Lécuyer
We evaluated the perception of affordances in virtual environments (VE). In our work, we considered the affordances for standing on a virtual slanted surface. Participants were asked to judge whether a virtual slanted surface supported upright stance. The objective was to evaluate whether this perception was possible in virtual reality (VR) and comparable to previous works conducted in real environments. We found that the perception of affordances for standing on a slanted surface in virtual reality is possible and comparable (with an underestimation) to previous studies conducted in real environments. We also found that participants were able to extract and to use virtual information about friction in order to judge whether a slanted surface supported an upright stance. Finally, results revealed that the person's position on the slanted surface is involved in the perception of affordances for standing on virtual grounds. Taken together, our results show quantitatively that the perception of affordances can be effective in virtual environments and influenced by both environmental and person properties. Such a perceptual evaluation of affordances in VR could guide VE designers to improve their designs and to better understand the effect of these designs on VE users.
Keywords: Perception of affordances; Visual perception; Posture; Slanted surface; Virtual environments; Head-mounted display
3D teleimmersion for collaboration and interaction of geographically distributed users BIBAKFull-Text 29-43
  Gregorij Kurillo; Ruzena Bajcsy
Teleimmersion is an emerging technology that enables users to collaborate remotely by generating realistic 3D avatars in real time and rendering them inside a shared virtual space. The teleimmersive environment thus provides a venue for collaborative work on 3D data such as medical imaging, scientific data and models, archaeological datasets, architectural or mechanical designs, remote training (e.g., oil rigs, military applications), and remote teaching of physical activities (e.g., rehabilitation, dance). In this paper, we present our research work performed over the course of several years in developing the teleimmersive technology using image-based stereo and more recently Kinect. We outline the issues pertaining to the capture, transmission, rendering, and interaction. We describe several applications where we have explored the use of the 3D teleimmersion for remote interaction and collaboration among professional and scientific users. We believe the presented findings are relevant for future developers in teleimmersion and apply across various 3D video capturing technologies.
Keywords: 3D video; 3D teleimmersion; Human-computer interaction; Remote collaboration; Telepresence
Haptic modules for palpatory diagnosis training of medical students BIBAKFull-Text 45-58
  Ernur Karadogan; Robert L., II Williams
We have developed and evaluated a novel tool based on haptics and virtual reality technology for augmenting the teaching of palpatory diagnosis. This novel tool can act as an automated expert and an animated textbook to illuminate palpatory diagnosis concepts by touch on a laptop PC and by using affordable haptic interfaces that can be housed in a medical student resource library. It can be used for unlimited student practice for improving skills from Osteopathic Manipulative Medicine laboratory and also as a repeatable and objective measure of palpatory skill to track student progress. The system was evaluated by 22 osteopathic medical students (16 first- and 6 second-year). The majority of the participating students (>90.9%) thought that future practice with the system may help them develop their palpatory skills. The majority (>77.3%) of the students also thought that the instructions on the module screens were clear. When the students were asked about the user interface, most of the students (>86.4%) responded that it was clear and easy to interpret. Evaluation results also showed that when the students were asked whether they would like to use the modules in the future for training at least 90.9% of them answered "Yes" or "Maybe." The achievement of purpose ratings for individual modules changed between 6.27 and 8.82 on a 10-point scale. This project has the potential to be extended from osteopathic medicine to allopathic medicine, veterinary medicine, physical therapy, massage therapy, and chiropractic schools.
Keywords: Haptic modules; Haptic medical simulation; Medical education; Osteopathic medicine; Palpatory diagnosis
Interaction design studio learning in virtual worlds BIBAKFull-Text 59-75
  Spyros Vosinakis; Panayiotis Koutsabasis
The paper suggests that virtual worlds (VWs) have many unique advantages for supporting interaction design studio activities, provided that they are designed to include appropriate workplaces and interactive tools to foster collaboration and creativity. We present an approach for employing VWs that proposes the use of prospective tools and workplaces throughout the following key activities of interaction design studio courses: design brief, design thinking, design practice (conceptual and detailed), the desk crit, design review and user evaluation. Then, we describe a blended interaction design studio course on the basis of this approach, which ran through a whole semester. We found that the VW design studio is an engaging and constructive experience for students: In the VW environment, students and tutors held many online meetings, and students constructed several models about their design project, developed a digital prototype and conducted a remote usability evaluation. In addition, the persistence of the environment and the developed VW tools helped students and tutors to achieve careful feedback and reflection during the design project lifetime. Nevertheless, a number of challenges remain for wider implementation: the refinement of the instructional design approach, the usability of VW tools, further integration of VWs to professional design tools and the conduction of other full-scale VW design studio courses.
Keywords: Virtual world; Design studio; Interaction design; Learning
Fast and stable simulation of virtual water scenes with interactions BIBAKFull-Text 77-88
  Shiguang Liu; Yuan Xiong
Simulation of large-scale water interacting with objects is essential in virtual reality, with wide applications in games, movie special effects, etc. As it involves much physical computation, how to achieve fast rendering is still a challenge. This paper proposed a novel Graphics Processing Unit-based method for rapid simulation of water interacting with objects. The interactions between dynamic objects and the surrounding environment were realized with a specially designed simulation grid. Perfectly Matched Layers method was introduced to ensure the continued stability of the simulation grid's boundary fluctuations. To model the rigid body efficiently, a pre-rigid body method was proposed to achieve plausible visual results at higher rendering rates. Various experiment results showed the validity of our method.
Keywords: Water; Rigid body; GPU; Perfectly matched layers; Interactions

VR 2013-06 Volume 17 Issue 2

Mixed and Augmented Reality

Interactive 3-D indoor modeler for virtualizing service fields BIBAKFull-Text 89-109
  Tomoya Ishikawa; Kalaivani Thangamani; Masakatsu Kourogi; Andrew P. Gee
This paper describes an interactive 3-D indoor modeler that effectively creates photo-realistic 3-D indoor models from multiple photographs. This modeler supports the creation of 3-D models from photographs by implementing interaction techniques that use geometric constraints estimated from photographs and visualization techniques that help to easily understand shapes of 3-D models. We evaluated the availability and usability by applying the modeler to model service fields where actual workers provide services and an experience-based exhibit. Our results confirmed that the modeler enables the creation of large-scale indoor environments such as hot-spring inns and event sites at a relatively modest cost. We also confirmed that school children could learn modeling operations and create 3-D models from a photograph for approximately 20 min because of the easy operations. In addition, we describe additional functions that increase the effectiveness of 3-D modeling based on knowledge from service-field modeling. We present applications for behavior analysis of service workers and for 3-D indoor navigation using augmented virtuality (AV)-based visualization realized by photo-realistic 3-D models.
Keywords: Interaction; 3-D indoor model; Service field; Augmented virtuality
Reducing the gap between Augmented Reality and 3D modeling with real-time depth imaging BIBAKFull-Text 111-123
  Svenja Kahn
Whereas 3D surface models are often used for augmented reality (e.g., for occlusion handling or model-based camera tracking), the creation and the use of such dense 3D models in augmented reality applications usually are two separated processes. The 3D surface models are often created in offline preparation steps, which makes it difficult to detect changes and to adapt the 3D model to these changes. This work presents a 3D change detection and model adjustment framework that combines AR techniques with real-time depth imaging to close the loop between dense 3D modeling and augmented reality. The proposed method detects the differences between a scene and a 3D model of the scene in real time. Then, the detected geometric differences are used to update the 3D model, thus bringing AR and 3D modeling closer together. The accuracy of the geometric difference detection depends on the depth measurement accuracy as well as on the accuracy of the intrinsic and extrinsic parameters. To evaluate the influence of these parameters, several experiments were conducted with simulated ground truth data. Furthermore, the evaluation shows the applicability of AR and depth image-based 3D modeling for model-based camera tracking.
Keywords: 3D modeling; Augmented Reality; Computer vision; Tracking; Depth imaging; Analysis-by-synthesis
Wii remote-based low-cost motion capture for automated assembly simulation BIBAKFull-Text 125-136
  Wenjuan Zhu; Anup M. Vader; Abhinav Chadda; Ming C. Leu; Xiaoqing F. Liu
This paper describes the development of a Wii remote (Wiimote) -- based low-cost motion capture system and demonstrates its application for automated assembly simulation. Multiple Wiimotes are used to form a vision system to perform motion capture in 3D space. A hybrid algorithm for calibrating a multi-camera stereo vision system has been developed based on Zhang's and Svoboda's calibration algorithms. This hybrid algorithm has been evaluated and shown accuracy improvement over Svoboda's algorithm for motion capture with multiple cameras. The captured motion data are used to automatically generate an assembly simulation of objects represented by CAD models in real time. The Wiimote-based motion capture system is practically attractive because it is inexpensive, wireless, and easily portable. Application examples have been developed for a single vision system with two Wiimotes to track the assembly of a microsatellite prototype frame and for an integrated vision system with four Wiimotes to track the assembly of a bookshelf.
Keywords: Wii remote; Low cost; Motion capture; Hybrid camera calibration algorithm; Assembly simulation
In-Situ interactive image-based model building for Augmented Reality from a handheld device BIBAKFull-Text 137-146
  Pished Bunnun; Sriram Subramanian; Walterio W. Mayol-Cuevas
Three-dimensional models of objects and their creation process are central for a variety of applications in Augmented Reality. In this article, we present a system that is designed for in-situ modeling using interactive techniques for two generic versions of handheld devices equipped with cameras. The system allows online building of 3D wireframe models through a combination of user interaction and automated methods. In particular, we concentrate in rigorous evaluation of the two devices and interaction methods in the context of 3D feature selection. We present the key components of our system, discuss our findings and results and identify design recommendations.
Keywords: Model building; Augmented Reality; Handheld device
Structure and motion in urban environments using upright panoramas BIBAKFull-Text 147-156
  Jonathan Ventura; Tobias Höllerer
Image-based modeling of urban environments is a key component of enabling outdoor, vision-based augmented reality applications. The images used for modeling may come from off-line efforts, or online user contributions. Panoramas have been used extensively in mapping cities and can be captured quickly by an end-user with a mobile phone. In this paper, we describe and evaluate a reconstruction pipeline for upright panoramas taken in an urban environment. We first describe how panoramas can be aligned to a common vertical orientation using vertical vanishing point detection, which we show to be robust for a range of inputs. The orientation sensors in modern cameras can also be used to correct the vertical orientation. Secondly, we introduce a pose estimation algorithm, which uses knowledge of a common vertical orientation as a simplifying constraint. This procedure is shown to reduce pose estimation error in comparison with the state of the art. Finally, we evaluate our reconstruction pipeline with several real-world examples.
Keywords: Structure and motion; Urban environments; Panoramas
Augmenting moving planar surfaces robustly with video projection and direct image alignment BIBAKFull-Text 157-168
  Samuel Audet; Masatoshi Okutomi; Masayuki Tanaka
Augmented reality applications based on video projection, to be effective, must track moving targets and make sure that the display remains aligned even when they move, but the projection can severely alter their appearances to the point where traditional computer vision algorithms fail. Current solutions consider the displayed content as interference and largely depend on channels orthogonal to visible light. They cannot directly align projector images with real-world surfaces, even though this may be the actual goal. We propose instead to model the light emitted by projectors and reflected into cameras and to consider the displayed content as additional information useful for direct alignment. Using a color camera, our implemented software successfully tracks with subpixel accuracy a planar surface of diffuse reflectance properties at an average of eight frames per second on commodity hardware, providing a solid base for future enhancements.
Keywords: Augmented reality; Vision-based tracking; Projector-camera systems; Video projection; Image alignment

VR 2013-09 Volume 17 Issue 3

RORSIM: a warship collision avoidance 3D simulation designed to complement existing Junior Warfare Officer training BIBAKFull-Text 169-179
  Neil Cooke; Robert Stone
Royal Navy Junior Warfare Officers (JWO) undergo a comprehensive training package in order to prepare them to be officers of the watch. One aspect of this training relates to their knowledge of the 'Rules of the Road' or 'COLREGS'; the rules for the manoeuvring and signalling that approaching vessels make in order to avoid collision. The training and assessment exercises undertaken predominantly use non-interactive static materials. These do not exercise the required skill in reconciling information from maritime charts, radar displays and 'out-of-the-window' monitoring. Consequently, performance during assessment on the VR-based bridge simulator falls short. This paper describes The Rules of the Road SIMulator (RORSIM) -- a proof of concept interactive 3D (i3D) simulator developed to bridge the training gap between classroom teaching and VR bridge simulator assessment. RORSIM's differentiation and its key functionality in terms of visualisaton, physics/interaction and game mechanics are influenced by the consideration of pedagogical learning models during requirements capture. This capture is formalised by a 'Training Gap Use Case' -- a graphical viewpoint using the Universal Modelling Language which can assist developers in requirements capture and development of i3D tools for existing training programmes. Key functionality, initial JWO feedback and a planned pilot study design are reported.
Keywords: Serious games; Game-based training; Virtual environments; Defence; Simulation
Spectrum-based synthesis of vibrotactile stimuli: active footstep display for crinkle of fragile structures BIBAKFull-Text 181-191
  Shogo Okamoto; Shun Ishikawa; Hikaru Nagano; Yoji Yamada
When a human crinkles or scrunches a fragile object, for which the yield force is very small that it is hardly perceived, they identify the material of the object based on tactile stimuli delivered to the skin. In addition, humans are able to recognize materials even when they are crinkled at different speeds. In order to realize these human recognition features of the crinkle of a fragile object, we develop a vibrotactile synthesis method. This method synthesizes the vibrotactile acceleration stimuli in response to a crinkle speed based on the preliminarily measured acceleration spectra. Using this method, we develop an active footstep display that presents a virtual crinkle of fragile structures made of different materials to its users. Experimental participants could identify three of the four types of virtual structure materials at rates significantly higher than the chance level. The four materials were copy and typing paper, aluminum foil, and polypropylene film. Furthermore, the trends of answer ratios exhibit good correspondence with those for the real cylindrical fragile objects. We conclude that the developed method is valid for the virtual crinkle of fragile structures and will enhance the validity of virtual reality systems, such as a virtual walkthrough system.
Keywords: Amplitude spectrum; Virtual material; Haptic interface
Mixing real and virtual conferencing: lessons learned BIBAKFull-Text 193-204
  Geetika Sharma; Ralph Schroeder
This paper describes a conference which linked several remote location sites via a virtual environment so that the virtual audience could follow the presentations and interact with real presenters. The aim was to assess the feasibility of linking distributed virtual audiences to an ongoing conference event. The conference consisted of an annual gathering of researchers and developers of a global information technology consultancy firm based in India. This firm developed a virtual environment specifically for distributed collaboration across sites. During the conference, researchers gathered various types of data, including participant observations, interviews, capture of the virtual environment and a survey of the audience. These data are analysed in the paper. The main finding is that a number of 'low tech' improvements could be made to the operation of the system that could greatly enhance this type of virtual conferencing. A related finding is that the visual fidelity of the environment and of the avatars plays a lesser role than other factors such as audio quality. Given the paucity of research on how virtual conferencing can substitute for travel, plus the urgency of this topic for environmental reasons, a number of suggestions are made for the implementation of remote virtual conference participation.
Keywords: Collaborative virtual environments; Real and virtual conferences
Haptic discrimination of virtual surface slope BIBAKFull-Text 205-218
  Inwook Hwang; Sunghoon Yim; Seungmoon Choi
We report the difference thresholds of the slope of a virtual surface rendered via a force-feedback haptic interface with the body frontal plane as a reference. The factors varied in experiments were the stiffness of a virtual plane, the lateral velocity with which the haptic probe scanned the plane, the length of a scanning interval, the movement direction of the probe to the body frontal plane (toward or away from the body), and lateral scanning direction (left-to-right or right-to-left). Measured slope thresholds ranged from 8.33° to 12.74° and were generally higher than or similar to previously published thresholds for haptic orientation or angle discrimination. The results suggested that haptic slope discriminability was independent of surface stiffness and lateral scanning velocity. Slope discrimination was largely affected by the lateral scan distance, indicating that the terminal difference of probe normal position can be an important sensory cue. In terms of scan direction, inward or rightward scans resulted in better slope discrimination than outward or leftward scans, respectively. These thresholds and findings have implications for haptics applications that involve geometric model modification or simplification of virtual objects while preserving their perceptual properties.
Keywords: Discrimination; Surface slope; Haptic rendering; Virtual surface; Mesh simplification; Mesh manipulation
An automatic method for motion capture-based exaggeration of facial expressions with personality types BIBAKFull-Text 219-237
  Seongah Chin; Chung Yeon Lee; Jaedong Lee
Facial expressions have always attracted considerable attention as a form of nonverbal communication. In visual applications such as movies, games, and animations, people tend to be interested in exaggerated expressions rather than regular expressions since the exaggerated ones deliver more vivid emotions. In this paper, we propose an automatic method for exaggeration of facial expressions from motion-captured data with a certain personality type. The exaggerated facial expressions are generated by using the exaggeration mapping (EM) that transforms facial motions into exaggerated motions. As all individuals do not have identical personalities, a conceptual mapping of the individual's personality type for exaggerating facial expressions needs to be considered. The Myers-Briggs type indicator, which is a popular method for classifying personality types, is employed to define the personality-type-based EM. Further, we have experimentally validated the EM and simulations of facial expressions.
Keywords: Facial expressions; Exaggeration; Facial motion capture; Facial motion cloning; Personality; MBTI; Nonnegative matrix factorization
Experiences in mixed reality-based collocated after action review BIBAKFull-Text 239-252
  John Quarles; Samsun Lampotang; Ira Fischler; Paul Fishwick
After action review (AAR) is a widely used training practice in which trainees and trainers review past training experiences and performance for the purpose of learning. AAR has often been conducted with video-based systems whereby a video of the action is reviewed afterward, usually at another location. This paper proposes collocated AAR of training experiences through mixed reality (MR). Collocated AAR allows users to review past training experiences in situ with the user's current, real-world experience, i.e., the AAR is conducted at the same location where the action being reviewed occurred. MR enables a user-controlled egocentric viewpoint, augmentation such as a visual overlay of virtual information like conceptual visualizations, and playback of recorded training experiences collocated with the user's current experience or that of an expert. Collocated AAR presents novel challenges for MR, such as collocating time, interactions, and visualizations of previous and current experiences. We created a collocated AAR system for anesthesia education, the augmented anesthesia machine visualization, and interactive debriefing system. The system enables collocated AAR in two applications related to anesthesia training: anesthesia machine operation training and skin disinfection training with a mannequin patient simulator. Collocated AAR was evaluated in two informal pilot studies by students (n=19) and an educator (n=1) not directly affiliated with the project. We review the anecdotal data collected from the studies and point toward ways to refine and improve collocated AAR.
Keywords: Mixed reality; After action review; Anesthesia machine; Human patient simulator; User studies; Skin prepping

VR 2013-11 Volume 17 Issue 4

Evaluation of tone mapping operators in night-time virtual worlds BIBAKFull-Text 253-262
  Josselin Petit; Roland Brémond; Ariane Tom
The subjective quality of a virtual world depends on the quality of displayed images. In the present paper, we address a technical aspect of image quality in virtual environments. Due to the recent development of high dynamic range (HDR) imaging in computer graphics applications, tone mapping operators (TMO) are needed in the graphic pipeline, and their impact on the final image quality needs to be tested. Previous evaluations of such operators have emphasized the fact that the specific merit of a given operator may depend on both the scene and the application. The dynamic behavior of tone mapping operators was not tested before, and we have designed two psychophysical experiments in order to assess the relevance of various TMO for a specific class of virtual worlds, outdoor scenes at night and an interactive application, to explore an outdoor virtual world at night. In a first experiment, 5 HDR video clips were tone-mapped using 8 operators from the literature, resulting in 40 videos. These 40 videos were presented to 14 subjects, which were asked to rate their realism. However, the subject's evaluation was not a direct comparison with the HDR videos. In a second experiment, 9 HDR photographs of urban scenes at night were tone-mapped with the same 8 operators. The resulting 72 photographs were presented to 13 subjects, at the location where the photographs were taken. The subjects were asked to rate the realism of each tone-mapped image, displayed on a laptop, with respect to the physical scene they experienced. The first experiment emphasized the importance of modeling the temporal visual adaptation for a night-time application.
Keywords: Virtual environments; Image rendering; Tone mapping; Presence; Subjective evaluation
Dwelling in Second Life? A phenomenological evaluation of online virtual worlds BIBAKFull-Text 263-278
  Chris Houliez; Edward Gamble
In previous research on virtual worlds, the question of whether virtual space can be evaluated just like "real world" space has not been fully addressed. This paper challenges the perceived commonsensical set of assumptions through which virtual world activities are usually unpacked and proposes a new method of evaluating virtual worlds based on Martin Heidegger's phenomenology. Various focus groups conducted in the virtual world Second Life confirmed that a phenomenological paradigm is more appropriate to fully make sense of and leverage this new medium. Besides questioning the relevancy of dealing with virtual worlds as if they were parallel spaces, this paper, by leveraging a new conceptualization of virtual worlds, also offers suggestions for new online qualitative methodologies.
Keywords: Virtual worlds; Second Life; Qualitative studies; Phenomenology; Presence
Augmented reality-based block piling game with superimposed collapse prediction BIBAKFull-Text 279-292
  Kazuya Okamoto; Naoto Kume; Tatsuya Tokunaga; Yoko Tanaka
Understanding what cannot be seen is difficult. Physical behavior can be explained on the basis of physical theories even if the behavior cannot be observed. Explanation of what is physically happening in the real world would become easy, however, if annotations were superimposed on the real objects. Herein, the authors demonstrate how an understanding of a physical event can be facilitated by overlapping a real-world situation with a simulation that predicts a future state. This idea is demonstrated in a game application in which a player stacks blocks into a pile until it collapses. In general, it is easy to estimate whether a block on the edge of a table will fall or not. However, it is more difficult to predict whether a stack of many blocks will collapse, and in what manner the stack will collapse. Even though previous research has demonstrated that the problem of how two-dimensionally stacked blocks collapse can be reduced to solving a sequence of convex quadratic programs, algorithms for convex quadratic programs require massive computational resources. Hence, the authors developed a fast and new algorithm based on a linear program. The proposed algorithm realizes real-time simulation based on physics that superimposes predicted collapse. The block that is predicted to fall is superimposed on the real block with a lit background projection. The system was evaluated in an experiment, and superimposed augmented reality annotation was observed to be efficient. The system was also demonstrated in game contests and received positive feedback and comments.
Keywords: Augmented reality; Block collapse; Linear program; Overhang problem; Physical simulation
A usability study of multimodal input in an augmented reality environment BIBAKFull-Text 293-305
  Minkyung Lee; Mark Billinghurst; Woonhyuk Baek; Richard Green
In this paper, we describe a user study evaluating the usability of an augmented reality (AR) multimodal interface (MMI). We have developed an AR MMI that combines free-hand gesture and speech input in a natural way using a multimodal fusion architecture. We describe the system architecture and present a study exploring the usability of the AR MMI compared with speech-only and 3D-hand-gesture-only interaction conditions. The interface was used in an AR application for selecting 3D virtual objects and changing their shape and color. For each interface condition, we measured task completion time, the number of user and system errors, and user satisfactions. We found that the MMI was more usable than the gesture-only interface conditions, and users felt that the MMI was more satisfying to use than the speech-only interface conditions; however, it was neither more effective nor more efficient than the speech-only interface. We discuss the implications of this research for designing AR MMI and outline directions for future work. The findings could also be used to help develop MMIs for a wider range of AR applications, for example, in AR navigation tasks, mobile AR interfaces, or AR game applications.
Keywords: Multimodal interface; Augmented reality; Usability; Efficiency; Effectiveness; Satisfaction
Human perception of a conversational virtual human: an empirical study on the effect of emotion and culture BIBAKFull-Text 307-321
  Chao Qu; Willem-Paul Brinkman; Yun Ling; Pascal Wiggers
Virtual reality applications with virtual humans, such as virtual reality exposure therapy, health coaches and negotiation simulators, are developed for different contexts and usually for users from different countries. The emphasis on a virtual human's emotional expression depends on the application; some virtual reality applications need an emotional expression of the virtual human during the speaking phase, some during the listening phase and some during both speaking and listening phases. Although studies have investigated how humans perceive a virtual human's emotion during each phase separately, few studies carried out a parallel comparison between the two phases. This study aims to fill this gap, and on top of that, includes an investigation of the cultural interpretation of the virtual human's emotion, especially with respect to the emotion's valence. The experiment was conducted with both Chinese and non-Chinese participants. These participants were asked to rate the valence of seven different emotional expressions (ranging from negative to neutral to positive during speaking and listening) of a Chinese virtual lady. The results showed that there was a high correlation in valence rating between both groups of participants, which indicated that the valence of the emotional expressions was as easily recognized by people from a different cultural background as the virtual human. In addition, participants tended to perceive the virtual human's expressed valence as more intense in the speaking phase than in the listening phase. The additional vocal emotional expression in the speaking phase is put forward as a likely cause for this phenomenon.
Keywords: Virtual reality; Virtual human; Emotion; Affective computing; Culture