HCI Bibliography Home | HCI Conferences | HCII Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
HCII Tables of Contents: 11-511-613-113-213-313-413-513-613-714-114-214-314-414-515-115-215-315-415-5

HCI International 2014: 16th International Conference on HCI, Part II: Advanced Interaction Modalities and Techniques

Fullname:HCI International 2014: 16th International Conference on Human-Computer Interaction, Part II: Advanced Interaction Modalities and Techniques
Editors:Masaaki Kurosu
Location:Heraklion, Crete, Greece
Dates:2014-Jun-22 to 2014-Jun-27
Volume:2
Publisher:Springer International Publishing
Series:Lecture Notes in Computer Science 8511
Standard No:DOI: 10.1007/978-3-319-07230-2 hcibib: HCII14-2; ISBN: 978-3-319-07229-6 (print), 978-3-319-07230-2 (online)
Papers:75
Pages:805
Links:Online Proceedings | Conference Website
  1. HCII 2014-06-22 Volume 2
    1. Gesture-Based Interaction
    2. Gesture, Gaze and Activity Recognition
    3. Speech, Natural Language and Conversational Interfaces
    4. Natural and Multimodal Interfaces
    5. Human-Robot Interaction
    6. Emotions Recognition

HCII 2014-06-22 Volume 2

Gesture-Based Interaction

RemoteHand: A Wireless Myoelectric Interface BIBAKFull-Text 3-11
  Andreas Attenberger; Klaus Buchenrieder
While myoeletric signals (MES) have long been employed for actuating hand prostheses, their potential as novel input for the interaction with computer systems has received little attention up until now. In this contribution, we present RemoteHand, a system that fosters remote device control through the transmission of myoelectric data over WLAN. This allows to manipulate objects through the user's muscle activity regardless of their physical location. In our setup, a mechanical hand is controlled through electromyographic (EMG) sensors placed over the user's forearm muscles. This approach is compared to a conventional remote device control exercised by a tablet touchpad. The results of our user study show that wireless interaction through myoelectric signals is a valid approach. Study participants achieved interaction speeds equal to those of a standard input method. Users especially value myoelectric input with regard to novelty and stimulation.
Keywords: EMG; Myoelectric Signals; Prosthetic Hand; Remote Control; Wireless
Early Prototyping of 3D-Gesture Interaction within the Presentation-Gesture-Dialog Design Space BIBAKFull-Text 12-23
  Birgit Bomsdorf; Rainer Blum
Development of gesture interaction requires a combination of three design matters: presentation, gesture and dialog. In this contribution a first version of the tool ProGesture is introduced. The objective of its development is to cope with the resulting presentation-gesture-dialog design space in a flexible way. On the one hand, it aims at the early development phases, i.e. at rapid prototyping of 3D-gestures in combination with first UI sketches, such as mockups. On the other hand, it focuses on dialog and presentation modeling, and on testing based on executable models aiming at a smooth transition from informal UI sketches to formal models.
Keywords: 3D-Gesture Interaction; Early Prototyping; Model-Based Development
The Study of the Full Cycle of Gesture Interaction, The Continuum between 2D and 3D BIBAKFull-Text 24-35
  Mohamed-Ikbel Boulabiar; Gilles Coppin; Franck Poirier
The goal of HCI researchers is to make interaction with computer interfaces simpler, efficient and more natural. In a context of object manipulation, we think that reaching this goal requires the ability to predict and recognize how humans grasp then manipulate objects. This is based on studies explaining human vision, reach, grasp taxonomies and manipulations. In this paper, we study the full cycle of gesture interaction using different points of view, then attempt to organize them using Norman's theory of Human Action, we link the psychology of object sensing to HCI goals and propose a simplification of gestures classes into four principal families. Our simplification of gestures classes still allow the expression of more detailed subclasses differentiated by the gesture properties.
Keywords: Gesture; 3D; Interaction; Hand; Grasping
iPanel: A Computer-Vision Based Solution for Interactive Keyboard and Mouse BIBAKFull-Text 36-47
  H. Chathushka Dilhan Hettipathirana; Pragathi Weerakoon
This paper represents an implementation of a computer vision based interface; iPanel which employs an arbitrary panel and tip pointers as a spontaneous, wireless and mobility device. Also the proposed system can accurately identify the tip movements of the panel and simulate the relevant events on the target environment. By detecting the key pressing, mouse clicking and dragging actions, the system can fulfill many tasks. Therefore, it enables users to use their fingers naturally to interact with any application as well as with any mobility enabled devices.
Keywords: Computer vision; Human computer interaction; gesture recognition; optical character recognition; wearable computing
Adding Multi-Touch Gesture Interaction in Mobile Web Applications BIBAKFull-Text 48-57
  Shah Rukh Humayoun; Franca-Alexandra Rupprecht; Steffen Hess; Achim Ebert
This paper describes the MTGest framework, an open library for adding multi-touch gesture interaction to HTML-based mobile web applications. MTGest was used in a comparative study to evaluate the multi-touch gesture interaction in a mobile web application in comparison to a native iOS mobile application. The results indicates that in most cases the web based gestures efficiency is either approximately the same or higher than the iOS-based app. The study was carried out as an initial experiment using isolated gestures, targeting the iOS platform only. For generalizing the results there is a need to perform detailed user evaluation studies with different platforms and for more complex interaction scenarios.
Keywords: Smart Devices; Smartphones; Tablets; Mobile Apps; Web Apps; Multi-Touch Gesture; Interaction Design; Mobile Environments
Harmonic Navigator: An Innovative, Gesture-Driven User Interface for Exploring Harmonic Spaces in Musical Corpora BIBAKFull-Text 58-68
  David Johnson; Bill Manaris; Yiorgos Vassilandonakis
We present Harmonic Navigator (HN), a system for navigating and exploring harmonic spaces extracted from large musical corpora, to be used in music composition and performance. A harmonic space is a set of harmonies (chords) and transitions between harmonies found in a music corpus. By navigating this space, the user can derive new harmonic progressions, which have correct voice leading. HN is controllable via a Kinect gesture interface. To aid the user, the system also incorporates stochastic and evolutionary techniques. HN offers for two primary modes of interaction: a harmonic transition selector, called harmonic palette, which utilizes a GUI to navigate harmonic transitions in a front-to-back manner; and a harmonic-flow scrubber, which presents a global overview of a harmonic flow and allows the user to perform common audio scrubbing and editing tasks. Both GUIs use colors to indicate harmonic density based on Legname's density degree theory.
Keywords: harmonic navigation; computer music; graphical user interface; gesture language; Kinect sensor; harmonic space; music composition; music performance
HandyScope: A Remote Control Technique Using Circular Widget on Tabletops BIBAKFull-Text 69-80
  Takuro Kuribara; Yusaku Mita; Kazusa Onishi; Buntarou Shizuki; Jiro Tanaka
A large multi-touch tabletop has remote areas that the users might not touch by their hands. This forces users to move around the tabletop. In this paper, we present a novel remote control technique which we call HandyScope. This technique allows users to manipulate those remote areas. Moreover, users can move an object between the nearby area and the remote areas using a widget. In addition, users can precisely point a remote area quickly because this system includes our proposed control-display ratio changing system. To evaluate the performance of HandyScope, we compared HandyScope with direct touch manipulation. The results show that HandyScope is significantly faster in selection.
Keywords: bimanual interaction; multi-touch; gesture; dynamic control-display gain; pointing; target acquisition; pull-out
Comparing Hand Gesture Vocabularies for HCI BIBAFull-Text 81-92
  Alexander Mehler; Tim vor der Brück; Andy Lücking
HCI systems are often equipped with gestural interfaces drawing on a predefined set of admitted gestures. We provide an assessment of the fitness of such gesture vocabularies in terms of their learnability and naturalness. This is done by example of rivaling gesture vocabularies of the museum information system WikiNect. In this way, we do not only provide a procedure for evaluating gesture vocabularies, but additionally contribute to design criteria to be followed by the gestures.
Effectiveness of Virtual Hands in 3D Learning Material BIBAKFull-Text 93-101
  Tetsufumi Mikami; Shu Matsuura
A virtual reality model for a motional electromotive force physics experiment, "Fleming's rail," was designed and developed. A hand gesture interface was constructed to control a virtual simulation using a Microsoft Kinect sensor and a finger-gesture interface SDK. A gesture-based object tracking test was performed to examine the effects of virtual hand visualization. In addition, motion trajectories of real hands with and without hand visualization were analyzed. Trajectories obtained with hand visualization exhibited higher Hurst exponent values compared with those obtained without virtual hand visualization. This suggests that the displacement change was more persistent with positive fluctuation feedback, indicating sensory feedback for real hand motions. For comparison, the effects of the model on learning Fleming's left- and right-hand rules were experimentally tested. Results exhibited that knowledge acquisition from the model was almost equivalent to that from the real experiment.
Keywords: Hand gesture interface; virtual reality learning material; Hurst exponent
Proposal of the Effective Method of Generating Characteristic Gestures in Nonverbal Communication BIBAKFull-Text 102-112
  Toshiya Naka; Toru Ishida
According to the rapid spread of the Internet, the new devices and web applications using the newest multimedia technologies are proposed one after another and they become commodity in an instant. In these new web communications, the natural and intelligible interaction corresponding to the user's various demands is required. In the communication in which persons do the direct dialogue in the interaction not only on the web but also in real world, it is widely known by the psychology field that the nonverbal information which is hard to express in words such as expression of face and gesture is playing the important role. In our research, the new analysis method of interaction using the dynamical model is proposed and paid our attention to the characteristic gestures especially. These gestures are the special motions such as lively or powerful actions which used effectively in Kabuki, anime, dance and the special gestures in the speech and presentation of attracting audiences. By analyzing the mechanisms of these characteristic gestures mathematically, we can design the new interactive interfaces easily which are natural and familiar for all users.
Keywords: Nonverbal Communication
Hand-Object Interaction: From Grasping to Using BIBAKFull-Text 113-120
  Long Ni; Ye Liu; Xiaolan Fu
Evidence from psychology has shown that visual man-made manipulable objects can afford grasping actions even without the observers' intention to grasp them, and humans are able to use grasping information to recognize objects. But little is known if visual man-made objects, especially tools, can potentiate much more complex actions associated with using an object. In the present study, a priming paradigm was used to explore if passively viewing manipulable objects could be enough to activate specific action information about how to use them. The results showed that target objects with similar functional manipulation information to the prime objects were identified significantly faster than that with dissimilar manipulation knowledge to the prime objects. This is the first evidence by using behavioral study to indicate that just passively viewing a manipulable object is sufficient to activate its specific manipulation information that could facilitate object identification even without participants' intention to use them. The implications of manipulation knowledge in object affordances and object representation are discussed.
Keywords: Structural manipulation; Functional manipulation; Object recognition; Object affordances
Model-Based Multi-touch Gesture Interaction for Diagram Editors BIBAKFull-Text 121-130
  Florian Niebling; Daniel Schropp; Romina Kühn; Thomas Schlegel
Many of todays software development processes include model-driven engineering techniques. They employ domain models, i.e. formal representations of knowledge about an application domain, to enable the automatic generation of parts of a software system. Tools supporting model-driven engineering for software development today are often desktop-based single user systems. In practice though, the design of components or larger systems often still is conducted on whiteboards or flip charts. Our work focuses on interaction techniques allowing for the development of gesture-based diagram editors that support teams in establishing domain models from a given meta-model during the development process. Users or groups of users are enabled to instantiate meta-models by free-hand or pen-based sketching of components on large multi-touch screens. In contrast to previous work, the description of multi-touch gestures is derived directly from the graphical model representing the data.
Keywords: Multi-touch gestures; model-based development
Multi-sensor Finger Ring for Authentication Based on 3D Signatures BIBAKFull-Text 131-138
  Mehran Roshandel; Aarti Munjal; Peyman Moghadam; Shahin Tajik; Hamed Ketabdar
Traditional methods of authenticating a user, including password, a Personal Identification Number (PIN), or a more secure PIN entry method (A PIN entry method resilient against shoulder surfing [14]), can be stolen or accessed easily and, therefore, make the authentication unsecure. In this work, we present the usability of our multi-sensor based and standalone finger ring called Pingu in providing a highly secure access system. Specifically, Pingu allows users to make a 3D signature and record the temporal pattern of the signature via an advanced set of sensors. As a result, the user creates a 3D signature in air using his finger. Our approach has two main contributions: (1) Compared to other wearable devices, a finger ring is more socially acceptable, and (2) signatures created via a finger in the air or on a surface leaves no visible track and, thus, are extremely hard to forge. In other words, a 3D signature allows much higher flexibility in choosing a safe signature. Our experiment shows that the proposed hardware and methodology could result in a very high level of user authentication/identification performance.
Keywords: Human Computer Interaction (HCI); Touch less gestural interaction; Wearable device; Finger ring
What You Draw Is What You Search: The Analog Gesture BIBAKFull-Text 139-147
  Benoit Rouxel; Franck Poirier; Jean-Yves Antoine; Gilles Coppin
This paper presents a new type of gesture for identifying spatio-temporal patterns: the analog gesture. Analog gestures can be characterized by some features (speed, acceleration, direction, and angle) which describe the dynamic morphology of the gesture. At first, we detail interactive tasks that should benefit for the use of analog gestures. Then we give a state of the art concerning gesture recognition and investigate the specificity and the main properties of the analog gesture. Then, we propose a review of the surveillance maritime system called Hyperion which uses analog gestures. Finally, we give an example of the use of this type of gesture by the operator. It concerns the interactive detection of ship abnormal trajectories in the context of maritime surveillance.
Keywords: Gesture recognition; time-space pattern search; tabletop computing
Remote Collaboration with Spatial AR Support BIBAKFull-Text 148-157
  Nobuchika Sakata; Yuuki Takano; Shogo Nishida
Typical view sharing system has same camera alignment that camera take images from back of remote instructor. We change this alignment to camera take images from front of remote instructor for preventing occlusions caused by a body of remote instructor self. Also as visual feedbacks, a mirror image of remote instructor is indicated in display of remote instructor side. Eventually remote instructor can confirm own instruction in the display. Therefore due to displaying the mirror image of remote instructor and changing camera alignment, we proposed and implement a novel remote collaboration system which prevents occlusion problems caused by instructor body self when he/she sends clear instructions by whole body gesture and allows instructor to use direct manipulation.
Keywords: Remote collaboration; Occlusion; Augmented Reality; View sharing system; Spatial AR
Prediction of Multi-touch Gestures during Input BIBAKFull-Text 158-169
  Michael Schmidt; Gerhard Weber
In the work at hand, a method is presented that can predict gestures during input. The scheme is based on the specification of prominent points defining subgestures within templates. Classification of a partial input is only against a small set of subgestures pre-selected by nearest neighbor searches regarding these prominent points. The gesture prediction is invariant against variations in scale, rotation, translation and speed of an input and handles single-touch, single-stroke and (sequential) multi-touch gestures. We provide thorough investigations of the classifiers performance on tests with two medium sized gesture sets. Results are promising and feasible for a wide range of applications. Even common direct manipulation operations can be reliably detected.
Keywords: gestures; multi-touch; prediction; classification; template-based
"Will Use It, Because I Want to Look Cool" A Comparative Study of Simple Computer Interactions Using Touchscreen and In-Air Hand Gestures BIBAKFull-Text 170-181
  Vidya Vaidyanathan; Daniel Rosenberg
The Xbox Kinect and now the Leap Motion Controller have brought about a paradigm shift in the way we interact with computers by making the recognition of 3D gestures affordable. Interfaces now understand natural user interfaces, integrating gestures, voice and various other kinds of multi-modal input simultaneously. In this paper we attempted to understand in-air gesturing better. The purpose of the study was to understand differences between touchscreen and in-air gesturing for simple human computer interactions. The comparison of the gestures was done in terms of Muscle effort/fatigue and Frustration, Satisfaction and Enjoyment We have also tried to study the learnability of in-air gesturing. In our research we found that in-air gesturing was significantly superior with respect to muscle effort and fatigue when compared with touchscreens. We also found that in-air gesturing was found to be more fun and preferred because of its "coolness factor". Lastly, in-air gesturing had a rapid learning curve.
Keywords: HCI; Touch Screens; in-air gestures; ergonomics; EMG; learnability; social acceptability; natural user interfaces (NUI)
Beyond Presentation -- Employing Proactive Intelligent Agents as Social Catalysts BIBAFull-Text 182-190
  Madlen Wuttke; Michael Heidt
Despite long standing attention from research communities, the technology of intelligent agents still harbours a large amount of unrealised potential. In this text, we argue that agent technology can benefit from a shift in focus from presentation to possible functionalities. In doing this, our focus is on the provision of pro-activity: The ability of agents not to merely react but to predictively shape their environments. In order to illustrate our arguments, we present an instance of interactive technology, showing how pro-active intelligent agents can be employed in exhibition contexts.
A Method for Lifelong Gesture Learning Based on Growing Neural Gas BIBAKFull-Text 191-202
  Paul M. Yanik; Anthony L. Threatt; Jessica Merino; Joe Manganelli; Johnell O. Brooks; Keith E. Green; Ian D. Walker
Gesture-based interfaces offer the possibility of an intuitive command language for assistive robotics and ubiquitous computing. As an individual's health changes with age, their ability to consistently perform standard gestures may decrease, particularly towards the end of life. Thus, such interfaces will need to be capable of learning commands which are not choreographed ahead of time by the system designers. This circumstance illustrates the need for a system which engages in lifelong learning and is capable of discerning new gestures and the user's desired response to them. This paper describes an innovative approach to lifelong learning based on clustered gesture representations identified through the Growing Neural Gas algorithm. The simulated approach utilizes a user-generated reward signal to progressively refine the response of an assistive robot toward a preferred goal configuration.
Keywords: machine learning; gesture recognition; human-robot interaction; assistive robotics

Gesture, Gaze and Activity Recognition

The Issues of 3D Hand Gesture and Posture Recognition Using the Kinect BIBAKFull-Text 205-214
  Mohamed-Ikbel Boulabiar; Gilles Coppin; Franck Poirier
Besides the emergence of many input devices and sensors, they are still unable to provide good and simple recognition of human postures and gestures. The recognition using simple algorithms implemented on top of these devices (like the Kinect) enlarges use cases for these gestures and postures to newer domains and systems. Our methods cuts the needed computation and allow the integration of other algorithms to run in parallel. We present a system able to track the hand in 3D, log its position and surface information during the time, and recognize hand postures and gestures. We present our solution based on simple geometric algorithms, other tried algorithms, and we discuss some concepts raised from our tests.
Keywords: Gesture; Posture; 3D; Kinect; Interaction; Hand
Frontal-Standing Pose Based Person Identification Using Kinect BIBAFull-Text 215-223
  Kingshuk Chakravarty; Tanushyam Chattopadhyay
In this paper we propose a person identification methodology from frontal standing posture using only skeleton information obtained from Kinect. In the first stage, features related to the physical characteristic of a person are calculated for every frame and then noisy frames are removed based on these features using unsupervised learning based approach. We have also proposed 6 new angle and area related features along with the physical build of a person for the supervised learning based identification. Experimental results indicate that the proposed algorithm is able to achieve 96% recognition accuracy and outperforms all the stat-of-the-art methods suggested by Sinha et al. and Preis et al.
A Virtual Handwriting Tablet Based on Pen Shadow Cues BIBAKFull-Text 224-233
  Chin-Shyurng Fahn; Bo-Yuan Su; Meng-Luen Wu
The handwriting tablet is an electronic product, which is a kind of human-computer interfaces acting as a computer input device comprising a set of a special pen and a tablet. The user holds the pen to draw contents within a region of the tablet as inputs, which imitates handwriting and is a replacement of mouse inputs. Some handwriting tablets not only imitate the handwriting and mouse functions, but also detect the pen tilts and pressures. The tilt and pressure information can be applied to some drawing software which can also render the thickness and depth of strokes. However, since the handwriting tablet is a piece of precise equipment, it has some drawbacks -- fragile, not easy to carry, and the weight is often heavy. Therefore, in this paper, we propose a new concept based on the computer vision technology to simulate the handwriting tablet. We put a rectangular plane in the FOV of a video camera to emulate a tablet, and use a conventional pen to emulate the stylus. Many experiments have been made for evaluating the effectiveness of the proposed methods. The performance of such a virtual handwriting tablet is very satisfactory and encouraged.
Keywords: virtual handwriting tablet; shadow cues; computer vision; human-computer interface
HOUDINI: Introducing Object Tracking and Pen Recognition for LLP Tabletops BIBAKFull-Text 234-244
  Adrian Hülsmann; Julian Maicher
Tangible objects on a \tabletop offer a lot of different opportunities to interact with an application. Most of the current tabletops are built using optical tracking principles and especially LLP tabletops provide very good tracking results for touch input. In this paper we introduce HOUDINI as a method for LLP object tracking and pen recognition, which is based on three different sizes of touch points that help us to identify touch points belonging to fingers, objects and pens. As a result, the whole recognition process is performed at the level of touch information rather than frame by frame image analysis. This leads to a very efficient and reliable tracking, thus allowing the objects to be moved very fast without being lost.
Keywords: tabletop; interactive surface; object tracking; LLP; pen recognition
Detecting Address Estimation Errors from Users' Reactions in Multi-user Agent Conversation BIBAKFull-Text 245-253
  Ryo Hotta; Hung-Hsuan Huang; Shochi Otogi; Kyoji Kawagoe
Nowadays, embodied conversational agents are gradually getting deployed in real-world applications like the guides in museums or exhibitions. In these applications, it is necessary for the agent to identify the addressee of each user utterance to deliberate appropriate responses in interacting with visitor groups. However, as long as the addressee identification mechanism is not completely correct, the agent makes error in its responses. Once there is an error, the agent's hypothesis collapses and the following decision-making path may go to a totally different direction. We are working on developing the mechanism to detect the error from the users' reactions and the mechanism to recover the error. This paper presents the first step, a method to detect laughing, surprises, and confused facial expressions after the agent's wrong responses. This method is machine learning base with the data (user reactions) collected in a WOZ (Wizard of Oz) experiment and reached an accuracy over 90%.
Keywords: Multi-party conversation; human-agent interaction; Gaze
Evaluation of Leap Motion Controller with a High Precision Optical Tracking System BIBAKFull-Text 254-263
  Grega Jakus; Joze Guna; Sašo Tomazic; Jaka Sodnik
The paper presents an evaluation of the performance of a Leap Motion Controller. A professional optical tracking system was used as a reference system. 37 stationary points were tracked in 3D space in order to evaluate the consistency and accuracy of the Controller's measurements. The standard deviation of these measurements varied from 8.1 µm to 490 µm, mainly depending on the azimuth and distance from the Controller. In the second part of the experiment, a constant distance was provided between two points, which were then moved and tracked within the entire sensory space. The deviation of the measured distance changed significantly with the height above the Controller. The sampling frequency also proved to be very non-uniform. The Controller represents a revolution in the field of gesture-based human-computer interaction; however, it is currently unsuitable as a replacement for professional motion tracking systems.
Keywords: Leap Motion Controller; motion capture system; consistency; accuracy
Proposal of a Method to Measure Difficulty Level of Programming Code with Eye-Tracking BIBAKFull-Text 264-272
  Tomoko Kashima; Shimpei Matsumoto; Shuichi Yamagishi
In recent years, guaranteeing the educational quality is required in university education of Japan. With this situation in mind, we built study support environment with the information technology. As a result, we utilized the result for programming education and obtained the effect. There are various technical elements in the programming skill. However, many evaluations have adopted a comprehensive evaluation method. Therefore, a student's attainment to each technical element is indefinite. Some students become difficult to perform learning activities. So, in this research, programming notes the point which is the implicit thinking skill which is strongly related in study. Accumulation experience analyzes strongly related eye movement, and we aim at the standard construction for skill.
Keywords: programming; difficulty level; educational support; eye-tracking
Expressing Observation Direction through Face and Body Rotation in a Multi-user Conversation Setting BIBAKFull-Text 273-280
  Satoshi Mieda; Shiro Ozawa; Munekazu Date; Hideaki Takada; Yoshiaki Kurokawa; Akira Kojima
In this paper we clarified the range of observing direction by rotating the 2D human image and it is possible to express the observing direction by face direction. We conducted two subjective experiments about direction expression of the person on an image. In the first experiment, we compared two types of human image expression, rotated 2D human image of rotated 2D and direction correct. In the second experiment, we evaluated the effect of human image rotation and the criterion for judging the direction. We showed that the direction of the user's face is the main factor in expressing the observation direction. Results clearly showed that it is possible to express the observation direction, which is required for effective communication, by using only the rotation of human facial image.
Keywords: communication; remote; human expression
Gaze Location Prediction with Depth Features as Auxiliary Information BIBAFull-Text 281-292
  Redwan Abdo A. Mohammed; Lars Schwabe; Oliver Staadt
We present the results of a first experimental study to improve the computation of saliency maps, by using luminance and depth images features. More specifically, we have recorded the center of gaze of users when they were viewing natural scenes. We used machine learning techniques to train a bottom-up, top-down model of saliency based on 2D and depth features/cues. We found that models trained on Itti & Koch and depth features combined outperform models trained on other individual features (i.e. only Gabor filter responses or only depth features), or trained on combination of these features. As a consequence, depth features combined with Itti & Koch features improve the prediction of gaze locations. This first characterization of using joint luminance and depth features is an important step towards developing models of eye movements, which operate well under natural conditions such as those encountered in HCI settings.
Study and Evaluation of Separability Techniques and Occlusion in Multitouch Surfaces BIBAKFull-Text 293-304
  Jessica Palomares; Manuel Loaiza; Alberto Raposo
Multitouch interfaces allow interacting with a virtual object directly, similar to a real object. However, there are several issues to be resolved, such as the accuracy of the manipulation, the occlusion, the separability of the manipulation, etc. Multitouch interfaces allow multiple spatial transformations that can be performed on a virtual object with only a gesture. For example, an object can be rotated, translated and scaled with two fingers with a single gesture. However, some unwanted movements may occur accidentally. Separability techniques appear with the intent to prevent unwanted movements on multitouch surfaces. Occlusion is another problem that occurs in multitouch interfaces. Often the user's hand hides the vision of the object with which he/she interacts; or the user's action on interface hinders the movement when it clicks on a bottom that triggers action. This paper proposes two techniques of separability, aiming to reduce the problems that arise due to excessive freedom of manipulation in multi-touch interfaces, and evaluates the efficiency of these techniques. The techniques developed are not only applicable in simple virtual objects; they are also for WIMP (windows, icons, menus, pointer) objects, aiming to reduce occlusion. A series of tests was performed to evaluate precision, occlusion time for completion of task, and ease of use.
Keywords: Human-Computer Interaction; multitouch interaction; Separability; Occlusion; Spatial Transformation
Human Activity Recognition from Kinect Captured Data Using Stick Model BIBAKFull-Text 305-315
  Vempada Ramu Reddy; Tanushyam Chattopadhyay
In this paper authors have presented a method to recognize basic human activities such as sitting, walking, laying, and standing in real time using simple features to accomplish a bigger goal of developing an elderly people health monitoring system using Kinect. We have used the skeleton joint positions obtained from the software development kit (SDK) of Microsoft as the input for the system. We have evaluated our proposed system against our own data set as well as on a subset of the MSR 3Ddaily activity data set and observed that our proposed method out performs state-of-the-art methods.
Keywords: Human activity; Human action; Kinect; Skeleton; Activity recognition
Multi-sensor Based Gestures Recognition with a Smart Finger Ring BIBAKFull-Text 316-324
  Mehran Roshandel; Aarti Munjal; Peyman Moghadam; Shahin Tajik; Hamed Ketabdar
Recently several optical and non-optical sensors based gesture recognition techniques have been developed to interact with computing devices. However, these techniques mostly suffer from problems such as occlusion and noise. In this work, we present Pingu, a multi-sensor based framework that is capable of recognizing simple, sharp, and tiny gestures without the problems mentioned above. Pingu has been calibrated in the form of a wearable finger ring, capable of interacting even when the device is not in the vicinity of the user. An advanced set of sensors, wireless connectivity, and feedback facilities enable Pingu for a wide range of potential applications, from novel gestures to social computing. In this paper, we present our results based on experiments conducted to explore Pingu's use as a general gestural interaction device. Our analysis, based on simple machine learning algorithms, shows that simple and sharp gestures performed by a finger can be detected with a high accuracy, thereby, establishing Pingu as a wearable ring to control a smart environment effectively.
Keywords: Human Computer Interaction (HCI); Touch less gestural interaction; Wearable device; Finger ring
View-Invariant Human Detection from RGB-D Data of Kinect Using Continuous Hidden Markov Model BIBAFull-Text 325-336
  Sangheeta Roy; Tanushyam Chattopadhyay
In this paper authors have presented a method to detect human from a Kinect captured Gray-Depth (G-D) using Continuous Hidden Markov models (C-HMMs). In our proposed approach, we initially generate multiple gray scale images from a single gray scale image/ video frame based on their depth connectivity. Thus, we initially segment the G image using depth information and then relevant components were extracted. These components were further filtered out and features were extracted from the candidate components only. Here a robust feature named Local gradients histogram (LGH) is used to detect human from G-D video. We have evaluated our system against the data set published by LIRIS in ICPR 2012 and on our own data set captured in our lab. We have observed that our proposed method can detect human from this data-set with a 94.25% accuracy.
A Survey of Datasets for Human Gesture Recognition BIBAKFull-Text 337-348
  Simon Ruffieux; Denis Lalanne; Elena Mugellini; Omar Abou Khaled
This paper presents a survey on datasets created for the field of gesture recognition. The main characteristics of the datasets are presented on two tables to provide researchers a clear and rapid access to the information. This paper also provides a comprehensive description of the datasets and discusses their general strengths and limitations. Guidelines for creation and selection of datasets for gesture recognition are proposed. This survey should be a key-access point for researchers looking to create or use datasets in the field of human gesture recognition.
Keywords: human-computer interaction; gesture recognition; datasets; survey

Speech, Natural Language and Conversational Interfaces

Accessing Cause-Result Relation and Diplomatic Information in Ancient "Journalistic" Texts with Universal Words BIBAKFull-Text 351-361
  Christina Alexandris
For the International Public, ancient historical and "journalistic" texts, such the "Peloponnesian War" of the Ancient Greek historian Thucydides, may allow an insight for the understanding of current international political and economic relations. The present approach targets to facilitate the accessibility of such texts for non-experts in the International Public, with no knowledge of the ancient language concerned, especially journalists, translators and students. The possibility of directly accessing text content and viewing features, as close as possible to the original text is attempted to be achieved here, using predefined sublanguage-specific keywords and Universal Words.
Keywords: Ancient Greek; keyword ontology; Universal Words; International Public; online Machine Translation
Human Factors in the Design of Arabic-Language Interfaces in Assistive Technologies for Learning Difficulties BIBAKFull-Text 362-369
  Sahar Alkhashrami; Huda Alghamdi; Areej Al-Wabil
This paper reports on insights gained from collaborations between multi-disciplinary research teams and practitioners in a Disability Service Center in King Saud University (KSU) in Saudi Arabia. Projects were conducted in the context of designing, developing and evaluating different assistive technologies in the university's Software and Knowledge Engineering Research Group. In these projects, methodological considerations have been reported for effectively involving domain specialists in research and development projects for assistive technologies. Subject Matter Experts (SMEs) are often involved in the technology design cycles of these projects in various roles (e.g. design partners, design informants, testers). This paper highlights the human factors relevant for the design and evaluation of interactive systems for SpLDs that were synthesized from these collaborative contexts. We also shed light on issues to consider in the design partnerships between researchers and practitioners for requirements engineering and user acceptance testing phases of system development. Implications for the design and development of systems for SpLDs in other languages and cultural contexts are discussed.
Keywords: SpLD; Learning Difficulty; Dyslexia; Brain-Computer Interaction; BCI; Usability; User Experience; Disability; Attention Deficit Disorder; ADHD; Augmentative and Alternative Communication; AAC; Arabic Interfaces
Design and Development of Speech Interaction: A Methodology BIBAKFull-Text 370-381
  Nuno Almeida; Samuel Silva; António Teixeira
Using speech in computer interaction is advantageous in many situation and more natural for the user. However, development of speech enabled applications presents, in general, a big challenge when designing the application, regarding the implementation of speech modalities and what the speech recognizer will understand.
   In this paper we present the context of our work, describe the major challenges involved in using speech modalities, summarize our approach to speech interaction design and share experiences regarding our applications, their architecture and gathered insights.
   In our approach we use a multimodal framework, responsible for the communication between modalities, and a generic speech modality allowing developers to quickly implement new speech enabled applications.
   As part of our methodology, in order to inform development, we consider two different applications, one targeting smartphones and the other tablets or home computers. These adopt a multimodal architecture and provide different scenarios for testing the proposed speech modality.
Keywords: Speech; multimodal architecture; decoupled modalities
Introducing Consciousnet: Internet Content as an Environment for Human-Machine Interaction BIBAFull-Text 382-393
  Vincenzo Catania; Davide Patti; Mariagrazia Sciacca
In this work we introduce Consciousnet, an open source architecture aimed to provide a general purpose environment for experimenting with human-machine language interaction. The main idea is exploiting the distributed and unsupervised complexity of the Internet in order to get all the semantic/syntactic material needed to carry on a linguistic text based interaction. After describing the main elements of the architecture, the results of a set of Turing-inspired tests are shown to demonstrate how the unpredictability and generality of the environment can be used as a basis for designing tests and experiments involving both psychologists and AI scientists.
Can User-Paced, Menu-free Spoken Language Interfaces Improve Dual Task Handling While Driving? BIBAKFull-Text 394-405
  Alexander Eriksson; Anders Lindström; Albert Seward; Alexander Seward; Katja Kircher
The use of speech-based interaction over traditional means of interaction in secondary tasks may increase safety in demanding environments with high requirements on operator attention. Speech interfaces have suffered from issues similar to those of visual displays, as they often rely on a complex menu structure that corresponds to that of visual systems. Recent advances in speech technology allow the use of natural language, eliminating the need for menu structures and offering a tighter coupling between the intention to act and the completion of the action. Modern speech technology may not only make already existing types of interaction safer, but also opens up for new applications, which may enhance safety. One such application is a speech-based hazard reporting system. A small fixed-base simulator study showed that drivers adapt the timing of the hazard reports to the situation at hand, such that an increase in reported workload was avoided.
Keywords: speech-based interface; natural language; compensatory behaviour; hazard reporting; human factors; VUI; strategic driving behaviour; simulated driving; IVIS
Chinese Romanization and Its Application in HCI BIBAKFull-Text 406-416
  Zhiwei Feng
Chinese Romanization can transcribe Chinese characters to Romanized Pinyin, It is very useful for natural language processing, documentation, language learning. It became an important tool for human-computer interaction.
Keywords: Chinese Romanization; Pinyin; documentation; Chinese characters; human-computer interaction
Driving with a Speech Interaction System: Effect of Personality on Performance and Attitude of Driver BIBAKFull-Text 417-428
  Ing-Marie Jonsson; Nils Dahlbäck
Personality has a huge effect on how we communicate and interact with others. This study is one in a series of three that investigates how a speech based in-car system matched with dominant and submissive drivers affects performance and attitude drivers. The study was conducted with 30 participants at Linköping University in Sweden. Data show that using a voice that combines feature from submissive and dominant speech patterns work well for both dominant and submissive drivers. The voice showed the same performance gain as when matching car voice personality with personality of driver, without the negative attitude ratings associated with the submissive car voice found in previous studies. Drivers assessment of the car system show that even though both dominant and submissive drivers find the system helpful, dominant drivers find the system more annoying and more likely to turn the system off. Design implications of in-vehicle systems are discussed.
Keywords: In-car System; Driving Simulator; Driving Performance; Speech system; Attitude; Personality; Dominant and Submissive
Effects of Language Variety on Personality Perception in Embodied Conversational Agents BIBAKFull-Text 429-439
  Brigitte Krenn; Birgit Endrass; Felix Kistler; Elisabeth André
In this paper, we investigate the effects of language variety in combination with bodily behaviour on the perceived personality of a virtual agent. In particular, we explore changes on the extroversion-introversion dimension of personality. An online perception study was conducted featuring a virtual character with different levels of expressive body behaviour and different synthetic voices representing German and Austrian language varieties. Clear evidence was found that synthesized language variety, and gestural expressivity influence the human perception of an agent's extroversion. Whereby Viennese and Austrian standard language are perceived as more extrovert than it is the case for the German standard.
Keywords: virtual agents; personality; extroversion-introversion; language variety and non-verbal behaviour
Long Text Reading in a Car BIBAKFull-Text 440-449
  Ladislav Kunc; Martin Labsky; Tomas Macek; Jan Vystrcil; Jan Kleindienst; Tereza Kasparova; David Luksch; Zeljko Medenica
We present here the results of a study focused on text reading in a car. The purpose of this work is to explore how machine synthesized reading is perceived by users. Are the users willing to tolerate deficiencies of machine synthesized speech and trade it off for more current content? What is the impact of listening to it on driver's distraction? How do the answers to the questions above differ for various types of text content? Those are the questions we try to answer in the presented study. We conducted the study with 12 participants, each facing three types of tasks. The tasks differed in the length and structure of the presented text. Reading out a fable represented an unstructured pleasure reading text. The news represented more structured short texts. Browsing a car manual was an example of working with structured text where the user looks for particular information without much focusing on surrounding content. The results indicate relatively good user acceptance for the presented tasks. Distraction of the driver was related to the amount of interaction with the system. Users opted for controlling the system by buttons on the steering wheel and made little use of the system's display.
Keywords: Architectures for interaction; CUI; SUI ad GUI; HCI methods and theories; Interaction design; Speech and natural language interfaces; Long text reading; car; UI; LCT
Let's Get Personal BIBAKFull-Text 450-461
  Nikita Mattar; Ipke Wachsmuth
Agents that are able to build relationships with the people they are interacting with are envisioned to be more successful in long-term interactions. Small talk about impersonal topics has been found an adequate tool in human-agent interactions for manipulation of such relationships. We suspect that an agent and the interaction with it will be evaluated even more positively when the agent talks about personal information it remembers about its interlocutor from previous encounters. In this paper a model of person memory that provides virtual agents with information needed in social conversations is presented. An interaction study demonstrates the impact of personal information in human-agent conversations and validates the performance of our model.
Keywords: conversational agents; intelligent virtual agents; human-agent interaction; person memory; social conversations; interaction study
Multimodal Behaviours in Comparable Danish and Polish Human-Human Triadic Spontaneous Interactions BIBAKFull-Text 462-471
  Costanza Navarretta; Magdalena Lis
This is a pilot study of multimodal behaviours in manually annotated comparable video recordings of Danish and Polish triadic naturally occurring conversations. The data are comparable with respect to the conversational settings, the familiarity degree, age and gender of the participants. Furthermore, they have been annotated according to the same annotation scheme following common coding strategies. The analysis of the annotations indicates that although the conversations in the two languages differ in content, Danes and Poles use the same type of head movements and with the same frequency. In both datasets the most common facial expressions are laughter and smile, however, facial expressions are much more frequent in the Polish data than in the Danish data. Furthermore, the facial expressions in the Polish data are often used as feedback signals to the interlocutors while the Danes use facial expression to comment their own spoken contribution. Finally, the Danes use more frequently hand gestures than the Poles and their hand gestures have a deictic function while the hand gestures of the Poles are iconic. The differences in the behaviours in the two corpora can partly depend on the language, but is also due to the type of relationship between the participants and the content of the conversations.
Keywords: Multimodal Corpora; Multilinguality; Human-human Interaction
Building Rapport between Human and ECA: A Pilot Study BIBAKFull-Text 472-480
  David Novick; Iván Gris
This study is part of a longer-term project to provide embodied conversational agents (ECAs) with behaviors that enable them to build and maintain rapport with their human partners. We focus on paralinguistic behaviors, and especially nonverbal behaviors, and their role in communicating rapport. Using an ECA that guides its players through a speech-controlled game, we attempt to measure the familiarity built between humans and ECAs across several interactions based on paralinguistic behaviors. In particular, we studied the effect of differences in the amplitude of nonverbal behaviors by an ECA interacting with a human across two conversational sessions. Our results suggest that increasing amplitude of nonverbal paralinguistic behaviors may lead to an increased perception of physical connectedness between humans and ECAs.
Keywords: Embodied conversational agent; familiarity; rapport; paralinguistic; nonverbal communication
The Effect of Voice Instruction on the Construction of Mental Model BIBAKFull-Text 481-491
  A Restyandito; Alan H. S. Chan; Umi Proboyekti
The goal of this study is to observe the effect of instruction deliverance method in the construction of mental model. A good mental model can help the user's learnability process. There were two methods tested in this study: a step-by-step instruction (SS) and a complete set of whole-steps instructions (WS) to finish a given task. The SS group performed better on the learning process, however they had the least score on both the information retention and transfer process. Their minds were not engaged in the process, as they seemed to simply follow the instructions without being critical. When error occurred, they tended to be less persistent in trying to finish the task. This might be caused by the incomplete mental model as a result of receiving the instruction step by step.
Keywords: voice instruction; mental model; learnability
Discourse Particles and User Characteristics in Naturalistic Human-Computer Interaction BIBAKFull-Text 492-501
  Ingo Siegert; Matthias Haase; Dmytro Prylipko; Andreas Wendemuth
In human-human interaction (HHI) the behaviour of the speaker is amongst others characterised by semantic and prosodic cues. These short feedback signals minimally communicate certain dialogue functions such as attention, understanding or other attitudinal reactions. Human-computer interaction (HCI) systems have failed to note and respond to these details so far, resulting in users trying to cope with and adapt to the machines behaviour. In order to enhance HCI, an adaptation to the user's behaviour, individual skills, and the integration of a general human behaviour understanding is indispensable. Another issue is the question if the usage of feedback signals is influenced by the user's individuality. In this paper, we investigate the influence of specific feedback signals, known as discourse particles (DPs), with communication style and psychological characteristics within a naturalistic HCI. This investigation showed that there is a significant difference in the usage of DPs for users of certain user characteristics.
Keywords: human-machine-interaction; discourse particles; personality; user characteristics
The Effects of Working Memory Load and Mental Imagery on Metaphoric Meaning Access in Metaphor Comprehension BIBAKFull-Text 502-510
  Xiaofang Sun; Ye Liu; Xiaolan Fu
Metaphor is a cognitive process that enables people to make mental mapping across distinct conceptual domains. The present study investigated metaphorical and literal meaning access in metaphor comprehension, and the effects of working memory load and mental imagery on metaphor comprehension. Three sentence priming experiments were conducted and the results showed that the literal meaning of a metaphor was accessed faster than the metaphorical meaning, but metaphorical meaning could be accessed as quickly as literal meaning if there was more cognitive resource involved. These findings indicated that the literal meaning of a metaphor is accessed first in the early stage of metaphor comprehension, and working memory load plays an important role in the process. The study didn't find any significant effect of imageability on metaphor comprehension; however, the results implied the metaphors with low imageability need more working memory load to understand. The implication for natural language processing of the computer science was discussed.
Keywords: metaphor comprehension; working memory load; mental imagery; imageability

Natural and Multimodal Interfaces

Human Factors in the Design of BCI-Controlled Wheelchairs BIBAKFull-Text 513-522
  Wafa Alrajhi; Manar Hosny; Areej Al-Wabil; Arwa Alabdulkarim
In this paper, we synthesize research on the type of cognitive commands that have been examined for controlling Brain Computer Interface (BCI) wheelchairs and the human factors that have been reported for the selection of different protocols of BCI commands for an individual user. Moreover, we investigate how different researchers have considered the necessity of sustained movement from a single thought/command, having an emergency stop, and the commands necessary for assisting users with a particular disability. We then highlight how these human factors and ergonomics' considerations were applied in the design and development of an EEG-controlled motorized wheelchair, aiming to emphasize users' requirements for people with severe physical disabilities. In this case study, we propose a brain controlled wheelchair navigation system that can help the user travel to a desired destination, without having to personally drive the wheelchair and frequently change the movement directions along the path to the destination. The user can choose the desired destination from a map of the environment, using his/her brain signals only. The user can navigate through the map using BCI cognitive commands. The system processes the brain signals, determines the required destination on the map, and constructs an optimized movement path from the source to the intended destination. To construct an obstacle-free path with the shortest possible distance and minimum number of turns, a path planning optimization problem is solved using a simple Simulated Annealing (SA) algorithm. The resulting optimized path will be translated into movement directions that are sent to the microcontroller to move the wheelchair to the desired destination.
Keywords: Brain Computer Interaction (BCI); electroencephalography (EEG); Path Planning Optimization; Simulated Annealing; Wheelchair
Interface Design and Dynamic Audio BIBAKFull-Text 523-531
  Luiz Roberto Carvalho; Alice T. Cybis Pereira
In the age of digital devices, text, image, sound, interactivity, blend themselves into a symbiotic and unique media, presenting a multifaceted specie of language called hypermedia. However, since many years ago, we have seen a notable emphasis on visual communication's interfaces, and due to its limitations, products and services in design can often present inconsistencies when other sensory properties are relevant, as in the case of sound information. This over-emphasis on visual displays has constrained the development of interactive systems that are capable of making better use of the auditory modality. Recognizing the HCI as an integrating element of media and visual, sound and tactile metaphors, this study will demonstrate investigations that contextualize the role of sound into interactive environments by proposing an overview for the term interactive sound, suggesting its classification into direct-interactive and indirect-adaptative sounds, and pointing out its meanings and applications.
Keywords: sound design; game sound; dynamic audio; interactive sound
A Pictorial Interaction Language for Children to Communicate with Cultural Virtual Characters BIBAKFull-Text 532-543
  Birgit Endrass; Lynne Hall; Colette Hume; Sarah Tazzyman; Elisabeth André
In this paper, we outline the creation of an engaging and intuitive pictorial language as an interaction modality to be used by school children aged 9 to 11 years to interact with virtual characters in a cultural learning environment. Interaction takes place on a touch screen tablet computer linked to a desktop computer on which the characters are displayed. To investigate the benefit of such an interaction style, we conducted an evaluation study to compare the pictorial interaction language with a menu-driven version for the same system. Results indicate that children found the pictorial interaction language more fun and more exciting than the menus, with users expressing a desire to interact for longer using the pictorial interaction language. Thus, we think the pictorial interaction language can help support the children's experiential learning, allowing them to concentrate on the content of the cultural learning scenario.
Keywords: Interaction Design; Interaction Modality; Virtual Agents; Culture; User Experience
Tangible or Not Tangible -- A Comparative Study of Interaction Types for Process Modeling Support BIBAKFull-Text 544-555
  Albert Fleischmann; Werner Schmidt; Christian Stary
Many organizations loose potential for optimizing their operation due to limited stakeholder participation when designing business processes. One of the reasons is that traditional modeling methods and (interactive) tools are not suitable for domain experts who neither want to struggle with complex or formal notations, nor with the respective modeling tool. Tangible modeling interfaces are a significant move towards stakeholder inclusion. We review their respective capabilities not only with regard to modeling, but also to implementation and execution of business processes, setting the stage for improving the effectiveness of interactive Business Process Management support, and thus, stakeholder participation in organizational development.
Keywords: Tangible user interface; process modeling; model documentation; model execution; Subject-oriented BPM; multi-modal interaction
Body Image and Body Schema: Interaction Design for and through Embodied Cognition BIBAKFull-Text 556-566
  Ozgun Eylul Iscen; Diane Gromala; Maryam Mobini
The interdisciplinary literature on body image/body schema (BIBS), which is within the larger realm of embodied cognition, can provide HCI practitioners and theorists new ideas of and approaches to human perception and experience. In very brief terms, body image consists of perceptions, attitudes and beliefs pertaining to one's own body, whereas body schema is a system of sensory-motor capabilities that function, usually without awareness or the necessity of perceptual monitoring. The dynamic relationality and plasticity of BIBS open up different avenues for interaction design. An overview of six main ideas deriving from BIBS literature are enumerated, followed by a discussion of projects designed for chronic pain patients that demonstrate how these ideas can be adopted in interaction design processes as a perspective or attitude rather than a mere application of traditional methods. Through bridging HCI and BIBS theories and research, we can develop a holistic framework in which we design for and through embodied cognition.
Keywords: Embodied cognition; body image; body schema; interaction design; virtual reality; chronic pain
Exploring Initiative Interactions on a Proxemic and Ambient Public Screen BIBAKFull-Text 567-577
  Huiliang Jin; Bertrand David; René Chalon
Public screens are common in modern society, and provide information services to audiences. However, as more and more screens are installed, it becomes a burden for users to find information concerning themselves quickly. This is because screens cannot understand what users really need, they only display pre-designed information related to a certain location. To ensure better cohabitation between people and screens, one solution is to make screens understand users rather than make users understand screens. Given that it is difficult, even for humans, to interpret other people's intentions, it is far harder for screens to understand users. We need first to decide which kinds of information about users could be helpful for a screen to estimate to users' needs. In this paper, we study a public interactive screen, which can speculate as to users' intentions by interpreting their proxemic attributes (such as distance, movement, etc.) and context information (identity, locations, etc.). Based on proxemic interaction semantics, we built an interactive public screen, which: 1) could interpret users' needs in advance and display relevant information; 2) be available for multi-users and display distinct information to them; 3) be open for data exchanges with users' mobile devices. Through a lab study, we demonstrate that the screen presented in this paper is more attractive to users and could provide users with useful information more rapidly and precisely than traditional screens.
Keywords: Proxemic Interaction; Proxemic Screen; Public Screen; Initiative Interactions
Evaluation of Tactile Drift Displays in Helicopter BIBAKFull-Text 578-588
  Patrik Lif; Per-Anders Oskarsson; Johan Hedström; Peter Andersson; Björn Lindahl; Christopher Palm
Brownout during helicopter landing and takeoff is a serious problem and has caused numerous accidents. Development of displays indicating drift is one part of the solution, and since the visual modality is already saturated one possibility is to use a tactile display. The main purpose in this study was to investigate how tactile displays should be coded to maintain or increase the ability to control lateral drift. Two different tactile drift display configurations were compared, each with three different onset rates to indicate the speed of lateral drift. A visual drift display was used as control condition. The results show that best performance is obtained with the basic display with slow onset, and with complex display with constant onset rate. The results also showed that performance with the best tactile drift display configurations was equal to the already validated visual display.
Keywords: Tactile display; helicopter; brownout
Development of Interaction Concepts for Touchless Human-Computer Interaction with Geographic Information Systems BIBAFull-Text 589-599
  Ronald Meyer; Jennifer Bützler; Jeronimo Dzaack; Christopher M. Schlick
Interaction concepts in 3D GIS are yet limited to 2D input methods like mouse and keyboard. This work describes elaboration of a concept of touchless interaction for a prototype that aims to be used in maritime GIS applications. Experts from the maritime field have been interviewed to construct a rigorous scenario settled in the maritime field. Besides the planning and conversion of a stereoscopic GIS prototype a touchless interaction concept for stereoscopic environments under consideration of three different hand models is developed and presented. Implementation of these different hand models is planned for future evaluation.
Spyractable: A Tangible User Interface Modular Synthesizer BIBAKFull-Text 600-611
  Spyridon Potidis; Thomas Spyrou
The purpose of this paper is the exploration of the possibilities that Tangible User Interface (TUI) may offer in the area of sound synthesis, by reconfiguring the functionality of the existing TUI tabletop musical instrument called "Reactable" and redesigning most features, adjusting it to a synthesizers needs. For this research we analyzed sound properties, physics and formation, as well as how human used these features to synthesize sound. Afterwards we present the properties and advantages of TUI technology, and its use in sound and music, distinguishing Reactable, for being the most even musical instrument using TUI. As an outcome we develop and present an initial prototype modular synthesizer, called Spyractable. Finally, we subject Spyractable to users' evaluation tests and we present the outcomes, making suggestions for further investigation and design guidelines.
Keywords: Sound wave; harmonics; modular synthesis; modules; tangible user interface (TUI); tangibles; patches; graphical controllers
Neural Interface Emotiv EPOC and Arduino: Brain-Computer Interaction in a Proof of Concept BIBAKFull-Text 612-623
  Eduardo Emilio Reder; Amilton Rodrigo de Quadros Martins; Vinícius Renato Thomé Ferreira; Fahad Kalil
This study aims to demonstrate the interaction between the human being and the machine through a neural pattern recognizing interface, namely Emotiv EPOC, and a robotic device made by Arduino. The union of these technologies is assessed in specific tests, seeking a usable and stable binding with the smallest possible rate of error, based on a study of how the human electrical synapses are produced and captured by the electroencephalogram device, through examples of projects that achieved success using these technologies. In this study, the whole configuration of the software used to bind these technologies, as well as how they work, is explained, and the result of the experiments through an analysis of the tests performed is addressed. The difference in the results between genders and the influence of user feedback, as well as the accuracy of the technologies, are explained during the analysis of the data captured.
Keywords: Emotiv EPOC; Arduino; Brain-Computer Interface; Interaction; Electroencephalogram
A Heuristic Model of Vibrotactile Haptic Feedbacks Elicitation Based on Empirical Review BIBAKFull-Text 624-632
  Anak Agung Gede Dharma; Kiyoshi Tomimatsu
We propose a novel heuristic model of vibrotactile feedbacks elicitation. The model is based on two known tactile elicitation principles, i.e. perceived tactile sensation and apparent haptic motion. Our previous studies, along with empirical reviews were used to provide an insight of how these two principles work individually. Our preceding works on the mapping of texture phase diagram of artificial vibrotactile stimuli reveals 3 main perceived vibrotactile sensation, i.e. dampness, friction, and hardness. Furthermore, we have conducted a preliminary research to observe apparent haptic motion in our proposed haptic vest interface. Our findings and the empirical reviews imply that these two haptic principles can be used concurrently to create a novel user experience.
Keywords: vibrotactile haptic feedback; heuristic model; tactile perception
Auditory Emoticons: Iterative Design and Acoustic Characteristics of Emotional Auditory Icons and Earcons BIBAKFull-Text 633-640
  Jason Sterkenburg; Myounghoon Jeon; Christopher Plummer
In recent decades there has been an increased interest in sonification research. Two commonly used sonification techniques, auditory icons and earcons, have been the subject of a lot of study. However, despite this there has been relatively little research investigating the relationship between these sonification techniques and emotions and affect. Additionally, despite their popularity, auditory icons and earcons are often treated separately and are rarely compared directly in studies. The current paper shows iterative design procedures to create emotional auditory icons and earcons. The ultimate goal of the study is to compare auditory icons and earcons in their ability to represent emotional states. The results show that there are some strong user preferences both within sonification categories and between sonification categories. The implications and extensions of this work are discussed.
Keywords: auditory icons; earcons; auditory emoticons; non-speech sounds; sonification
Natural Forms of Communication and Adaptive Behaviour in Human-Computer-Interaction BIBAKFull-Text 641-647
  Madlen Wuttke; Kai-Uwe Martin
Scientific research over the last two decades imputes a beneficial effect on human-computer interaction by depicting a virtual communication partner onscreen due to the persona effect and the media equation theory. On the other hand, looking back at the historic component of human-computer interactions, the burden of adaptation has always been on humans to understand the machine and to communicate in accordance with its standards. This paper describes natural communication and interaction strategies of humans and computers as well as their importance to scientific research.
Keywords: Intelligent and agent systems; Pedagogical Agents; Natural Forms of Communication; Adaptive; Mobile Learning

Human-Robot Interaction

Backchannel Head Nods in Danish First Meeting Encounters with a Humanoid Robot: The Role of Physical Embodiment BIBAKFull-Text 651-662
  Anders Krogsager; Nicolaj Segato; Matthias Rehm
Head nods have been shown to play an important role for communication management in human communication, e.g. as a non-verbal feedback signal from the listener. Based on a study with virtual agents, which showed that the use of head nods helps eliciting more verbal input from the user, we investigate the use of head nods in communications between a user and a humanoid robot (Nao) that they meet for the first time. Contrary to the virtual agent case, the robot elicited less talking from the user when it was using head nods as a feedback signal. A follow-up experiment revealed that the physical embodiment of the robot had a huge impact on the users' behavior in the first encounters.
Keywords: Culture-aware robots; backchannels; feedback; physical embodiment
Recommended Considerations for Human-Robot Interaction Communication Requirements BIBAKFull-Text 663-674
  Stephanie J. Lackey; Daniel J. Barber; Sushunova G. Martinez
Emerging robot systems increasingly exhibit greater levels of autonomy, requiring improvements in interaction capabilities to enable robust human-robot communication. This paper summarizes the present level of supervisory control in robots, both fielded and experimental, and the type of communication interfaces needed for successful Human-Robot Interaction (HRI). The focus of this research is to facilitate direct interactions between humans and robot systems within dismounted military operations and similar applications (e.g., law enforcement, homeland security, etc.). Achieving this goal requires advancing audio, visual, and tactile communication capabilities beyond the state-of-the-art. Thus, the requirement for a communication standard supporting supervisory control of robot teammates is recommended.
Keywords: Supervisory control; autonomy; human-robot interaction
An Emotional Framework for a Real-Life Worker Simulation BIBAKFull-Text 675-686
  Nicholas H. Müller; Martina Truschzinski
Within the framework of the project 'The Smart Virtual Worker' we put forward a sound and functioning emotional model which adequately simulates a worker's emotional feelings throughout a typical task in an industrial setting. We restricted the model to represent the basic emotions by Ekman and focused on the implementation of 'joy' and 'anger'. Since emotions are uniquely generated, based on the interpretation of a stimulus by an individual, we linked the genesis of emotions to empirical findings of the sports sciences to infer an emotional reaction. This paper describes the concept of the model from a theoretical and practical point of view as well as the preliminary state of implementation and upcoming steps of the project.
Keywords: emotion framework; work simulation; workflow simulator; emotional valence; emotional model
Behavioral Persona for Human-Robot Interaction: A Study Based on Pet Robot BIBAKFull-Text 687-696
  Thiago Freitas dos Santos; Danilo Gouveia de Castro; Andrey Araujo Masiero; Plinio Thomaz Aquino Junior
With the advancement of technology robots have become more common in every day applications, like Paro and GOSTAI Jazz for health care or Pleo and Genibo for entertainment. Since these robots are designed to constantly interact with people, during the development process it should be considered how people would feel and behave when they interact with those artifacts. However there might be some issues in collecting this type of data or how to efficiently use it in the development of new features. In this study we report a process for creating Personas that will help in the design of subject-focused applications for robots interactions.
Keywords: User modeling and profiling; Human-Robot Interaction; Personas
Robotic Border Crosser TNG -- Creating an Interactive Mixed Reality BIBAKFull-Text 697-706
  Anke Tallig
In this paper is described an interactive mixed reality which is presented by a mobile robot. It explains the structure and functionality of the mixed reality and illustrated, how the combination works with the robot. In addition some evaluation results of the interactive screen are presented. Usage scenario for the interactive mixed reality is the Industriemuseum Chemnitz. This kind of exhibition is suitable for viewing the inner functions of an exhibit to see how this technology works. The view into a technical device can occurs with the help of a public screen which is projected on the exhibits surface. Via an interactive layer it's possible for users to interact with the indicated contents. This interactive projection system is mobile thereby the robot can transport the public screen through the museum and from exhibit to exhibit. This interactive screen contains videos, animations and pictures of the functionality of exhibits. With the assistance of this mobile system the visitor can learn more about the exhibits in general and their specific functionality.
Keywords: Human-Computer Interaction; Human-Robot Interaction; Mixed Reality; Robotic Mediator; Interdisciplinary Collaboration; Blended Museum
Emotion Transmission System Using a Cellular Phone-Type Teleoperated Robot with a Mobile Projector BIBAFull-Text 707-714
  Yu Tsuruda; Maiya Hori; Hiroki Yoshimura; Yoshio Iwai
We propose an emotion transmission system using a cellular phone-type teleoperated robot with a mobile projector. Elfoid has a soft exterior that provides the look and feel of human skin and is designed to transmit a speaker's presence to their communication partner using a camera and microphone. To transmit the speaker's presence, Elfoid transmits not only the voice of the speaker but also their facial expression as captured by the camera. In this research, facial expressions are recognized by a machine learning technique. Elfoid cannot, however, physically display facial expressions because of its compactness and a lack of sufficiently small actuator motors. The recognized facial expressions are displayed using a mobile projector installed in Elfoid's head to convey emotions. We build a prototype system and experimentally evaluate its subjective usability.

Emotions Recognition

Design of an Emotion Elicitation Framework for Arabic Speakers BIBAKFull-Text 717-728
  Sharifa Alghowinem; Sarah Alghuwinem; Majdah Alshehri; Areej Al-Wabil; Roland Goecke; Michael Wagner
The automatic detection of human affective states has been of great interest lately for its applications not only in the field of Human-Computer Interaction, but also for its applications in physiological, neurobiological and sociological studies. Several standardized techniques to elicit emotions have been used, with emotion eliciting movie clips being the most popular. To date, there are only four studies that have been carried out to validate emotional movie clips using three different languages (English, French, Spanish) and cultures (French, Italian, British / American). The context of language and culture is an underexplored area in affective computing. Considering cultural and language differences between Western and Arab countries, it is possible that some of the validated clips, even when dubbed, will not achieve similar results. Given the unique and conservative cultures of the Arab countries, a standardized and validated framework for affect studies is needed in order to be comparable with current studies of different cultures and languages. In this paper, we describe a framework and its prerequisites for eliciting emotions that could be used for affect studies on an Arab population. We present some aspects of Arab culture values that might affect the selection and acceptance of emotion eliciting video clips. Methods for rating and validating Arab emotional clips are presented to derive at a list of clips that could be used in the proposed emotion elicitation framework. A pilot study was conducted to evaluate a basic version of our framework, which showed great potential to succeed in eliciting emotions.
Keywords: Emotion elicitation framework; Arabic emotion data collection; emotional movie clips
Analysing Emotional Video Using Consumer EEG Hardware BIBAKFull-Text 729-738
  Jeroen de Man
Low-cost, easy to use EEG hardware produced for the consumer-market provide interesting possibilities for human-computer interaction in a wide variety of applications. Recent years have produced numerous papers discussing the use of these types of devices in various ways, but only some of this work looks into what these devices can actually measure. In this paper, data is used that has been collected using a Myndplay Brainband, while 30 participants viewed emotional videos eliciting different mental states. This data is analysed by looking at average power in multiple frequency bands and eSense™ values, as well as peaks in the measurements detected throughout the videos. Although average values do not differentiate well between the mental states, peak detection provides some promising results worthy of future research.
Keywords: emotional response; emotion analysis; affective HCI; EEG
Emotracking Digital Art BIBAKFull-Text 739-749
  Isabelle Hupont; Eva Cerezo; Sandra Baldassarri; Rafael Del-Hoyo
Art and emotions are intimately related. This work proposes the application to arts of Emotracker, a novel tool that mixes eye tracking technology and facial emotions detection to track user behaviour. This combination offers intuitive and highly visual possibilities of relating eye gaze, emotions and artistic contents. The results obtained after carrying out "5-second emotracking tests" over art illustrations and the use of the gathered information to create real-time artistic effects are presented.
Keywords: affect analysis; gaze; face analysis; digital arts
Estimation of Emotion by Electroencephalography for Music Therapy BIBAKFull-Text 750-757
  Kensyo Kinugasa; Hiroki Yoshimura; Maiya Hori; Satoru Kishida; Yoshio Iwai
A system for providing music employing electroencephalography for music therapy is described. Music therapy for the treatment of patients suffering mental illness has been attempted over a period of 20 years. To reduce stress, it is preferable to listen to music that matches a person's emotions. However, it is difficult to know exactly the person's emotion. It is necessary to calibrate the proposed system employing electroencephalography to emotions. We discuss a method of calibration especially used in canonical correlation analysis. Experimental results show that it is possible to roughly estimate feelings. We consider that it is possible to use our system in practice.
Keywords: electroencephalography; music therapy; canonical correlation analysis
Evaluating User's Emotional Experience in HCI: The PhysiOBS Approach BIBAKFull-Text 758-767
  Alexandros Liapis; Nikos Karousos; Christos Katsanos; Michalis Xenos
As computing is changing parameters, apart from effectiveness and efficiency in human-computer interaction, such as emotion have become more relevant than before. In this paper, a new tool-based evaluation approach of user's emotional experience during human-computer interaction is presented. The proposed approach combines user's physiological signals, observation data and self-reported data in an innovative tool (PhysiOBS) that allows continuous and multiple emotional states analysis. To the best of our knowledge, such an approach that effectively combines all these user-generated data in the context of user's emotional experience evaluation does not exist. Results from a preliminary evaluation study of the tool were rather encouraging revealing that the proposed approach can provide valuable insights to user experience practitioners.
Keywords: User Emotional Experience; Human Computer Interaction; Evaluation; Physiological Signals; Emotions
Proposal for the Model of Occurrence of Negative Response toward Humanlike Agent Based on Brain Function by Qualitative Reasoning BIBAKFull-Text 768-778
  Yoshimasa Tawatsuji; Keiichi Muramatsu; Tatsunori Matsui
For designing the rounded communication between human and agent, humanlike appearance of agent can contribute to human understandability towards their intension. However, the excessive humanlike-ness can cause human to feel repulsive against the agent, which is well known as the uncanny valley. In this study, we propose a model providing an explanation for how the human negative response is formed, based on the brain regions and its function, including the amygdala, hippocampus, cortex and striatum. This model is described with quantitative reasoning and simulated. The results indicate that as human observes a humanlike agent, the emotion goes negative and the brain regions were more activated in comparison with the case human observes a person.
Keywords: Human Agent Interaction; uncanny valley; brain function; qualitative reasoning
Current and New Research Perspectives on Dynamic Facial Emotion Detection in Emotional Interface BIBAKFull-Text 779-787
  Tessa-Karina Tews; Michael Oehl; Helmut Faasch; Taro Kanno
In recent years there has been an increasing interdisciplinary exchange between psychology and computer science in the field of recognizing emotions for future-oriented Human-Computer and Human-Machine Interfaces. Although affective computing research has made enormous progress in automatically recognizing facial expressions, it has not yet been fully clarified how algorithms can learn to encode or decode a human face in a real environment. Consequently, our research focuses on the detection of emotions or affective states in a Human-Machine setting. In contrast to other approaches, we use a psychology driven approach trying to minimize complex computations by using a simple dot-based feature extraction method. We suggest a new approach within, but not limited to, a Human-Machine Interface context which detects emotions by analyzing the dynamic change in facial expressions. In order to compare our approach, we discuss our software with respect to other developed facial expression studies in context of its application in a chat environment. Our approach indicates promising results that the program could accurately detect emotions. Implications for further research as well as for applied issues in many areas of Human-Computer Interaction, particularly for affective and social computing, will be discussed and outlined.
Keywords: Emotional Interfaces; Affective Computing; Facial Expression; Human Machine Interface
Evaluation of Graceful Movement in Virtual Fitting through Expressed Emotional Response and Emotion Expressed via Physiology Measures BIBAKFull-Text 788-797
  Wan Adilah Wan Adnan; Nor Laila Md. Noor; Fauzi Mohd Saman; Siti Nurnabillah Zailani; Wan Norizan Wan Hashim
Graceful interaction is a form of interaction that incorporates quality movement that can invoke the emotional appeal of users engaged with it. However method of evaluation of the quality graceful interaction has not been discussed. As we argue that graceful interaction can evoke emotion, we explore the use of possible instruments to evaluate graceful interaction based on the valence-arousal model. To measure emotional response of arousal the response is using verbal and non-verbal instruments. The former is based on self-report emotions and the later through autonomic measures of emotion via bio-physical measures of skin conductance. We conducted an experiment with six participants who were given the tasks to perform movement tasks in virtual fitting using three different virtual fitting room (VFR) applications available on e-commerce fashion retailing websites. The selection of the VFR applications was based on the presence of two identified graceful interaction design elements, which are tempo and sequence as prescribed by the graceful interaction design model. While performing the tasks, each participant's physiology measure of emotional response was recorded using the tool BioGraph Infiniti. Upon completion, the participants were requested to report their emotional response in an instrument constructed based on the valence arousal model. Finally each participant was also interviewed to state the VFR applications they preferred. The analysis of each type emotional response were made and the findings showed the congruence between the verbally expressed emotional response and physiology measure of emotional response in performing graceful interaction tasks. This suggests that the evaluation of graceful interaction can be made by the use of verbally and non-verbally expressed emotional respond instruments.
Keywords: aesthetic experience; graceful interaction; emotional design; virtual task; physiological measure; human computer interaction