| Gestures in Assisted Living Environments | | BIBAK | Full-Text | 1-12 | |
| Dimitra Anastasiou | |||
| This paper is concerned with multimodal assisted living environments,
particularly based on gesture interaction. The research of ambient assisted
living is about the provision of a safe, comfortable, and independent lifestyle
at a domestic environment. We refer to spatial gestures and gesture recognition
software and present an observational user study related to gestures in the
Bremen Ambient Assisted Living Lab (BAALL), a 60m² apartment suitable for
the elderly and people with physical or cognitive impairments. Keywords: assisted environment; localization; smart devices; spatial gestures | |||
| Using Wiimote for 2D and 3D Pointing Tasks: Gesture Performance Evaluation | | BIBA | Full-Text | 13-23 | |
| Georgios Kouroupetroglou; Alexandros Pino; Athanasios Balmpakakis; Dimitrios Chalastanis; Vasileios Golematis; Nikolaos Ioannou; Ioannis Koutsoumpas | |||
| We present two studies to comparatively evaluate the performance of
gesture-based 2D and 3D pointing tasks. In both of them, a Wiimote controller
and a standard mouse were used by six participants. For the 3D experiments we
introduce a novel configuration analogous to the ISO 9241-9 standard
methodology. We examine the pointing devices' conformance to Fitts' law and we
measure eight extra parameters that describe more accurately the cursor
movement trajectory. For the 2D tasks using Wiimote, Throughput is 41,2% lower
than using the mouse, target re-entry is almost the same, and missed clicks
count is three times higher. For the 3D tasks using Wiimote, Throughput is
56,1% lower than using the mouse, target re-entry is increased by almost 50%,
and missed clicks count is sixteen times higher.
Fitts' law, 3D pointing, Gesture User Interface, Wiimote | |||
| Choosing and Modeling the Hand Gesture Database for a Natural User Interface | | BIBAK | Full-Text | 24-35 | |
| Przemyslaw Glomb; Michal Romaszewski; Sebastian Opozda; Arkadiusz Sochan | |||
| This paper presents a database of natural hand gestures ('IITiS Gesture
Database') recorded with motion capture devices. For the purpose of
benchmarking and testing the gesture interaction system we have selected
twenty-two natural hand gestures and recorded them on three different motion
capture gloves with a number of participants and movement speeds. The
methodology for the gesture selection, details of the acquisition process, and
data analysis results are presented in the paper. Keywords: human-computer interaction; gesture interfaces; reference gesture database;
choosing gestures for HCI; gesture recognition; hand gesture vocabulary design;
motion capture gloves | |||
| User Experience of Gesture Based Interfaces: A Comparison with Traditional Interaction Methods on Pragmatic and Hedonic Qualities | | BIBAK | Full-Text | 36-47 | |
| Maurice H. P. H. van Beurden; Wijnand A. IJsselsteijn; Yvonne de Kort | |||
| Studies into gestural interfaces -- and interfaces in general -- typically
focus on pragmatic or usability aspects (e.g., ease of use, learnability). Yet
the merits of gesture-based interaction likely go beyond the purely pragmatic
and impact a broader class of experiences, involving also qualities such as
enjoyment, stimulation, and identification. The current study compared
gesture-based interaction with device-based interaction, in terms of both their
pragmatic and hedonic qualities. Two experiments were performed, one in a
near-field context (mouse vs. gestures), and one in a far-field context (Wii
vs. gestures). Results show that, whereas device-based interfaces generally
scored higher on perceived performance, and the mouse scored higher on
pragmatic quality, embodied interfaces (gesture-based interfaces, but also the
Wii) scored higher in terms of hedonic quality and fun. A broader perspective
on evaluating embodied interaction technologies can inform the design of such
technologies and allow designers to tailor them to the appropriate application. Keywords: Interaction technologies; gesture-based interaction; user experience;
hedonic quality; pragmatic quality; user interfaces; embodied interaction | |||
| Low Cost Force-Feedback Interaction with Haptic Digital Audio Effects | | BIBAK | Full-Text | 48-56 | |
| Alexandros Kontogeorgakopoulos; Georgios Kouroupetroglou | |||
| We present the results of an experimental study of Haptic Digital Audio
Effects with and without force feedback. Participants experienced through a low
cost Falcon haptic device two new real-time physical audio effect models we
have developed under the CORDIS-ANIMA formalism. The results indicate that the
haptic modality changed the user's experience significantly. Keywords: haptics; digital audio effects; physical modeling; CORDIS-ANIMA | |||
| The Role of Spontaneous Gestures in Spatial Problem Solving | | BIBAK | Full-Text | 57-68 | |
| Mingyuan Chu; Sotaro Kita | |||
| When solving spatial problems, people often spontaneously produce hand
gestures. Recent research has shown that our knowledge is shaped by the
interaction between our body and the environment. In this article, we review
and discuss evidence on: 1) how spontaneous gesture can reveal the development
of problem solving strategies when people solve spatial problems; 2) whether
producing gestures can enhance spatial problem solving performance. We argue
that when solving novel spatial problems, adults go through deagentivization
and internalization processes, which are analogous to young children's
cognitive development processes. Furthermore, gesture enhances spatial problem
solving performance. The beneficial effect of gesturing can be extended to
non-gesturing trials and can be generalized to a different spatial task that
shares similar spatial transformation processes. Keywords: gesture; spatial problem solving; mental rotation; cognitive development | |||
| Effects of Spectral Features of Sound on Gesture Type and Timing | | BIBAK | Full-Text | 69-80 | |
| Mariusz Kozak; Kristian Nymoen; Rolf Inge Godøy | |||
| In this paper we present results from an experiment in which infrared motion
capture technology was used to record participants' movement in synchrony to
different rhythms and different sounds. The purpose was to determine the
effects of the sounds' spectral and temporal features on synchronization and
gesture characteristics. In particular, we focused on the correlation between
sounds and three gesture features: maximum acceleration, discontinuity, and
total quantity of motion. Our findings indicate that discrete, discontinuous
motion resulted in better synchronization, while spectral features of sound had
a significant effect on the quantity of motion. Keywords: Gesture; Music; Synchronization; Sound envelope | |||
| Human-Motion Saliency in Complex Scenes | | BIBAK | Full-Text | 81-92 | |
| Fiora Pirri; Matia Pizzoli; Matei Mancas | |||
| We present a new and original method for human motion analysis and
evaluation, developed on the basis of the role played by attention in the
perception of human motion. Attention is particularly relevant both in a
multi-motion scene and in social interactions, when it comes to select and
discern why and what to focus on. The first crucial role of attention concerns
the saliency of human motion within a scene where other dynamics might occur.
The second role, in social-close interactions, is highlighted by the
selectivity shown towards gesture modalities both in peripheral and central
vision. Experiments for both modeling and testing have been based on a dynamic
3D gaze tracker. Keywords: human motion; selective attention; saliency; gestures segmentation | |||
| What, Why, Where and How Do Children Think? Towards a Dynamic Model of Spatial Cognition as Action | | BIBAK | Full-Text | 93-105 | |
| Marilyn Panayi | |||
| The Spatial Cognition in Action (SCA) model described here takes a dynamic
systems approach to the biological concept of the Perception-Cognition-Action
cycle. This partial descriptive feature based model is theoretically developed
and empirically informed by the examination of children's embodied gestures
that are rooted in action. The model brings together ecological and corporeal
paradigms with evidence from neurobiological and cognitive science research.
Such a corporeal approach places the 'action ready body' centre stage. A corpus
of gesture repertoires from both neuro-atypical and neuro-typical children has
been created from ludic interaction studies. The model is proposed as a dynamic
construct for intervention, involving the planning and design of interaction
technology for neuro-atypical children's pedagogy and rehabilitation. Keywords: spatial cognition; model; embodied child gesture; action-readybody;
neurobiological; atypical; future technology; intervention; pedagogy;
rehabilitation; dynamic systems; emergent; perception-action-cycle | |||
| A Labanotation Based Ontology for Representing Dance Movement | | BIBAK | Full-Text | 106-117 | |
| Katerina El Raheb; Yannis E. Ioannidis | |||
| In this paper, we present a Knowledge Based System for describing and
storing dances that takes advantage of the expressivity of Description Logics.
We propose exploiting the tools of the Semantic Web Technologies in
representing and archiving dance choreographies by developing a Dance Ontology
in OWL-2. Description Logics allow us to express complex relations and
inference rules for the domain of dance movement, while Reasoning capabilities
make it easy to extract new knowledge from existing knowledge. Furthermore, we
can search within the ontology based on the steps and movements of dances by
writing SPARQL queries. The constructing elements of the ontology and their
relationships to construct the dance model are based on the semantics of the
Labanotation system, a widely applied language that uses symbols to denote
dance choreographies. Keywords: Semantic Web Technologies; Ontology; Description Logics; Dance Notation;
Labanotation | |||
| ISOcat Data Categories for Signed Language Resources | | BIBAK | Full-Text | 118-128 | |
| Onno Crasborn; Menzo Windhouwer | |||
| As the creation of signed language resources is gaining speed world-wide,
the need for standards in this field becomes more acute. This paper discusses
the state of the field of signed language resources, their metadata
descriptions, and annotations that are typically made. It then describes the
role that ISOcat may play in this process and how it can stimulate
standardisation without imposing standards. Finally, it makes some initial
proposals for the thematic domain 'sign language' that was introduced in 2011. Keywords: signed language resources; metadata; data categories; standardization | |||
| Assessing Agreement on Segmentations by Means of Staccato, the Segmentation Agreement Calculator according to Thomann | | BIBA | Full-Text | 129-138 | |
| Andy Lücking; Sebastian Ptock; Kirsten Bergmann | |||
| Staccato, the Segmentation Agreement Calculator According to Thomann, is a software tool for assessing the degree of agreement of multiple segmentations of some time-related data (e.g., gesture phases or sign language constituents). The software implements an assessment procedure developed by Bruno Thomann and will be made publicly available. The article discusses the rationale of the agreement assessment procedure and points at future extensions of Staccato. | |||
| How Do Iconic Gestures Convey Visuo-Spatial Information? Bringing Together Empirical, Theoretical, and Simulation Studies | | BIBA | Full-Text | 139-150 | |
| Hannes Rieser; Kirsten Bergmann; Stefan Kopp | |||
| We investigate the question of how co-speech iconic gestures are used to convey visuo-spatial information in an interdisciplinary way, starting with a corpus-based empirical and theoretical perspective on how a typology of gesture form and a partial ontology of gesture meaning are related. Results provide the basis for a computational modeling approach that allows us to simulate the production of speaker-specific gesture forms to be realized with virtual agents. An evaluation of our simulation results and our methodology shows that the model is able to successfully approximate human gestural behavior use of iconic gestures, and moreover, that gestural behavior can improve how humans rate a virtual agent in terms of eloquence, competence, human-likeness, or likeability. | |||
| Thumb Modelling for the Generation of Sign Language | | BIBAK | Full-Text | 151-160 | |
| Maxime Delorme; Michael Filhol; Annelies Braffort | |||
| We present a simple kinematic model of the thumb for the animation of
virtual characters. The animation is made through a purely kinematic approach,
thus requires very precise limitations on the rotations of the thumb to be
realistic. The thumb is made opposable thanks to the addition of two bones
simulating the carpo-metacarpal complex. The bones are laid out to build a
virtual axis of rotation allowing the thumb to move in the opposed position.
The model is then evaluated by generating 22 static hand-shapes of Sign
Language and compared to previous work in animation. Keywords: Sign Language Synthesis; Skeleton Modelling; Inverse Kinematics; Thumb Model | |||
| Toward a Motor Theory of Sign Language Perception | | BIBAK | Full-Text | 161-172 | |
| Sylvie Gibet; Pierre-François Marteau; Kyle Duarte | |||
| Researches on signed languages still strongly dissociate linguistic issues
related on phonological and phonetic aspects, and gesture studies for
recognition and synthesis purposes. This paper focuses on the imbrication of
motion and meaning for the analysis, synthesis and evaluation of sign language
gestures. We discuss the relevance and interest of a motor theory of perception
in sign language communication. According to this theory, we consider that
linguistic knowledge is mapped on sensory-motor processes, and propose a
methodology based on the principle of a synthesis-by-analysis approach, guided
by an evaluation process that aims to validate some hypothesis and concepts of
this theory. Examples from existing studies illustrate the different concepts
and provide avenues for future work. Keywords: motor theory; sign language | |||
| Analysis and Description of Blinking in French Sign Language for Automatic Generation | | BIBAK | Full-Text | 173-182 | |
| Annelies Braffort; Emilie Chételat-Pelé | |||
| The present paper tackles blinking description within the context of
automatic generation of Sign Languages (SLs). Blinking is not much taken into
account into SL processing systems even though its importance is underlined in
several studies. Our purpose is to improve knowledge on blinking so as to be
able to generate them. We present the methodology we used for this purpose and
the results we obtained. We list the main categories we have identified in our
corpus, and present in more details an excerpt of our results corresponding to
the most frequent category, i.e. segmentation. Keywords: Sign Language; Non Manuals Gestures; Blinking | |||
| Grammar/Prosody Modelling in Greek Sign Language: Towards the Definition of Built-In Sign Synthesis Rules | | BIBAK | Full-Text | 183-193 | |
| Athanasia-Lida Dimou; Theodore Goulas; Eleni Efthimiou | |||
| The aim of the present work is to discuss a limited set of issues which
concern the grammar modelling of Greek Sign Language (GSL) within the framework
of improving the naturalness and more specifically the grammaticality of
synthetic GSL signing. This preliminary study addresses the linguistic issues
relating to specific grammar structures and their related prosodic expressive
markers through experimental implementation of the respective rules within a
sign synthesis support environment that was initially developed in order to
create lexical resources for GSL synthesis. Keywords: Greek Sign Language; sign synthesis; sign language grammar/ prosodic
modelling | |||
| Combining Two Synchronisation Methods in a Linguistic Model to Describe Sign Language | | BIBAK | Full-Text | 194-203 | |
| Michael Filhol | |||
| The context is Sign Language modelling for synthesis with 3d virtual signers
as output. Sign languages convey multi-linear information, hence allow for many
synchronisation patterns between the articulators of the body. Addressing the
problem that current models usually at best only cover one type of those
patterns, and in the wake of the recent description model Zebedee, we introduce
the Azalee extension, made to enable the description of any type of
synchronisation in Sign Language. Keywords: Sign Language modelling; synchronisation; AZee | |||
| Sign Segmentation Using Dynamics and Hand Configuration for Semi-automatic Annotation of Sign Language Corpora | | BIBAK | Full-Text | 204-215 | |
| Matilde Gonzalez; Christophe Collet | |||
| This paper address the problem of sign language video annotation. Nowadays
sign language segmentation is manually performed. This is time consuming, error
prone and no reproducible. In this paper we intend to provide an automatic
approach to segment signs. We use a particle filter based approach to track
hands and head. Motion features are used to classify segments performed with
one or two hands and to detect events. Events that have been detected in the
middle of a sign are removed considering hand shape features. Hand shape is
characterized using similarity measurements. Evaluation has been performed and
has shown the performance and limitation of the proposed approach. Keywords: Sign language; sign segmentation; automatic annotation | |||
| Integration of Gesture and Verbal Language: A Formal Semantics Approach | | BIBAK | Full-Text | 216-227 | |
| Gianluca Giorgolo | |||
| The paper presents a formal framework to model the fusion of gesture meaning
with the meaning of the co-occurring verbal fragment. The framework is based on
the formalization of two simple concepts, intersectivity and iconicity, which
form the core of most descriptive accounts of the interaction of the two
modalities. The formalization is presented as an extension of a well-known
framework for the analysis of meaning in natural language. We claim that a
proper formalization of these two concepts is sufficient to provide a
principled explanation of gestures accompanying different types of linguistic
expressions. The formalization also aims at providing a general mechanism
(iconicity) by which the meaning of gestures is extracted from their formal
appearance. Keywords: gesture language integration; formal gesture semantics; applied spatial
logics | |||
| Generating Co-speech Gestures for the Humanoid Robot NAO through BML | | BIBAK | Full-Text | 228-237 | |
| Quoc Anh Le; Catherine Pelachaud | |||
| We extend and develop an existing virtual agent system to generate
communicative gestures for different embodiments (i.e. virtual or physical
agents). This paper presents our ongoing work on an implementation of this
system for the NAO humanoid robot. From a specification of multi-modal
behaviors encoded with the behavior markup language, BML, the system
synchronizes and realizes the verbal and nonverbal behaviors on the robot. Keywords: Conversational humanoid robot; expressive gestures; gesture-speech
production and synchronization; Human-Robot Interaction; NAO; GRETA; FML; BML;
SAIBA | |||
| Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects | | BIBAK | Full-Text | 238-249 | |
| Thies Pfeiffer | |||
| Referring to objects using multimodal deictic expressions is an important
form of communication. This work addresses the question on how pragmatic
factors affect content distribution between the modalities speech and gesture.
This is done by analyzing a study on deictic pointing gestures to objects under
two conditions: with and without speech. The relevant pragmatic factor was the
distance to the referent object. As one main result two strategies were
identified which were used by participants to adapt their gestures to the
condition. This knowledge can be used, e.g., to improve the naturalness of
pointing gestures employed by embodied conversational agents. Keywords: object deixis; pointing; multimodal expressions | |||
| Making Space for Interaction: Architects Design Dialogues | | BIBAK | Full-Text | 250-261 | |
| Claude P. R. Heath; Patrick G. T. Healey | |||
| This exploratory research has taken a set of theoretical concepts as the
basis for testing a visualisation of body-centric gesture space: 1). Kendon's
transactional segments, 2). the manubrium as a central anatomical marker for
bodily movement, and 3). physical reach space. Using these, a 3D model of
gesture space has been designed in order to be applied to empirical data from
architects design meetings, articulating the role of gesture space overlaps
within the interaction.
Multi-dimensional drawing techniques have resulted in detailed visualisations of these overlaps. Illustrations show that the dialogue contributions can be mapped to distinct locations in the changing shared spaces, creating a spatial framework for the analysis and visualisation of the multi-dimensional topology of the interaction. This paper discusses a Case Study where this type of modelling can be applied empirically, indexing speech and gesture to the drawing subspaces of a group of architects. Keywords: Gesture space; interactional topologies; spatial indexing; spatial
resources; visuo-spatial deixis | |||
| Iconic Gestures in Face-to-Face TV Interviews | | BIBAK | Full-Text | 262-273 | |
| Maria Koutsombogera; Harris Papageorgiou | |||
| This paper presents a study of iconic gestures as attested in a corpus of
Greek face-to-face television interviews. The communicative significance of the
iconic gestures situated in an interactional context is examined with regards
to their semantics as well as the syntactic properties of the accompanying
speech. Iconic gestures are classified according to their semantic equivalents,
and are further linked to the phrasal units of the words co-occurring with
them, in order to provide evidence about the actual syntactic structures that
induce them. The findings support the communicative power of iconic gestures
and suggest a framework for their interpretation based on the interplay of
semantic and syntactic cues. Keywords: iconic gestures; face-to-face interviews; multimodal interaction; semantic
equivalents; syntactic structures | |||