Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness
to Touch-based Interactions
Touch Interaction
/
Wilkinson, Gerard
/
Kharrufa, Ahmed
/
Hook, Jonathan
/
Pursglove, Bradley
/
Wood, Gavin
/
Haeuser, Hendrik
/
Hammerla, Nils Y.
/
Hodges, Steve
/
Olivier, Patrick
Proceedings of the ACM CHI'16 Conference on Human Factors in Computing
Systems
2016-05-07
v.1
p.2832-2844
© Copyright 2016 ACM
Summary: Expressiveness, which we define as the extent to which rich and complex
intent can be conveyed through action, is a vital aspect of many human
interactions. For instance, paint on canvas is said to be an expressive medium,
because it affords the artist the ability to convey multifaceted emotional
intent through intricate manipulations of a brush. To date, touch devices have
failed to offer users a level of expressiveness in their interactions that
rivals that experienced by the painter and those completing other skilled
physical tasks. We investigate how data about hand movement -- provided by a
motion sensor, similar to those found in many smart watches or fitness trackers
-- can be used to expand the expressiveness of touch interactions. We begin by
introducing a conceptual model that formalizes a design space of possible
expressive touch interactions. We then describe and evaluate Expressy, an
approach that uses a wrist-worn inertial measurement unit to detect and
classify qualities of touch interaction that extend beyond those offered by
today's typical sensing hardware. We conclude by describing a number of sample
applications, which demonstrate the enhanced, expressive interaction
capabilities made possible by Expressy.
Video Showcase: Using Expressy to Showcase Expressiveness in Touch-based
Interactions
Video Showcase Presentations
/
Wilkinson, Gerard
/
Green, David Philip
/
Wood, Gavin
/
Kharrufa, Ahmed
/
Hook, Jonathan
/
Pursglove, Bradley
/
Haeuser, Hendrik
/
Hammerla, Nils Y.
/
Hodges, Steve
/
Olivier, Patrick
Extended Abstracts of the ACM CHI'16 Conference on Human Factors in
Computing Systems
2016-05-07
v.2
p.11
© Copyright 2016 ACM
Summary: We present a video demonstration of how information about hand movements,
generated from a wrist-worn IMU (inertial measurement unit), can be used to
provide expressiveness to touch-based interactions. The IMU identifies features
that were not previously accessible, such as instantaneous force, wrist roll
and pitch. We demonstrate a range of applications that have been extended using
Expressy, a system we describe in more detail in the full paper [1]. Tap force
allows users to express their intent behind an interaction before touch. Wrist
roll and pitch enriches the touch during the interaction. Flick force and wrist
roll allows users to follow-up their touch interaction.
Interactivity: Using Expressy to Demonstrate Expressiveness in Touch-based
Interactions
Interactivity Demos
/
Wilkinson, Gerard
/
Wood, Gavin
/
Hook, Jonathan
/
Nappey, Tom
/
Kharrufa, Ahmed
/
Pursglove, Bradley
/
Haeuser, Hendrik
/
Hammerla, Nils Y.
/
Hodges, Steve
/
Olivier, Patrick
Extended Abstracts of the ACM CHI'16 Conference on Human Factors in
Computing Systems
2016-05-07
v.2
p.3800-3803
© Copyright 2016 ACM
Summary: We present an interactivity demonstration of Expressy. A system that
augments existing touchscreen devices with a variety of continuous expressive
interaction capabilities, using movement data from a wrist-worn IMU. Our
demonstration comprises a set of applications that show how the expressive
touch interaction capabilities, offered by Expressy, can enable intuitive and
meaningful interactions, in contexts including productivity, entertainment and
lifestyle apps. This demo submission accompanies a full paper, describing a
conceptual model of expressive touch interaction and the implementation and
evaluation of Expressy.
