Project Jacquard: Interactive Digital Textiles at Scale
Everyday Objects as Interaction Surfaces
/
Poupyrev, Ivan
/
Gong, Nan-Wei
/
Fukuhara, Shiho
/
Karagozler, Mustafa Emre
/
Schwesig, Carsten
/
Robinson, Karen E.
Proceedings of the ACM CHI'16 Conference on Human Factors in Computing
Systems
2016-05-07
v.1
p.4216-4227
© Copyright 2016 ACM
Summary: Project Jacquard presents manufacturing technologies that enable deploying
invisible ubiquitous interactivity at scale. We propose novel interactive
textile materials that can be manufactured inexpensively using existing textile
weaving technology and equipment.
The development of touch-sensitive textiles begins with the design and
engineering of a new highly conductive yarn. The yarns and textiles can be
produced by standard textile manufacturing processes and can be dyed to any
color, made with a number of materials, and designed to a variety of
thicknesses and textures to be consistent with garment designers' needs.
We describe the development of yarn, textiles, garments, and user
interactivity; we present the opportunities and challenges of creating a
manufacturable interactive textile for wearable computing.
"I don't Want to Wear a Screen": Probing Perceptions of and Possibilities
for Dynamic Displays on Clothing
Body and Fashion
/
Devendorf, Laura
/
Lo, Joanne
/
Howell, Noura
/
Lee, Jung Lin
/
Gong, Nan-Wei
/
Karagozler, M. Emre
/
Fukuhara, Shiho
/
Poupyrev, Ivan
/
Paulos, Eric
/
Ryokai, Kimiko
Proceedings of the ACM CHI'16 Conference on Human Factors in Computing
Systems
2016-05-07
v.1
p.6028-6039
© Copyright 2016 ACM
Summary: This paper explores the role dynamic textile displays play in relation to
personal style: What does it mean to wear computationally responsive clothing
and why would one be motivated to do so? We developed a novel textile display
technology, called Ebb, and created several woven and crochet fabric swatches
that explored clothing-specific design possibilities. We engaged fashion
designers and non-designers in imagining how Ebb would integrate into their
design practice or personal style of dressing. Participants evaluated the
appeal and utility of clothing-based displays according to a very different set
of criteria than traditional screen-based computational displays. Specifically,
the slowness, low-resolution, and volatility of Ebb tended to be seen as assets
as opposed to technical limitations in the context of personal style.
Additionally, participants envisioned various ways that ambiguous, ambient, and
abstract displays of information could prompt new experiences in their everyday
lives. Our paper details the complex relationships between display and personal
style and offers a new design metaphor and extension of Gaver et al.'s original
descriptions of ambiguity in order to guide the design of clothing-based
displays for everyday life.
Protecting Your Children from Inappropriate Content in Mobile Apps: An
Automatic Maturity Rating Framework
Session 5F: Sentiment and Content Analysis
/
Hu, Bing
/
Liu, Bin
/
Gong, Neil Zhenqiang
/
Kong, Deguang
/
Jin, Hongxia
Proceedings of the 2015 ACM Conference on Information and Knowledge
Management
2015-10-19
p.1111-1120
© Copyright 2015 ACM
Summary: Mobile applications (Apps) could expose children or adolescents to mature
themes such as sexual content, violence and drug use, which results in an
inappropriate security and privacy risk for them. Therefore, mobile platforms
provide rating policies to label the maturity levels of Apps and the reasons
why an App has a given maturity level, which enables parents to select
maturity-appropriate Apps for their children. However, existing approaches to
implement these maturity rating policies are either costly (because of
expensive manually labeling) or inaccurate (because of no centralized
controls). In this work, we aim to design and build a machine learning
framework to automatically predict maturity levels for mobile Apps and the
associated reasons with a high accuracy and a low cost.
To this end, we take a multi-label classification approach to predict the
mature contents in a given App and then label the maturity level according to a
rating policy. Specifically, we extract novel features from App descriptions by
leveraging deep learning technique to automatically capture the semantic
similarity of pairwise words and adapt Support Vector Machine to capture label
correlations with pearson correlation in a multi-label classification setting.
Moreover, we evaluate our approach and various baseline methods using datasets
that we collected from both App Store and Google Play. We demonstrate that,
with only App descriptions, our approach already achieves 85% Precision for
predicting mature contents and 79% Precision for predicting maturity levels,
which substantially outperforms baseline methods.
Cutting Edge Vision: Metal Embedded Optics for Smart Knives
Video Showcase Presentations
/
Zoran, Amit
/
Gong, Nan-Wei
/
Shilkrot, Roy
/
Yan, Shuo
/
Maes, Pattie
Extended Abstracts of the ACM CHI'15 Conference on Human Factors in
Computing Systems
2015-04-18
v.2
p.185
© Copyright 2015 ACM
Summary: This video presents a novel technique for embedding optic fibers into a
metal blade to sense objects that the knife is cutting. In particular, we
present a design for a kitchen knife with fiber optics between the edge of the
blade and the handle, with a skin-color sensor that overcomes the complex
conditions in the kitchen. Hoping this design will lead to future work on
minimizing cooking injuries, our handheld device also includes a simple
finger-protection mechanism in the form of a retracting blade. We present our
novel hardware design, an initial study of imaging capabilities, and a
discussion of future directions.
Cutting Edge Vision: Metal Embedded Optics for Smart Knives
WIP Theme: Gesture and Multimodal
/
Zoran, Amit
/
Gong, Nan-Wei
/
Shilkrot, Roy
/
Yan, Shuo
/
Maes, Pattie
Extended Abstracts of the ACM CHI'15 Conference on Human Factors in
Computing Systems
2015-04-18
v.2
p.1223-1228
© Copyright 2015 ACM
Summary: We present a novel technique for embedding optic fibers into a metal blade
to sense objects that the knife is cutting. In particular, we present a design
for a kitchen knife with fiber optics between the edge of the blade and the
handle, with a skin-color sensor that overcomes the complex conditions in the
kitchen. Hoping this design will lead to future work on minimizing cooking
injuries, our handheld device also includes a simple finger-protection
mechanism in the form of a retracting blade. We present our novel hardware
design, an initial study of imaging capabilities, and a discussion of future
directions.
Connect the dots by understanding user status and transitions
Programming competition
/
Bao, Xuan
/
Shen, Yilin
/
Gong, Neil Zhenqiang
/
Jin, Hongxia
/
Hu, Bing
Adjunct Proceedings of the 2014 International Joint Conference on Pervasive
and Ubiquitous Computing
2014-09-13
v.2
p.361-366
© Copyright 2014 ACM
Summary: Human lives are composed by series of events and activities. Considerable
research effort has been made to probe, sense, and understand them. In our
research, we are interested in exploring the intrinsic string that connects all
these events together, that is, user status and transitions. Such transitions
can be reflected from multiple activity dimensions, ranging from our daily
mobility trajectories, app usage sequences, to communication patterns and
motion state switches. In this paper, we aim to identify whether a personalized
model can be learned to capture various user states from different sensing
dimensions and whether a unified view can be established to explain the state
transitions that drive the changes in user context during day-to-day routines.
To this end, we have explored two types of traces -- connected wifi
sequences and cell location trajectories. We first model the states among these
two individual dimensions. In the end, the identified states from both
dimensions are linked together to recognize the spatial-temporal relationship
between them. As we evaluate with the DeviceAnalyzer dataset, our method is
able to recognize a range of states such as "at home", "working", "commute" and
the transitions between them, all in an unsupervised manner.
PrintSense: a versatile sensing technique to support multimodal flexible
surface interaction
On and above the surface
/
Gong, Nan-Wei
/
Steimle, Jürgen
/
Olberding, Simon
/
Hodges, Steve
/
Gillian, Nicholas Edward
/
Kawahara, Yoshihiro
/
Paradiso, Joseph A.
Proceedings of ACM CHI 2014 Conference on Human Factors in Computing Systems
2014-04-26
v.1
p.1407-1410
© Copyright 2014 ACM
Summary: We present a multimodal on-surface and near-surface sensing technique for
planar, curved and flexible surfaces. Our technique leverages temporal
multiplexing of signals coming from a universal interdigitated electrode
design, which is printed as a single conductive layer on a flexible substrate.
It supports sensing of touch and proximity input, and moreover is capable of
capturing several levels of pressure and flexing. We leverage recent
developments in conductive inkjet printing as a way to prototype electrode
patterns, and combine this with our hardware module for supporting the full
range of sensing methods. As the technique is low-cost and easy to implement,
it is particularly well-suited for prototyping touch- and hover-based user
interfaces, including curved and deformable ones.
Inkjet-printed conductive patterns for physical manipulation of audio
signals
Adjunct 1: demonstrations
/
Gong, Nan-Wei
/
Zoran, Amit
/
Paradiso, Joseph A.
Adjunct Proceedings of the 2013 ACM Symposium on User Interface Software and
Technology
2013-10-08
v.2
p.13-14
© Copyright 2013 ACM
Summary: In this demo paper, we present the realization of a completely aesthetically
driven conductive image as a multi-modal music controller. Combining two
emerging technologies -- rapid prototyping with an off-the-shelf inkjet printer
using conductive ink and parametric graphic design, we are able to create an
interactive surface that is thin, flat, and flexible. This sensate surface can
be conformally wrapped around a simple curved surface, and unlike touch
screens, can accommodate complex structures and shapes such as holes on a
surface. We present the design and manufacturing flow and discuss the
technology behind this multi-modal sensing design. Our work seeks to offer a
new dimension of designing sonic interaction with graphic tools, playing and
learning music from a visual perspective and performing with expressive
physical manipulation.
Mime: compact, low power 3D gesture sensing for interaction with head
mounted displays
Sensing
/
Colaço, Andrea
/
Kirmani, Ahmed
/
Yang, Hye Soo
/
Gong, Nan-Wei
/
Schmandt, Chris
/
Goyal, Vivek K.
Proceedings of the 2013 ACM Symposium on User Interface Software and
Technology
2013-10-08
v.1
p.227-236
© Copyright 2013 ACM
Summary: We present Mime, a compact, low-power 3D sensor for unencumbered free-form,
single-handed gestural interaction with head-mounted displays (HMDs). Mime
introduces a real-time signal processing framework that combines a novel
three-pixel time-of-flight (TOF) module with a standard RGB camera. The TOF
module achieves accurate 3D hand localization and tracking, and it thus enables
motion-controlled gestures. The joint processing of 3D information with RGB
image data enables finer, shape-based gestural interaction.
Our Mime hardware prototype achieves fast and precise 3D gestural control.
Compared with state-of-the-art 3D sensors like TOF cameras, the Microsoft
Kinect and the Leap Motion Controller, Mime offers several key advantages for
mobile applications and HMD use cases: very small size, daylight insensitivity,
and low power consumption. Mime is built using standard, low-cost
optoelectronic components and promises to be an inexpensive technology that can
either be a peripheral component or be embedded within the HMD unit. We
demonstrate the utility of the Mime sensor for HMD interaction with a variety
of application scenarios, including 3D spatial input using close-range
gestures, gaming, on-the-move interaction, and operation in cluttered
environments and in broad daylight conditions.
A cuttable multi-touch sensor
Sensing
/
Olberding, Simon
/
Gong, Nan-Wei
/
Tiab, John
/
Paradiso, Joseph A.
/
Steimle, Jürgen
Proceedings of the 2013 ACM Symposium on User Interface Software and
Technology
2013-10-08
v.1
p.245-254
© Copyright 2013 ACM
Summary: We propose cutting as a novel paradigm for ad-hoc customization of printed
electronic components. As a first instantiation, we contribute a printed
capacitive multi-touch sensor, which can be cut by the end-user to modify its
size and shape. This very direct manipulation allows the end-user to easily
make real-world objects and surfaces touch-interactive, to augment physical
prototypes and to enhance paper craft. We contribute a set of technical
principles for the design of printable circuitry that makes the sensor more
robust against cuts, damages and removed areas. This includes novel physical
topologies and printed forward error correction. A technical evaluation
compares different topologies and shows that the sensor remains functional when
cut to a different shape.
Conductive inkjet printed DIY music control surface
Video showcase presentations
/
Gong, Nan-Wei
/
Zhao, Nan
/
Paradiso, Joseph A.
Extended Abstracts of ACM CHI'13 Conference on Human Factors in Computing
Systems
2013-04-27
v.2
p.2895-2896
© Copyright 2013 ACM
Summary: We developed a novel music control sensate surface, which enables retrofit
integration between any musical instruments with a versatile, customizable, and
essentially cost-effective user interface. Our project presents new
opportunities in customizable, flexible interface design since, unlike just
using a touch screen, it adapts very well to non-square or non-flat surfaces or
surfaces with holes. Our design requires an interactive circuit that is made in
a computer-aided design environment and printed from a conductive inkjet
printer on a PET substrate. This method allows us to create a functional
decoration on the controller surface, combining graphic design and music
performance with expressive physical manipulation. We present an example of
implementation on an electric ukulele and provide several design examples to
demonstrate the versatile capabilities of this system.
A Customizable Sensate Surface for Music Control
Posters
/
Gong, Nan-Wei
/
Zhao, Nan
/
Paradiso, Joseph
NIME 2012: New Interfaces for Musical Expression
2012-05-21
p.201
Keywords: Sensate surface, music controller skin, customizable controller surface,
flexible electronics
© Copyright 2012 Authors
Summary: This paper describes a novel music control sensate surface, which enables
integration between any musical instruments with a versatile, customizable, and
essentially cost-effective user interface. This sensate surface is based on
conductive inkjet printing technology which allows capacitive sensor electrodes
and connections between electronics components to be printed onto a large roll
of flexible substrate that is unrestricted in length. The high dynamic range
capacitive sensing electrodes can not only infer touch, but near-range,
non-contact gestural nuance in a music performance. With this sensate surface,
users can "cut" out their desired shapes, "paste" the number of inputs, and
customize their controller interface, which can then send signals wirelessly to
effects or software synthesizers. We seek to find a solution for integrating
the form factor of traditional music controllers seamlessly on top of one's
music instrument and meanwhile adding expressiveness to the music performance
by sensing and incorporating movements and gestures to manipulate the musical
output. We present an example of implementation on an electric ukulele and
provide several design examples to demonstrate the versatile capabilities of
this system.
Leveraging conductive inkjet technology to build a scalable and versatile
surface for ubiquitous sensing
Novel ubiquitous technologies
/
Gong, Nan-Wei
/
Hodges, Steve
/
Paradiso, Joseph A.
Proceedings of the 2011 International Conference on Ubiquitous Computing
2011-09-17
p.45-54
© Copyright 2011 ACM
Summary: In this paper we describe the design and implementation of a new versatile,
scalable and cost-effective sensate surface. The system is based on a new
conductive inkjet technology, which allows capacitive sensor electrodes and
different types of RF antennas to be cheaply printed onto a roll of flexible
substrate that may be many meters long. By deploying this surface on (or under)
a floor it is possible to detect the presence and whereabouts of users through
both passive and active capacitive coupling schemes. We have also incorporated
GSM and NFC electromagnetic radiation sensing and piezoelectric pressure and
vibration detection. We report on a number of experiments which evaluate
sensing performance based on a 2.5m x 0.3m hardware test-bed. We describe some
potential applications for this technology and highlight a number of
improvements we have in mind.
Dynamic Privacy Management in Pervasive Sensor Networks
/
Gong, Nan-Wei
/
Laibowitz, Mathew
/
Paradiso, Joseph A.
Proceedings of the 2010 International Joint Conference on Ambient
Intelligence
2010-11-10
p.96-106
Keywords: dynamic privacy management; ubiquitous computing; active badge system;
pervasive sensor networks
© Copyright 2010 Springer-Verlag
Summary: This paper describes the design and implementation of a dynamic privacy
management system aimed at enabling tangible privacy control and feedback in a
pervasive sensor network. Our work began with the development of a potentially
invasive sensor network (with high resolution video, audio, and motion tracking
capabilities) featuring different interactive applications that created
incentive for accepting this network as an extension of people's daily social
space. A user study was then conducted to evaluate several privacy management
approaches -- an active badge system for both online and on-site control,
on/off power switches for physically disabling the hardware, and touch screen
input control. Results from a user study indicated that an active badge for
on-site privacy control is the most preferable method among all provided
options. We present a set of results that yield insight into the
privacy/benefit tradeoff from various sensing capabilities in pervasive sensor
networks and how privacy settings and user behavior relate in these
environments.
MusicGrip: A Writing Instrument for Music Control
/
Gong, Nan-Wei
/
Laibowitz, Mat
/
Paradiso, Joseph A.
NIME 2009: New Interfaces for Musical Expression
2009-06-04
p.74-77
© Copyright 2009 Authors
