| Interactive viscosity | | BIBAK | Full-Text | 1-2 | |
| Stephen L. Macknik; Susana Martinez-Conde | |||
| Your conscious experience is not a function of the world, it is a function
of the neural networks of your brain. Therefore, the biology of the brain and
consciousness is as fundamental to understanding the universe, as we know it,
as the high-energy physics of subatomic particles. This is especially true for
the study of sensory and cognitive illusions, since they represent effects that
clearly stand out as not representing the real world. That is, since illusions
don't match reality we can know that by studying illusions we are studying
exactly what the brain is actually doing, and not just what we think the brain
should be doing. Your brain does a staggering amount of pragmatic self-dealing
guesswork and outright confabulation in order to construct the highly imperfect
mental simulation of reality known as "consciousness." This is not to say that
objective reality isn't "out there" in a very real sense -- but no one lives
there. No one's ever even been there for a visit. Ironically, the fact that
consciousness feels like a solid, robust, fact-rich transcript of reality is
just one of the countless illusions your brain creates for itself.
Illusions are not errors of the brain. Far from it. Illusions arise from processes that are critical to our survival. Our brains have developed illusory processes so that we may experience the world in a ready-to-consume manner. Remove the machinery of illusion, and you unwind the entire tapestry of human awareness. Illusions are those perceptual experiences that do not match the physical reality. They are therefore exquisite tools with which to analyze the neural correlates of human perception and consciousness. Neuroscientists have long known that they can only be sure of where they stand, in terms of correlating neural responses to awareness, when they correlate the awareness of an illusion to the brain's response, specifically because of the illusions' mismatch with reality. The study of illusions is therefore of critical importance to the understanding of the basic mechanisms of sensory perception and conscious awareness. If you've ever seen a good magician perform, you know how thrilling it is to watch the impossible happening before your eyes. The laws of physics, probability, psychology and common sense -- the four trusty compass points in your mental map of reality -- are suddenly turned into liabilities. Objects and people appear, vanish, levitate, transpose, transform, and with all your smarts you can't imagine how it's being done. Magicians are the premier artists of attention and awareness, and they manipulate our cognition like clay on a potter's wheel. And the mechanisms underlying magic perception have implications for our daily lives. The magical arts work because humans have hardwired processes of attention and awareness that are hackable. By understanding how magicians hack our brains, we can better understand how we work. Keywords: illusions, magic, the mind | |||
| Ripples: utilizing per-contact visualizations to improve user interaction with touch displays | | BIBAK | Full-Text | 3-12 | |
| Daniel Wigdor; Sarah Williams; Michael Cronin; Robert Levy; Katie White; Maxim Mazeev; Hrvoje Benko | |||
| We present Ripples, a system which enables visualizations around each
contact point on a touch display and, through these visualizations, provides
feedback to the user about successes and errors of their touch interactions.
Our visualization system is engineered to be overlaid on top of existing
applications without requiring the applications to be modified in any way, and
functions independently of the application's responses to user input. Ripples
reduces the fundamental problem of ambiguity of feedback when an action results
in an unexpected behaviour. This ambiguity can be caused by a wide variety of
sources. We describe the ambiguity problem, and identify those sources. We then
define a set of visual states and transitions needed to resolve this ambiguity,
of use to anyone designing touch applications or systems. We then present the
Ripples implementation of visualizations for those states, and the results of a
user study demonstrating user preference for the system, and demonstrating its
utility in reducing errors. Keywords: fat fingers, feedback ambiguity, multi-touch, no touch left behind,
precision input, tabletop, touch | |||
| Contact area interaction with sliding widgets | | BIBAK | Full-Text | 13-22 | |
| Tomer Moscovich | |||
| We show how to design touchscreen widgets that respond to a finger's contact
area. In standard touchscreen systems a finger often appears to touch several
screen objects, but the system responds as though only a single pixel is
touched. In contact area interaction all objects under the finger respond to
the touch. Users activate control widgets by sliding a movable element, as
though flipping a switch. These Sliding Widgets resolve selection ambiguity and
provide designers with a rich vocabulary of self-disclosing interaction
mechanism. We showcase the design of several types of Sliding Widgets, and
report study results showing that the simplest of these widgets, the Sliding
Button, performs on-par with medium-sized pushbuttons and offers greater
accuracy for small-sized buttons. Keywords: interactive surfaces, touch input, touchscreen | |||
| Detecting and leveraging finger orientation for interaction with direct-touch surfaces | | BIBAK | Full-Text | 23-32 | |
| Feng Wang; Xiang Cao; Xiangshi Ren; Pourang Irani | |||
| Current interactions on direct-touch interactive surfaces are often modeled
based on properties of the input channel that are common in traditional
graphical user interfaces (GUI) such as x-y coordinate information. Leveraging
additional information available on the surfaces could potentially result in
richer and novel interactions. In this paper we specifically explore the role
of finger orientation. This property is typically ignored in touch-based
interactions partly because of the ambiguity in determining it solely from the
contact shape. We present a simple algorithm that unambiguously detects the
directed finger orientation vector in real-time from contact information only,
by considering the dynamics of the finger landing process. Results of an
experimental evaluation show that our algorithm is stable and accurate. We then
demonstrate how finger orientation can be leveraged to enable novel
interactions and to infer higher-level information such as hand occlusion or
user position. We present a set of orientation-aware interaction techniques and
widgets for direct-touch surfaces. Keywords: direct-touch surface, finger orientation, orientation aware interface | |||
| Mouse 2.0: multi-touch meets the mouse | | BIBAK | Full-Text | 33-42 | |
| Nicolas Villar; Shahram Izadi; Dan Rosenfeld; Hrvoje Benko; John Helmes; Jonathan Westhues; Steve Hodges; Eyal Ofek; Alex Butler; Xiang Cao; Billy Chen | |||
| In this paper we present novel input devices that combine the standard
capabilities of a computer mouse with multi-touch sensing. Our goal is to
enrich traditional pointer-based desktop interactions with touch and gestures.
To chart the design space, we present five different multi-touch mouse
implementations. Each explores a different touch sensing strategy, which leads
to differing form-factors and hence interactive possibilities. In addition to
the detailed description of hardware and software implementations of our
prototypes, we discuss the relative strengths, limitations and affordances of
these novel input devices as informed by the results of a preliminary user
study. Keywords: desktop computing, input devices, mouse, multi-touch, novel hardware,
surface computing | |||
| PhotoelasticTouch: transparent rubbery tangible interface using an LCD and photoelasticity | | BIBAK | Full-Text | 43-50 | |
| Toshiki Sato; Haruko Mamiya; Hideki Koike; Kentaro Fukuchi | |||
| PhotoelasticTouch is a novel tabletop system designed to intuitively
facilitate touch-based interaction via real objects made from transparent
elastic material. The system utilizes vision-based recognition techniques and
the photoelastic properties of the transparent rubber to recognize deformed
regions of the elastic material. Our system works with elastic materials over a
wide variety of shapes and does not require any explicit visual markers.
Compared to traditional interactive surfaces, our 2.5 dimensional interface
system enables direct touch interaction and soft tactile feedback. In this
paper we present our force sensing technique using photoelasticity and describe
the implementation of our prototype system. We also present three practical
applications of PhotoelasticTouch, a force-sensitive touch panel, a tangible
face application, and a paint application. Keywords: direct manipulation, interactive surfaces, multi-touch, photoelasticity,
tabletop interfaces | |||
| A reconfigurable ferromagnetic input device | | BIBAK | Full-Text | 51-54 | |
| Jonathan Hook; Stuart Taylor; Alex Butler; Nicolas Villar; Shahram Izadi | |||
| We present a novel hardware device based on ferromagnetic sensing, capable
of detecting the presence, position and deformation of any ferrous object
placed on or near its surface. These objects can include ball bearings,
magnets, iron filings, and soft malleable bladders filled with ferrofluid. Our
technology can be used to build reconfigurable input devices -- where the
physical form of the input device can be assembled using combinations of such
ferrous objects. This allows users to rapidly construct new forms of input
device, such as a trackball-style device based on a single large ball bearing,
tangible mixers based on a collection of sliders and buttons with ferrous
components, and multi-touch malleable surfaces using a ferrofluid bladder. We
discuss the implementation of our technology, its strengths and limitations,
and potential application scenarios. Keywords: ferromagnetic sensing, malleable surface, multi-touch, reconfigurable input
device, tangibles | |||
| A practical pressure sensitive computer keyboard | | BIBAK | Full-Text | 55-58 | |
| Paul H. Dietz; Benjamin Eidelson; Jonathan Westhues; Steven Bathiche | |||
| A pressure sensitive computer keyboard is presented that independently
senses the force level on every depressed key. The design leverages existing
membrane technologies and is suitable for low-cost, high-volume manufacturing.
A number of representative applications are discussed. Keywords: force pressure sensitive keyboard | |||
| EverybodyLovesSketch: 3D sketching for a broader audience | | BIBAK | Full-Text | 59-68 | |
| Seok-Hyung Bae; Ravin Balakrishnan; Karan Singh | |||
| We present EverybodyLovesSketch, a gesture-based 3D curve sketching system
for rapid ideation and visualization of 3D forms, aimed at a broad audience. We
first analyze traditional perspective drawing in professional practice. We then
design a system built upon the paradigm of ILoveSketch, a 3D curve drawing
system for design professionals. The new system incorporates many interaction
aspects of perspective drawing with judicious automation to enable novices with
no perspective training to proficiently create 3D curve sketches.
EverybodyLovesSketch supports a number of novel interactions: tick-based sketch
plane selection, single view definition of arbitrary extrusion vectors,
multiple extruded surface sketching, copy-and-project of 3D curves, freeform
surface sketching, and an interactive perspective grid. Finally, we present a
study involving 49 high school students (with no formal artistic training) who
each learned and used the system over 11 days, which provides detailed insights
into the popularity, power and usability of the various techniques, and shows
our system to be easily learnt and effectively used, with broad appeal. Keywords: 3D sketching, axis widget, gestural interface, learnability, perspective
sketching, sketch surface | |||
| A screen-space formulation for 2D and 3D direct manipulation | | BIBAK | Full-Text | 69-78 | |
| Jason L. Reisman; Philip L. Davidson; Jefferson Y. Han | |||
| Rotate-Scale-Translate (RST) interactions have become the de facto standard
when interacting with two-dimensional (2D) contexts in single-touch and
multi-touch environments. Because the use of RST has thus far focused almost
entirely on 2D, there are not yet standard techniques for extending these
principles into three dimensions. In this paper we describe a screen-space
method which fully captures the semantics of the traditional 2D RST multi-touch
interaction, but also allows us to extend these same principles into
three-dimensional (3D) interaction. Just like RST allows users to directly
manipulate 2D contexts with two or more points, our method allows the user to
directly manipulate 3D objects with three or more points. We show some novel
interactions, which take perspective into account and are thus not available in
orthographic environments. Furthermore, we identify key ambiguities and
unexpected behaviors that arise when performing direct manipulation in 3D and
offer solutions to mitigate the difficulties each presents. Finally, we show
how to extend our method to meet application-specific control objectives, as
well as show our method working in some example environments. Keywords: constraints, direct manipulation, multi-touch, optimization, pressure | |||
| Activity analysis enabling real-time video communication on mobile phones for deaf users | | BIBAK | Full-Text | 79-88 | |
| Neva Cherniavsky; Jaehong Chon; Jacob O. Wobbrock; Richard E. Ladner; Eve A. Riskin | |||
| We describe our system called MobileASL for real-time video communication on
the current U.S. mobile phone network. The goal of MobileASL is to enable Deaf
people to communicate with Sign Language over mobile phones by compressing and
transmitting sign language video in real-time on an off-the-shelf mobile phone,
which has a weak processor, uses limited bandwidth, and has little battery
capacity. We develop several H.264-compliant algorithms to save system
resources while maintaining ASL intelligibility by focusing on the important
segments of the video. We employ a dynamic skin-based region-of-interest (ROI)
that encodes the skin at higher quality at the expense of the rest of the
video. We also automatically recognize periods of signing versus not signing
and raise and lower the frame rate accordingly, a technique we call variable
frame rate (VFR).
We show that our variable frame rate technique results in a 47% gain in battery life on the phone, corresponding to an extra 68 minutes of talk time. We also evaluate our system in a user study. Participants fluent in ASL engage in unconstrained conversations over mobile phones in a laboratory setting. We find that the ROI increases intelligibility and decreases guessing. VFR increases the need for signs to be repeated and the number of conversational breakdowns, but does not affect the users' perception of adopting the technology. These results show that our sign language sensitive algorithms can save considerable resources without sacrificing intelligibility. Keywords: computer vision, mobileasl, region-of-interest, sign language, variable
frame rate, video compression | |||
| User guided audio selection from complex sound mixtures | | BIBAK | Full-Text | 89-92 | |
| Paris Smaragdis | |||
| In this paper we present a novel interface for selecting sounds in audio
mixtures. Traditional interfaces in audio editors provide a graphical
representation of sounds which is either a waveform, or some variation of a
time/frequency transform. Although with these representations a user might be
able to visually identify elements of sounds in a mixture, they do not
facilitate object-specific editing (e.g. selecting only the voice of a singer
in a song). This interface uses audio guidance from a user in order to select a
target sound within a mixture. The user is asked to vocalize (or otherwise
sonically represent) the desired target sound, and an automatic process
identifies and isolates the elements of the mixture that best relate to the
user's input. This way of pointing to specific parts of an audio stream allows
a user to perform audio selections which would have been infeasible otherwise. Keywords: audio interfaces | |||
| TapSongs: tapping rhythm-based passwords on a single binary sensor | | BIBAK | Full-Text | 93-96 | |
| Jacob Otto Wobbrock | |||
| TapSongs are presented, which enable user authentication on a single
"binary" sensor (e.g., button) by matching the rhythm of tap down/up events to
a jingle timing model created by the user. We describe our matching algorithm,
which employs absolute match criteria and learns from successful logins. We
also present a study of 10 subjects showing that after they created their own
TapSong models from 12 examples (< 2 minutes), their subsequent login
attempts were 83.2% successful. Furthermore, aural and visual eavesdropping of
the experimenter's logins resulted in only 10.7% successful impostor logins by
subjects. Even when subjects heard the target jingles played by a synthesized
piano, they were only 19.4% successful logging in as imposters. These results
are attributable to subtle but reliable individual differences in people's
tapping, which are supported by prior findings in music psychology. Keywords: binary sensors, jingles, mobile devices, password entry, rhythm, songs,
tapping, temporal strings, user authentication | |||
| Collabio: a game for annotating people within social networks | | BIBAK | Full-Text | 97-100 | |
| Michael Bernstein; Desney Tan; Greg Smith; Mary Czerwinski; Eric Horvitz | |||
| We present Collabio, a social tagging game within an online social network
that encourages friends to tag one another. Collabio's approach of
incentivizing members of the social network to generate information about each
other produces personalizing information about its users. We report usage log
analysis, survey data, and a rating exercise demonstrating that Collabio tags
are accurate and augment information that could have been scraped online. Keywords: human computation, social computing, social tagging | |||
| Disappearing mobile devices | | BIBAK | Full-Text | 101-110 | |
| Tao Ni; Patrick Baudisch | |||
| In this paper, we extrapolate the evolution of mobile devices in one
specific direction, namely miniaturization. While we maintain the concept of a
device that people are aware of and interact with intentionally, we envision
that this concept can become small enough to allow invisible integration into
arbitrary surfaces or human skin, and thus truly ubiquitous use. This outcome
assumed, we investigate what technology would be most likely to provide the
basis for these devices, what abilities such devices can be expected to have,
and whether or not devices that size can still allow for meaningful
interaction. We survey candidate technologies, drill down on gesture-based
interaction, and demonstrate how it can be adapted to the desired form factors.
While the resulting devices offer only the bare minimum in feedback and only
the most basic interactions, we demonstrate that simple applications remain
possible. We complete our exploration with two studies in which we investigate
the affordance of these devices more concretely, namely marking and text entry
using a gesture alphabet. Keywords: gesture, input device, interaction technique, miniaturization, mobile
device, sensor, ubicomp, wearable | |||
| SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices | | BIBAK | Full-Text | 111-120 | |
| Koji Yatani; Khai Nhut Truong | |||
| One of the challenges with using mobile touch-screen devices is that they do
not provide tactile feedback to the user. Thus, the user is required to look at
the screen to interact with these devices. In this paper, we present SemFeel, a
tactile feedback system which informs the user about the presence of an object
where she touches on the screen and can offer additional semantic information
about that item. Through multiple vibration motors that we attached to the
backside of a mobile touch-screen device, SemFeel can generate different
patterns of vibration, such as ones that flow from right to left or from top to
bottom, to help the user interact with a mobile device. Through two user
studies, we show that users can distinguish ten different patterns, including
linear patterns and a circular pattern, at approximately 90% accuracy, and that
SemFeel supports accurate eyes-free interactions. Keywords: mobile device, multiple vibration motors, tactile feedback, touch screen | |||
| Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices | | BIBAK | Full-Text | 121-124 | |
| Chris Harrison; Scott E. Hudson | |||
| We present Abracadabra, a magnetically driven input technique that offers
users wireless, unpowered, high fidelity finger input for mobile devices with
very small screens. By extending the input area to many times the size of the
device's screen, our approach is able to offer a high C-D gain, enabling fine
motor control. Additionally, screen occlusion can be reduced by moving
interaction off of the display and into unused space around the device. We
discuss several example applications as a proof of concept. Finally, results
from our user study indicate radial targets as small as 16 degrees can achieve
greater than 92% selection accuracy, outperforming comparable radial,
touch-based finger input. Keywords: cursor, finger input, gesture, interaction techniques, magnetic, mobile
devices, pointing, small screens | |||
| Virtual shelves: interactions with orientation aware devices | | BIBAK | Full-Text | 125-128 | |
| Frank Chun Yat Li; David Dearman; Khai N. Truong | |||
| Triggering shortcuts or actions on a mobile device often requires a long
sequence of key presses. Because the functions of buttons are highly dependent
on the current application's context, users are required to look at the display
during interaction, even in many mobile situations when eyes-free interactions
may be preferable. We present Virtual Shelves, a technique to trigger
programmable shortcuts that leverages the user's spatial awareness and
kinesthetic memory. With Virtual Shelves, the user triggers shortcuts by
orienting a spatially-aware mobile device within the circular hemisphere in
front of her. This space is segmented into definable and selectable regions
along the phi and theta planes. We show that users can accurately point to 7
regions on the theta and 4 regions on the phi plane using only their
kinesthetic memory. Building upon these results, we then evaluate a
proof-of-concept prototype of the Virtual Shelves using a Nokia N93. The
results show that Virtual Shelves is faster than the N93's native interface for
common mobile phone tasks. Keywords: kinesthetic memory, mobile computing, spatial memory, spatially aware
devices | |||
| Bonfire: a nomadic system for hybrid laptop-tabletop interaction | | BIBAK | Full-Text | 129-138 | |
| Shaun K. Kane; Daniel Avrahami; Jacob O. Wobbrock; Beverly Harrison; Adam D. Rea; Matthai Philipose; Anthony LaMarca | |||
| We present Bonfire, a self-contained mobile computing system that uses two
laptop-mounted laser micro-projectors to project an interactive display space
to either side of a laptop keyboard. Coupled with each micro-projector is a
camera to enable hand gesture tracking, object recognition, and information
transfer within the projected space. Thus, Bonfire is neither a pure laptop
system nor a pure tabletop system, but an integration of the two into one new
nomadic computing platform. This integration (1) enables observing the
periphery and responding appropriately, e.g., to the casual placement of
objects within its field of view, (2) enables integration between physical and
digital objects via computer vision, (3) provides a horizontal surface in
tandem with the usual vertical laptop display, allowing direct pointing and
gestures, and (4) enlarges the input/output space to enrich existing
applications. We describe Bonfire's architecture, and offer scenarios that
highlight Bonfire's advantages. We also include lessons learned and insights
for further development and use. Keywords: ambient interaction, computer vision, extended display, gestures, laptop,
micro-projector, object recognition, peripheral display, surface, tabletop,
tangible bits | |||
| Interactions in the air: adding further depth to interactive tabletops | | BIBAK | Full-Text | 139-148 | |
| Otmar Hilliges; Shahram Izadi; Andrew D. Wilson; Steve Hodges; Armando Garcia-Mendoza; Andreas Butz | |||
| Although interactive surfaces have many unique and compelling qualities, the
interactions they support are by their very nature bound to the display
surface. In this paper we present a technique for users to seamlessly switch
between interacting on the tabletop surface to above it. Our aim is to leverage
the space above the surface in combination with the regular tabletop display to
allow more intuitive manipulation of digital content in three-dimensions. Our
goal is to design a technique that closely resembles the ways we manipulate
physical objects in the real-world; conceptually, allowing virtual objects to
be 'picked up' off the tabletop surface in order to manipulate their three
dimensional position or orientation. We chart the evolution of this technique,
implemented on two rear projection-vision tabletops. Both use special
projection screen materials to allow sensing at significant depths beyond the
display. Existing and new computer vision techniques are used to sense hand
gestures and postures above the tabletop, which can be used alongside more
familiar multi-touch interactions. Interacting above the surface in this way
opens up many interesting challenges. In particular it breaks the direct
interaction metaphor that most tabletops afford. We present a novel
shadow-based technique to help alleviate this issue. We discuss the strengths
and limitations of our technique based on our own observations and initial user
feedback, and provide various insights from comparing, and contrasting, our
tabletop implementations. Keywords: 3D, 3D graphics, computer vision, depth-sensing cameras, holoscreen,
interactive surfaces, surfaces, switchable diffusers, tabletop | |||
| Augmenting interactive tables with mice & keyboards | | BIBAK | Full-Text | 149-152 | |
| Björn Hartmann; Meredith Ringel Morris; Hrvoje Benko; Andrew D. Wilson | |||
| This note examines the role traditional input devices can play in surface
computing. Mice and keyboards can enhance tabletop technologies since they
support high fidelity input, facilitate interaction with distant objects, and
serve as a proxy for user identity and position. Interactive tabletops, in
turn, can enhance the functionality of traditional input devices: they provide
spatial sensing, augment devices with co-located visual content, and support
connections among a plurality of devices. We introduce eight interaction
techniques for a table with mice and keyboards, and we discuss the design space
of such interactions. Keywords: interactive tabletops, surface computing | |||
| ARC-Pad: absolute+relative cursor positioning for large displays with a mobile touchscreen | | BIBAK | Full-Text | 153-156 | |
| David C. McCallum; Pourang Irani | |||
| We introduce ARC-Pad (Absolute+Relative Cursor pad), a novel technique for
interacting with large displays using a mobile phone's touchscreen. In ARC-Pad
we combine ab-solute and relative cursor positioning. Tapping with ARC-Pad
causes the cursor to jump to the corresponding location on the screen,
providing rapid movement across large distances. For fine position control,
users can also clutch using relative mode. Unlike prior hybrid cursor
positioning techniques, ARC-Pad does not require an explicit switch between
relative and absolute modes. We compared ARC-Pad with the relative positioning
commonly found on touchpads. Users were given a target acquisition task on a
large display, and results showed that they were faster with ARC-Pad, without
sacrificing accuracy. Users welcomed the benefits associated with ARC-Pad. Keywords: absolute position, clutching, cursor, touchpad | |||
| Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines | | BIBAK | Full-Text | 157-166 | |
| Erin Treacy Solovey; Audrey Girouard; Krysta Chauncey; Leanne M. Hirshfield; Angelo Sassaroli; Feng Zheng; Sergio Fantini; Robert J. K. Jacob | |||
| Because functional near-infrared spectroscopy (fNIRS) eases many of the
restrictions of other brain sensors, it has potential to open up new
possibilities for HCI research. From our experience using fNIRS technology for
HCI, we identify several considerations and provide guidelines for using fNIRS
in realistic HCI laboratory settings. We empirically examine whether typical
human behavior (e.g. head and facial movement) or computer interaction (e.g.
keyboard and mouse usage) interfere with brain measurement using fNIRS. Based
on the results of our study, we establish which physical behaviors inherent in
computer usage interfere with accurate fNIRS sensing of cognitive state
information, which can be corrected in data analysis, and which are acceptable.
With these findings, we hope to facilitate further adoption of fNIRS brain
sensing technology in HCI research. Keywords: BCI, brain-computer interface, fNIRS, functional near-infrared spectroscopy,
human cognition | |||
| Enabling always-available input with muscle-computer interfaces | | BIBAK | Full-Text | 167-176 | |
| T. Scott Saponas; Desney S. Tan; Dan Morris; Ravin Balakrishnan; Jim Turner; James A. Landay | |||
| Previous work has demonstrated the viability of applying offline analysis to
interpret forearm electromyography (EMG) and classify finger gestures on a
physical surface. We extend those results to bring us closer to using
muscle-computer interfaces for always-available input in real-world
applications. We leverage existing taxonomies of natural human grips to develop
a gesture set covering interaction in free space even when hands are busy with
other objects. We present a system that classifies these gestures in real-time
and we introduce a bi-manual paradigm that enables use in interactive systems.
We report experimental results demonstrating four-finger classification
accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88%
when carrying a weighted bag. We further show generalizability across different
arm postures and explore the tradeoffs of providing real-time visual feedback. Keywords: electromyography (EMG), input, interaction, muscle-computer interface | |||
| Optically sensing tongue gestures for computer input | | BIBAK | Full-Text | 177-180 | |
| T. Scott Saponas; Daniel Kelly; Babak A. Parviz; Desney S. Tan | |||
| Many patients with paralyzing injuries or medical conditions retain the use
of their cranial nerves, which control the eyes, jaw, and tongue. While
researchers have explored eye-tracking and speech technologies for these
patients, we believe there is potential for directly sensing explicit tongue
movement for controlling computers. In this paper, we describe a novel approach
of using infrared optical sensors embedded within a dental retainer to sense
tongue gestures. We describe an experiment showing our system effectively
discriminating between four simple gestures with over 90% accuracy. In this
experiment, users were also able to play the popular game Tetris with their
tongues. Finally, we present lessons learned and opportunities for future work. Keywords: gestures, infrared, tongue-computer interface | |||
| The rise of the expert amateur: DIY culture and citizen science | | BIBAK | Full-Text | 181-182 | |
| Eric Paulos | |||
| We are at an important technological inflection point. Most of our computing
systems have been designed and built by professionally trained experts (i.e. us
-- computer scientists, engineers, and designers) for use in specific domains
and to solve explicit problems. Artifacts often called "user manuals"
traditionally prescribed the appropriate usage of these tools and implied an
acceptable etiquette for interaction and experience. A fringe group of
individuals usually labeled "hackers" or "nerds" have challenged this
producer-consumer model of technology by hacking novel hardware and software
features to "improve" our research and products while a similar creative group
of technicians called "artists" have re-directed the techniques, tools, and
tenets of accepted technological usage away from their typical manifestations
in practicality and product. Over time the technological artifacts of these
fringe groups and the support for their rhetoric have gained them a foothold
into computing culture and eroded the established power discontinuities within
the practice of computing research. We now expect our computing tools to be
driven by an architecture of open participation and democracy that encourages
users to add value to their tools and applications as they use them. Similarly,
the bar for enabling the design of novel, personal computing systems and
"hardware remixes" has fallen to the point where many non-experts and novices
are readily embracing and creating fascinating and ingenious computing
artifacts outside of our official and traditionally sanctioned academic
research communities.
But how have we as "expert" practitioners been influencing this discussion? By constructing a practice around the design and development of technology for task based and problem solving applications, we have unintentionally established such work as the status quo for the human computing experience. We have failed in our duty to open up alternate forums for technology to express itself and touch our lives beyond productivity and efficiency. Blinded by our quest for "smart technologies" we have forgotten to contemplate the design of technologies to inspire us to be smarter, more curious, and more inquisitive. We owe it to ourselves to rethink the impact we desire to have on this historic moment in computing culture. We must choose to participate in and perhaps lead a dialogue that heralds an expansive new acceptable practice of designing to enable participation by experts and non-experts alike. We are in the milieu of the rise of the "expert amateur". We must change our mantra: "not just usability but usefulness and relevancy to our world, its citizens, and our environment". We must design for the world and what matters. This means discussing our computing research alongside new keywords such as the economy, the environment, activism, poverty, healthcare, famine, homelessness, literacy, religion, and politics. This talk will explore the design territory and potential opportunities for all of us to collaborate and benefit as a society from this cultural movement. Keywords: DIY culture, citizen science | |||
| Sikuli: using GUI screenshots for search and automation | | BIBAK | Full-Text | 183-192 | |
| Tom Yeh; Tsung-Hsiang Chang; Robert C. Miller | |||
| We present Sikuli, a visual approach to search and automation of graphical
user interfaces using screenshots. Sikuli allows users to take a screenshot of
a GUI element (such as a toolbar button, icon, or dialog box) and query a help
system using the screenshot instead of the element's name. Sikuli also provides
a visual scripting API for automating GUI interactions, using screenshot
patterns to direct mouse and keyboard events. We report a web-based user study
showing that searching by screenshot is easy to learn and faster to specify
than keywords. We also demonstrate several automation tasks suitable for visual
scripting, such as map navigation and bus tracking, and show how visual
scripting can improve interactive help systems previously proposed in the
literature. Keywords: automation, image search, online help | |||
| CommunityCommands: command recommendations for software applications | | BIBAK | Full-Text | 193-202 | |
| Justin Matejka; Wei Li; Tovi Grossman; George Fitzmaurice | |||
| We explore the use of modern recommender system technology to address the
problem of learning software applications. Before describing our new command
recommender system, we first define relevant design considerations. We then
discuss a 3 month user study we conducted with professional users to evaluate
our algorithms which generated customized recommendations for each user.
Analysis shows that our item-based collaborative filtering algorithm generates
2.1 times as many good suggestions as existing techniques. In addition we
present a prototype user interface to ambiently present command recommendations
to users, which has received promising initial user feedback. Keywords: recommender | |||
| Mining web interactions to automatically create mash-ups | | BIBAK | Full-Text | 203-212 | |
| Jeffrey P. Bigham; Ryan S. Kaminsky; Jeffrey Nichols | |||
| The deep web contains an order of magnitude more information than the
surface web, but that information is hidden behind the web forms of a large
number of web sites. Metasearch engines can help users explore this information
by aggregating results from multiple resources, but previously these could only
be created and maintained by programmers. In this paper, we explore the
automatic creation of metasearch mash-ups by mining the web interactions of
multiple web users to find relations between query forms on different web
sites. We also present an implemented system called TX2 that uses those
connections to search multiple deep web resources simultaneously and integrate
the results in context in a single results page. TX2 illustrates the promise of
constructing mash-ups automatically and the potential of mining web
interactions to explore deep web resources. Keywords: deep web, mash-ups, meta-search, programming-by-example, web forms | |||
| Relaxed selection techniques for querying time-series graphs | | BIBAK | Full-Text | 213-222 | |
| Christian Holz; Steven Feiner | |||
| Time-series graphs are often used to visualize phenomena that change over
time. Common tasks include comparing values at different points in time and
searching for specified patterns, either exact or approximate. However, tools
that support time-series graphs typically separate query specification from the
actual search process, allowing users to adapt the level of similarity only
after specifying the pattern. We introduce relaxed selection techniques, in
which users implicitly define a level of similarity that can vary across the
search pattern, while creating a search query with a single-gesture
interaction. Users sketch over part of the graph, establishing the level of
similarity through either spatial deviations from the graph, or the speed at
which they sketch (temporal deviations). In a user study, participants were
significantly faster when using our temporally relaxed selection technique than
when using traditional techniques. In addition, they achieved significantly
higher precision and recall with our spatially relaxed selection technique
compared to traditional techniques. Keywords: similarity queries, time-series data | |||
| Integrated videos and maps for driving directions | | BIBAK | Full-Text | 223-232 | |
| Billy Chen; Boris Neubert; Eyal Ofek; Oliver Deussen; Michael F. Cohen | |||
| While onboard navigation systems are gaining in importance, maps are still
the medium of choice for laying out a route to a destination and for way
finding. However, even with a map, one is almost always more comfortable
navigating a route the second time due to the visual memory of the route. To
make the first time navigating a route feel more familiar, we present a system
that integrates a map with a video automatically constructed from panoramic
imagery captured at close intervals along the route. The routing information is
used to create a variable speed video depicting the route. During playback of
the video, the frame and field of view are dynamically modulated to highlight
salient features along the route and connect them back to the map. A user
interface is demonstrated to allow exploration of the combined map, video, and
textual driving directions. We discuss the construction of the hybrid map and
video interface. Finally, we report the results of a study that provides
evidence of the effectiveness of such a system for route following. Keywords: driving directions, map, routes, video | |||
| Perceptual interpretation of ink annotations on line charts | | BIBAK | Full-Text | 233-236 | |
| Nicholas Kong; Maneesh Agrawala | |||
| Asynchronous collaborators often use freeform ink annotations to point to
visually salient perceptual features of line charts such as peaks or humps,
valleys, rising slopes and declining slopes. We present a set of techniques for
interpreting such annotations to algorithmically identify the corresponding
perceptual parts. Our approach is to first apply a parts-based segmentation
algorithm that identifies the visually salient perceptual parts in the chart.
Our system then analyzes the freeform annotations to infer the corresponding
peaks, valleys or sloping segments. Once the system has identified the
perceptual parts it can highlight them to draw further attention and reduce
ambiguity of interpretation in asynchronous collaborative discussions. Keywords: deixis, freeform ink annotation, line charts, pointing, selection, visual
perception, visualization | |||
| Changing how people view changes on the web | | BIBAK | Full-Text | 237-246 | |
| Jaime Teevan; Susan T. Dumais; Daniel J. Liebling; Richard L. Hughes | |||
| The Web is a dynamic information environment. Web content changes regularly
and people revisit Web pages frequently. But the tools used to access the Web,
including browsers and search engines, do little to explicitly support these
dynamics. In this paper we present DiffIE, a browser plug-in that makes content
change explicit in a simple and lightweight manner. DiffIE caches the pages a
person visits and highlights how those pages have changed when the person
returns to them. We describe how we built a stable, reliable, and usable
system, including how we created compact, privacy-preserving page
representations to support fast difference detection. Via a longitudinal user
study, we explore how DiffIE changed the way people dealt with changing
content. We find that much of its benefit came not from exposing expected
change, but rather from drawing attention to unexpected change and helping
people build a richer understanding of the Web content they frequent. Keywords: change, dynamic information, re-finding, revisitation, web browsing, web
dynamics | |||
| Overview based example selection in end user interactive concept learning | | BIBAK | Full-Text | 247-256 | |
| Saleema Amershi; James Fogarty; Ashish Kapoor; Desney Tan | |||
| Interaction with large unstructured datasets is difficult because existing
approaches, such as keyword search, are not always suited to describing
concepts corresponding to the distinctions people want to make within datasets.
One possible solution is to allow end users to train machine learning systems
to identify desired concepts, a strategy known as interactive concept learning.
A fundamental challenge is to design systems that preserve end user flexibility
and control while also guiding them to provide examples that allow the machine
learning system to effectively learn the desired concept. This paper presents
our design and evaluation of four new overview based approaches to guiding
example selection. We situate our explorations within CueFlik, a system
examining end user interactive concept learning in Web image search. Our
evaluation shows our approaches not only guide end users to select better
training examples than the best performing previous design for this
application, but also reduce the impact of not knowing when to stop training
the system. We discuss challenges for end user interactive concept learning
systems and identify opportunities for future research on the effective design
of such systems. Keywords: end-user interactive concept learning | |||
| The web page as a WYSIWYG end-user customizable database-backed information management application | | BIBAK | Full-Text | 257-260 | |
| David R. Karger; Scott Ostler; Ryan Lee | |||
| Dido is an application (and application development environment) in a web
page. It is a single web page containing rich structured data, an AJAXy
interactive visualizer/editor for that data, and a "metaeditor" for WYSIWYG
editing of the visualizer/editor. Historically, users have been limited to the
data schemas, visualizations, and interactions offered by a small number of
heavyweight applications. In contrast, Dido encourages and enables the end user
to edit (not code) in his or her web browser a distinct ephemeral interaction
"wrapper" for each data collection that is specifically suited to its intended
use. Dido's active document metaphor has been explored before but we show how,
given today's web infrastructure, it can be deployed in a small self-contained
HTML document without touching a web client or server. Keywords: databases, end user programming, wysiwyg | |||