[1]
Retargeting Technical Documentation to Augmented Reality
Augmented & Virtual Reality in the Real World
/
Mohr, Peter
/
Kerbl, Bernhard
/
Donoser, Michael
/
Schmalstieg, Dieter
/
Kalkofen, Denis
Proceedings of the ACM CHI'15 Conference on Human Factors in Computing
Systems
2015-04-18
v.1
p.3337-3346
© Copyright 2015 ACM
Summary: We present a system which automatically transfers printed technical
documentation, such as handbooks, to three-dimensional Augmented Reality. Our
system identifies the most frequent forms of instructions found in printed
documentation, such as image sequences, explosion diagrams, textual annotations
and arrows indicating motion. The analysis of the printed documentation works
automatically, with minimal user input. The system only requires the
documentation itself and a CAD model or 3D scan of the object described in the
documentation. The output is a fully interactive Augmented Reality application,
presenting the information from the printed documentation in 3D, registered to
the real object.
[2]
MultiFi: Multi Fidelity Interaction with Displays On and Around the Body
Multi-Device Interaction
/
Grubert, Jens
/
Heinisch, Matthias
/
Quigley, Aaron
/
Schmalstieg, Dieter
Proceedings of the ACM CHI'15 Conference on Human Factors in Computing
Systems
2015-04-18
v.1
p.3933-3942
© Copyright 2015 ACM
Summary: Display devices on and around the body such as smartwatches, head-mounted
displays or tablets enable users to interact on the go. However, diverging
input and output fidelities of these devices can lead to interaction seams that
can inhibit efficient mobile interaction, when users employ multiple devices at
once. We present MultiFi, an interactive system that combines the strengths of
multiple displays and overcomes the seams of mobile interaction with widgets
distributed over multiple devices. A comparative user study indicates that
combined head-mounted display and smartwatch interfaces can outperform
interaction with single wearable devices.
[3]
Show me the invisible: visualizing hidden content
Desktop search and history
/
Geymayer, Thomas
/
Steinberger, Markus
/
Lex, Alexander
/
Streit, Marc
/
Schmalstieg, Dieter
Proceedings of ACM CHI 2014 Conference on Human Factors in Computing Systems
2014-04-26
v.1
p.3705-3714
© Copyright 2014 ACM
Summary: Content on computer screens is often inaccessible to users because it is
hidden, e.g., occluded by other windows, outside the viewport, or overlooked.
In search tasks, the efficient retrieval of sought content is important.
Current software, however, only provides limited support to visualize hidden
occurrences and rarely supports search synchronization crossing application
boundaries. To remedy this situation, we introduce two novel visualization
methods to guide users to hidden content. Our first method generates awareness
for occluded or out-of-viewport content using see-through visualization. For
content that is either outside the screen's viewport or for data sources not
opened at all, our second method shows off-screen indicators and an on-demand
smart preview. To reduce the chances of overlooking content, we use visual
links, i.e., visible edges, to connect the visible content or the visible
representations of the hidden content. We show the validity of our methods in a
user study, which demonstrates that our technique enables a faster localization
of hidden content compared to traditional search functionality and thereby
assists users in information retrieval tasks.
[4]
Audio stickies: visually-guided spatial audio annotations on a mobile
augmented reality platform
Audio and speech
/
Langlotz, Tobias
/
Regenbrecht, Holger
/
Zollmann, Stefanie
/
Schmalstieg, Dieter
Proceedings of the 2013 Australian Computer-Human Interaction Conference
2013-11-25
2013-11-25
p.545-554
© Copyright 2013 ACM
Summary: This paper describes spatially aligned user-generated audio annotations and
the integration with visual augmentations into a single mobile AR system.
Details of our prototype system are presented, along with an explorative
usability study and technical evaluation of the design. Mobile Augmented
Reality applications allow for visual augmentations as well as tagging and
annotation of the surrounding environment. Texts and graphics are currently the
media of choice for these applications with GPS coordinates used to determine
spatial location. Our research demonstrates that the use of visually guided
audio annotations that are positioned and orientated in augmented outdoor space
successfully provides for additional, novel, and enhanced mobile user
experience.
[5]
Playing it real again: a repeated evaluation of magic lens and static
peephole interfaces in public space
Select and interact
/
Grubert, Jens
/
Schmalstieg, Dieter
Proceedings of 2013 Conference on Human-computer interaction with mobile
devices and services
2013-08-27
2013-08-27
p.99-102
© Copyright 2013 ACM
Summary: We repeated a study on the usage of a magic lens and a static peephole
interface for playing a find-and-select game in a public space. While we
reproduced the study setup and procedure the task was conducted in a public
transportation stop with different characteristics. The results on usage
duration and user preference were significantly different from those reported
for previous conditions. We investigate possible causes, specifically the
differences in the spatial characteristics and the social contexts in which the
study took place.
[6]
OmniKinect: real-time dense volumetric data acquisition and applications
Apply yourself (systems track: applications)
/
Kainz, Bernhard
/
Hauswiesner, Stefan
/
Reitmayr, Gerhard
/
Steinberger, Markus
/
Grasset, Raphael
/
Gruber, Lukas
/
Veas, Eduardo
/
Kalkofen, Denis
/
Seichter, Hartmut
/
Schmalstieg, Dieter
Proceedings of the 2012 ACM Symposium on Virtual Reality Software and
Technology
2012-12-10
p.25-32
© Copyright 2012 ACM
Summary: Real-time three-dimensional acquisition of real-world scenes has many
important applications in computer graphics, computer vision and human-computer
interaction. Inexpensive depth sensors such as the Microsoft Kinect allow to
leverage the development of such applications. However, this technology is
still relatively recent, and no detailed studies on its scalability to dense
and view-independent acquisition have been reported. This paper addresses the
question of what can be done with a larger number of Kinects used
simultaneously. We describe an interference-reducing physical setup, a
calibration procedure and an extension to the KinectFusion algorithm, which
allows to produce high quality volumetric reconstructions from multiple Kinects
whilst overcoming systematic errors in the depth measurements. We also report
on enhancing image based visual hull rendering by depth measurements, and
compare the results to KinectFusion. Our system provides practical insight into
achievable spatial and radial range and into bandwidth requirements for depth
data acquisition. Finally, we present a number of practical applications of our
system.
[7]
Indoor navigation with mixed reality world-in-miniature views and sparse
localization on mobile devices
Mobile interfaces
/
Mulloni, Alessandro
/
Seichter, Hartmut
/
Schmalstieg, Dieter
Proceedings of the 2012 International Conference on Advanced Visual
Interfaces
2012-05-22
p.212-215
© Copyright 2012 ACM
Summary: We present the design of an interface that provides continuous navigational
support for indoor scenarios where localization is only available at sparse,
discrete locations (info points). Our interface combines turn-by-turn
instructions with a World-in-Miniature (WIM). In a previous study, we showed
that using an Augmented Reality WIM at info points, and turn-by-turn
instructions elsewhere, is a valid support for navigation inside an unknown
building. In particular, we highlighted that users value the WIM as a tool for
monitoring their location in the building. In this work, we focus on using the
WIM continuously, not only at info points, to support navigation. We adapt the
WIM views to the quality of localization by transitioning within Mixed Reality:
we use Augmented Reality to provide an overview of the whole path at info
points and Virtual Reality to communicate the next instruction when
localization is not available. Our results from a new user study validate our
interface design and show that users exploit not only turn-by-turn instructions
but also the WIM throughout the path, to navigate with our interface. This
paper provides insight on how a low-infrastructure indoor solution can support
human navigational abilities effectively.
[8]
360° panoramic overviews for location-based services
Right where i am: UX in complex environments
/
Mulloni, Alessandro
/
Seichter, Hartmut
/
Dünser, Andreas
/
Baudisch, Patrick
/
Schmalstieg, Dieter
Proceedings of ACM CHI 2012 Conference on Human Factors in Computing Systems
2012-05-05
v.1
p.2565-2568
© Copyright 2012 ACM
Summary: We investigate 360° panoramas as overviews to support users in the task
of locating objects in the surrounding environment. Panoramas are typically
visualized as rectangular photographs, but this does not provide clear cues for
physical directions in the environment. In this paper, we conduct a series of
studies with three different shapes: Frontal, Top-Down and Bird's Eye; the last
two shapes are chosen because they provide a clearer representation of the
spatial mapping between panorama and environment. Our results show that good
readability of the panorama is most important and that a clear representation
of the spatial mapping plays a secondary role. This paper is the first to
provide understanding on how users exploit 360° panoramic over-views to
locate objects in the surrounding environment and how different design factors
can affect user performance.
[9]
Display-adaptive window management for irregular surfaces
Multi-surface
/
Waldner, Manuela
/
Grasset, Raphael
/
Steinberger, Markus
/
Schmalstieg, Dieter
Proceedings of the 2011 ACM International Conference on Interactive
Tabletops and Surfaces
2011-11-13
p.222-231
© Copyright 2011 ACM
Summary: Current projectors can easily be combined to create an everywhere display,
using all suitable surfaces in offices or meeting rooms for the presentation of
information. However, the resulting irregular display is not well supported by
traditional desktop window managers, which are optimized for rectangular
screens. In this paper, we present novel display-adaptive window management
techniques, which provide semi-automatic placement for desktop elements (such
as windows or icons) for users of large, irregularly shaped displays. We report
results from an exploratory study, which reveals interesting emerging
strategies of users in the manipulation of windows on large irregular displays
and shows that the new techniques increase subjective satisfaction with the
window management interface.
[10]
Handheld augmented reality indoor navigation with activity-based
instructions
Indoor navigation
/
Mulloni, Alessandro
/
Seichter, Hartmut
/
Schmalstieg, Dieter
Proceedings of the 13th Conference on Human-computer interaction with mobile
devices and services
2011-08-30
p.211-220
© Copyright 2011 ACM
Summary: We present a novel design of an augmented reality interface to support
indoor navigation. We combine activity-based instructions with sparse 3D
localisation at selected info points in the building. Based on localisation
accuracy and the users' activities, such as walking or standing still, the
interface adapts the visualisation by changing the density and quality of
information shown. We refine and validate our design through user involvement
in pilot studies. We finally present the results of a comparative study
conducted to validate the effectiveness of our design and to explore how the
presence of info points affects users' performance on indoor navigation tasks.
The results of this study validate our design and show an improvement in task
performance when info points are present, which act as confirmation points and
provide an overview of the task.
[11]
Importance-driven compositing window management
Predicting & modeling human behaviors
/
Waldner, Manuela
/
Steinberger, Markus
/
Grasset, Raphael
/
Schmalstieg, Dieter
Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems
2011-05-07
v.1
p.959-968
© Copyright 2011 ACM
Summary: In this paper we present importance-driven compositing window management,
which considers windows not only as basic rectangular shapes but also
integrates the importance of the windows' content using a bottom-up visual
attention model. Based on this information, importance-driven compositing
optimizes the spatial window layout for maximum visibility and interactivity of
occluded content in combination with see-through windows. We employ this
technique for emerging window manager functions to minimize information overlap
caused by popping up windows or floating toolbars and to improve the access to
occluded window content. An initial user study indicates that our technique
provides a more effective and satisfactory access to occluded information than
the well-adopted Alt+Tab window switching technique and see-through windows
without optimized spatial layout.
[12]
Directing attention and influencing memory with visual saliency modulation
Visualization & perception
/
Veas, Eduardo E.
/
Mendez, Erick
/
Feiner, Steven K.
/
Schmalstieg, Dieter
Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems
2011-05-07
v.1
p.1471-1480
© Copyright 2011 ACM
Summary: In augmented reality, it is often necessary to draw the user's attention to
particular objects in the real world without distracting her from her task. We
explore the effectiveness of directing a user's attention by imperceptibly
modifying existing features of a video. We present three user studies of the
effects of applying a saliency modulation technique to video; evaluating
modulation awareness, attention, and memory. Our results validate the saliency
modulation technique as an alternative means to convey information to the user,
suggesting attention shifts and influencing recall of selected regions without
perceptible changes to visual input.
[13]
Bridging gaps with pointer warping in multi-display environments
Short papers
/
Waldner, Manuela
/
Kruijff, Ernst
/
Schmalstieg, Dieter
Proceedings of the Sixth Nordic Conference on Human-Computer Interaction
2010-10-16
p.813-816
Keywords: multi-display environments, pointer warping
© Copyright 2010 ACM
Summary: Pointer warping can be an effective alternative to relocate the mouse
pointer to a remote display in multi-display environments. It minimizes the
mouse pointer travel and does not require the user to search for a path to the
target display. However, little is known about the factors that influence the
performance of pointer warping. In this paper we explore the characteristics of
pointer warping compared to standard mouse behavior on a dual-monitor setup
with varying physical distance. Our results show that the performance of
pointer warping is hardly affected by the distance of the pointer warp, but is
influenced by the direction of the warp.
[14]
Zooming interfaces for augmented reality browsers
Visualization and interaction techniques
/
Alessandro, Mulloni
/
Dünser, Andreas
/
Schmalstieg, Dieter
Proceedings of 12th Conference on Human-computer interaction with mobile
devices and services
2010-09-07
p.161-170
Keywords: mobile augmented reality, zooming interfaces
© Copyright 2010 ACM
Summary: Augmented Reality combines real world and virtual information in interactive
visualizations. Since phones started integrating GPS, compass and
accelerometer, several Augmented Reality browsers for phones have hit the
market. These are applications that access large amounts of geo-referenced
information from online sources and present it at corresponding physical
locations, superimposed onto a live video stream. However, Augmented Reality is
constrained by the camera's field of view and restricted to first-person views,
limiting the amount of overview that users can gain. We present two zooming
interfaces that compensate for these constraints by enabling users to smoothly
zoom between the Augmented Reality view and (1) an egocentric panoramic view of
360°, and (2) an exocentric top-down view. We present the results from two
studies that show how in most search tasks our zooming interfaces are faster
and require less panning than an overlay-based tool, scaling better as the
amount of information grows.
[15]
Visual links across applications
Best student papers
/
Waldner, Manuela
/
Puff, Werner
/
Lex, Alexander
/
Streit, Marc
/
Schmalstieg, Dieter
Proceedings of the 2010 Conference on Graphics Interface
2010-05-31
p.129-136
© Copyright 2010 Authors
Summary: The tasks carried out by modern information workers become increasingly
complex and time-consuming. They often require to evaluate, interpret, and
compare information from different sources presented in multiple application
windows. With large, high resolution displays, multiple application windows can
be arranged in a way so that a large amount of information is visible
simultaneously. However, individual application windows' contents and visual
representations are isolated and relations between information items contained
in these windows are not explicit. Thus, relating and comparing information
across applications has to be executed manually by the user, which is a tedious
and error-prone task.
In this paper we present visual links connecting related pieces of
information across application windows and thereby guiding the user's attention
to relevant information. Applications are coordinated by a management
application accessible via a light-weight interface. User selections are
synchronized across registered applications and visual links are rendered on
top of the desktop content by a window manager. Initial user feedback was very
positive and indicates that visual links improve task efficiency when analyzing
information from multiple sources.
[16]
Techniques for view transition in multi-camera outdoor environments
Virtual and augmented reality
/
Veas, Eduardo
/
Mulloni, Alessandro
/
Kruijff, Ernst
/
Regenbrecht, Holger
/
Schmalstieg, Dieter
Proceedings of the 2010 Conference on Graphics Interface
2010-05-31
p.193-200
© Copyright 2010 Authors
Summary: Environment monitoring using multiple observation cameras is increasingly
popular. Different techniques exist to visualize the incoming video streams,
but only few evaluations are available to find the best suitable one for a
given task and context. This article compares three techniques for browsing
video feeds from cameras that are located around the user in an unstructured
manner. The techniques allow mobile users to gain extra information about the
surroundings, the objects and the actors in the environment by observing a site
from different perspectives. The techniques relate local and remote cameras
topologically, via a tunnel, or via bird's eye viewpoint. Their common goal is
to enhance spatial awareness of the viewer, without relying on a model or
previous knowledge of the environment. We introduce several factors of spatial
awareness inherent to multi-camera systems, and present a comparative
evaluation of the proposed techniques with respect to spatial understanding and
workload.
[17]
Bimanual handheld mixed reality interfaces for urban planning
Interfaces to go
/
Sareika, Markus
/
Schmalstieg, Dieter
Proceedings of the 2010 International Conference on Advanced Visual
Interfaces
2010-05-26
p.189-196
Keywords: 3D interaction, Urban planning, architecture, augmented reality, bimanual
interaction, design, mixed reality
© Copyright 2010 ACM
Summary: Tabletop models are common in architectural and urban planning tasks. We
report here on an investigation for view navigation in and manipulation of
tracked tabletop models using a handheld Mixed Reality interface targeted at a
user group with varying professional background and skill level. Users were
asked to complete three basic task types: searching, inserting and creating
content in a mixed reality scene, each requiring the user to navigate in the
scene while interacting. This study was designed to naturally progress on
classic problems like travel, selection and manipulation in an applied scenario
concerned with urban planning. The novel bimanual interface configurations
utilize a handheld touch screen display for Mixed Reality, with the
camera/viewpoint attached or handheld separately. Usability aspects and user
satisfaction are scrutinized by a user study, aimed at optimizing usability and
supporting the user's intentions in a natural way. We present the results from
the user study showing significant differences in task completion times as well
as user preferences and practical issues concerning both interface and view
navigation design.
[18]
Experiences with mouse control in multi-display environments
Workshops
/
Waldner, Manuela
/
Schmalstieg, Dieter
Proceedings of the 2010 International Conference on Advanced Visual
Interfaces
2010-05-26
p.411
© Copyright 2010 ACM
Summary: It is now increasingly common to extend private workstations with large
public displays into a shared multi-display environment (MDE). Mouse-based
interaction across multiple displays provides a convenient way to quickly shift
between private work on the personal monitor and tightly coupled collaboration
on shared display spaces (Figure 1). However, mouse pointer navigation can be
negatively influenced by display factors in the environment and thereby limit
fluid interaction across displays. We report findings from an experiment
comparing four mouse pointer navigation techniques in a heterogeneous MDE.
[19]
Gaze-directed ubiquitous interaction using a Brain-Computer Interface
/
Schmalstieg, Dieter
/
Bornik, Alexander
/
Müller-Putz, Gernot
/
Pfurtscheller, Gert
Proceedings of the 2010 Augmented Human International Conference
2010-04-02
p.4
Keywords: authentication, biometrics, brain computer interface, electroencephalogram,
gaze tracking, object selection
© Copyright 2010 ACM
Summary: n this paper, we present a first proof-of-concept for using a mobile
Brain-Computer Interface (BCI) coupled to a wearable computer as an ambient
input device for a ubiquitous computing service. BCI devices, such as
electroencephalogram (EEG) based BCI, can be used as a novel form of
human-computer interaction device. A user can log into a nearby computer
terminal by looking at its screen. This feature is enabled by detecting a
user's gaze through the analysis of the brain's response to visually evoked
patterns. We present the experimental setup and discuss opportunities and
limitations of the technique.
[20]
Automatic configuration of spatially consistent mouse pointer navigation in
multi-display environments
Short paper presentations (posters)
/
Waldner, Manuela
/
Pirchheim, Christian
/
Kruijff, Ernst
/
Schmalstieg, Dieter
Proceedings of the 2010 International Conference on Intelligent User
Interfaces
2010-02-07
p.397-400
Keywords: cross-display mouse navigation, multi-display environment
© Copyright 2010 ACM
Summary: Multi-display environments combine displays of various form factors into a
common interaction space. Cross-display navigation techniques have to provide
transitions to move the mouse pointer across display boundaries to reach
distant display locations. A spatially consistent description of display
relationships thereby supports fluid cross-display navigation. In this paper,
we present two spatially consistent navigation techniques for seamless
cross-display navigation in multi-user multi-display environments. These
navigation techniques are automatically configured from a spatial model of the
environment, which is generated in a camera-assisted calibration step. We
describe the implementation in a distributed system and present results of a
comparative experiment.
[21]
Importance masks for revealing occluded objects in augmented reality
Poster abstracts
/
Mendez, Erick
/
Schmalstieg, Dieter
Proceedings of the 2009 ACM Symposium on Virtual Reality Software and
Technology
2009-11-18
p.247-248
Keywords: augmented reality, focus and context, importance masks, x-ray vision
© Copyright 2009 ACM
Summary: When simulating "X-ray vision" in Augmented Reality, a critical aspect is
ensuring correct perception of the occluded objects position. Naïve
overlay rendering of occluded objects on top of real-world occluders can lead
to a misunderstanding of the visual scene and a poor perception of the depth.
We present a simple technique to enhance the perception of the spatial
arrangements in the scene. An importance mask associated with occluders informs
the rendering what information can be overlaid and what should be preserved.
This technique is independent of scene properties such as illumination and
surface properties, which may be unknown. The proposed solution is computed
efficiently in a single-pass fragment shaders on the GPU.
[22]
Interactive editing of segmented volumetric datasets in a hybrid 2D/3D
virtual environment
Interaction
/
Bornik, Alexander
/
Beichel, Reinhard
/
Schmalstieg, Dieter
Proceedings of the 2006 ACM Symposium on Virtual Reality Software and
Technology
2006-11-01
p.197-206
Keywords: 3D user interfaces, hybrid user interfaces, interactive segmentation,
segmentation refinement, virtual reality
© Copyright 2006 ACM
Summary: In this paper we present a novel system for segmentation refinement, which
allows for interactive correction of surface models generated from imperfect
automatic segmentations of arbitrary volumetric data. The proposed approach is
based on a deformable surface model allowing interactive manipulation with a
hybrid user interface consisting of an immersive stereoscopic display and a
Tablet PC. The user interface features visualization methods and manipulation
tools specifically designed for quick inspection and correction of typical
defects resulting from automated segmentation of medical datasets. A number of
experiments show that typical segmentation problems can be fixed within a few
minutes using the system, while maintaining real-time responsiveness of the
system.
[23]
Extending the scene graph with a dataflow visualization system
Hardware & systems
/
Kalkusch, Michael
/
Schmalstieg, Dieter
Proceedings of the 2006 ACM Symposium on Virtual Reality Software and
Technology
2006-11-01
p.252-260
Keywords: dataflow visualization system, object hierarchies, scene graph,
visualization
© Copyright 2006 ACM
Summary: Data ow graphs are a very successful paradigm in scientific visualization,
while scene graphs are a leading approach in interactive graphics and virtual
reality. Both approaches have their distinct advantages, and both build on a
common set of basic techniques based on graph data structures. However, despite
these similarities, no unified implementation of the two paradigms exists. This
paper presents an in-depth analysis of the architectural components of dataflow
visualization and scene graphs, and derives a design that integrates both these
approaches.
The implementation of this design builds on a common software infrastructure
based on a scene graph, and extends it with virtualized dataflow, which allows
the use of the scene graph structure and traversal mechanism for dynamically
building and evaluating dataflow.
[24]
Tactylus, a pen-input device exploring audiotactile sensory binding
Haptics in VR (special session)
/
Kruijff, Ernst
/
Wesche, Gerold
/
Riege, Kai
/
Goebbels, Gernot
/
Kunstman, Martijn
/
Schmalstieg, Dieter
Proceedings of the 2006 ACM Symposium on Virtual Reality Software and
Technology
2006-11-01
p.312-315
Keywords: 3D user interfaces, audiotactile feedback, sensory, substitution
© Copyright 2006 ACM
Summary: Recent studies have shown that through a careful combination of multiple
sensory channels, so called multisensory binding effects can be achieved that
can be beneficial for collision detection and texture recognition feedback.
During the design of a new pen-input device called Tactylus, specific focus was
put on exploring multisensory effects of audiotactile cues to create a new, but
effective way to interact in virtual environments with the purpose to overcome
several of the problems noticed in current devices.
[25]
Using neuromuscular electrical stimulation for pseudo-haptic feedback
Haptics in VR (special session)
/
Kruijff, Ernst
/
Schmalstieg, Dieter
/
Beckhaus, Steffi
Proceedings of the 2006 ACM Symposium on Virtual Reality Software and
Technology
2006-11-01
p.316-319
Keywords: 3D user interfaces, biofeedback, haptic feedback, neuroelectrical
stimulation
© Copyright 2006 ACM
Summary: This paper focuses at the usage of neuromuscular electrical stimulation
(NMES) for achieving pseudo-haptic feedback. By stimulating the motor nerves,
muscular contractions can be triggered that can be matched to a haptic event.
Reflecting an initial user test, we will explain how this process can be
realized, by investigating the physiological processes involved. Relating the
triggered feedback to general haptics, its potential in future interfaces will
be identified and laid out in a development roadmap.
