Setwise Comparison: Consistent, Scalable, Continuum Labels for Computer
Vision
Mining Human Behaviors
/
Sarkar, Advait
/
Morrison, Cecily
/
Dorn, Jonas F.
/
Bedi, Rishi
/
Steinheimer, Saskia
/
Boisvert, Jacques
/
Burggraaff, Jessica
/
D'Souza, Marcus
/
Kontschieder, Peter
/
Bulò, Samuel Rota
/
Walsh, Lorcan
/
Kamm, Christian P.
/
Zaykov, Yordan
/
Sellen, Abigail
/
Lindley, Siân
Proceedings of the ACM CHI'16 Conference on Human Factors in Computing
Systems
2016-05-07
v.1
p.261-271
© Copyright 2016 ACM
Summary: A growing number of domains, including affect recognition and movement
analysis, require a single, real number ground truth label capturing some
property of a video clip. We term this the provision of continuum labels.
Unfortunately, there is often an uncacceptable trade-off between label
consistency and the efficiency of the labelling process with current tools. We
present a novel interaction technique, setwise comparison, which leverages the
intrinsic human capability for consistent relative judgements and the TrueSkill
algorithm to solve this problem. We describe SorTable, a system demonstrating
this technique. We conducted a real-world study where clinicians labelled
videos of patients with multiple sclerosis for the ASSESS MS computer vision
system. In assessing the efficiency-consistency trade-off of setwise versus
pairwise comparison, we demonstrated that not only is setwise comparison more
efficient, but it also elicits more consistent labels. We further consider how
our findings relate to the interactive machine learning literature.
Collaborating with computer vision systems: an exploration of audio feedback
Sound
/
Morrison, Cecily
/
Smyth, Neil
/
Corish, Robert
/
O'Hara, Kenton
/
Sellen, Abigail
Proceedings of DIS'14: Designing Interactive Systems
2014-06-21
v.1
p.229-238
© Copyright 2014 ACM
Summary: Computer visions (CV) systems are increasingly finding new roles in domains
such as healthcare. These collaborative settings are a new challenge for CV
systems, requiring the design of appropriate interaction paradigms. The
provision of feedback, particularly of what the CV system can 'see,' is a key
aspect, and may not always be possible to present visually. We explore the
design space for audio feedback for a scenario of interest, the clinical
assessment of Multiple Sclerosis using a CV system. We then present a
mixed-methods experimental study aimed at providing some first insights into
the challenges and opportunities of designing audio feedback of this kind.
Specifically, we compare audio feedback that differentiates which body parts
the CV system can see to audio feedback that is undifferentiated. The findings
reveal that it is not enough to simply convey that something might be out of
view of the camera as what the camera can 'see' depends on the specific
configuration of participants and the peculiarities of the skeleton inference
algorithms. The results highlight the importance of providing feedback which
more naturally conveys spatial information in developing CV systems for
collaborative use.
Place-onas: shared resource for designing body tracking applications
Works-in-progress
/
Morrison, Cecily
/
Corish, Robert
/
Sellen, Abigail J.
Proceedings of ACM CHI 2014 Conference on Human Factors in Computing Systems
2014-04-26
v.2
p.1861-1866
© Copyright 2014 ACM
Summary: Developments in computer vision technology have led to a plethora of new
body tracking applications. These applications share a challenge in accounting
for characteristics of the specific places in which they are intended to be
used. We present the concept of Place-onas, representations of "typical"
places, as a shared resource to support multidisciplinary team discussions
during the development of body tracking applications. We present an example
Place-ona drawn from ASSESS MS, a computer vision application that supports the
clinical assessment of Multiple Sclerosis. We describe its usage, drawing out
how it supported design work, and conclude with a discussion of future work.
Towards Understanding How Humans Teach Robots
Short Research Papers
/
Kaochar, Tasneem
/
Peralta, Raquel Torres
/
Morrison, Clayton T.
/
Fasel, Ian R.
/
Walsh, Thomas J.
/
Cohen, Paul R.
Proceedings of the 2011 Conference on User Modeling, Adaptation and
Personalization
2011-07-11
p.347-352
© Copyright 2011 Springer-Verlag
Summary: Our goal is to develop methods for non-experts to teach complex behaviors to
autonomous agents (such as robots) by accommodating "natural" forms of human
teaching. We built a prototype interface allowing humans to teach a simulated
robot a complex task using several techniques and report the results of 44
human participants using this interface. We found that teaching styles varied
considerably but can be roughly categorized based on the types of interaction,
patterns of testing, and general organization of the lessons given by the
teacher. Our study contributes to a better understanding of human teaching
patterns and makes specific recommendations for future human-robot interaction
systems.
Hospital user research using new media arts
/
Morrison, Cecily
/
Blackwell, Alan F.
Proceedings of the HCI'09 Conference on People and Computers XXIII
2009-09-01
p.345-353
Keywords: intensive care, mobile devices, new media arts
© Copyright 2009 Author
Summary: This paper presents a comparative analysis of group interaction around two
display types, shared and individual, using a 'new media' arts application as a
way to explore the physical technology setup for an intensive care unit in a
hospital. We propose this method for laboratory settings when the research
questions derive from socially complex environments, but real-world
interventions are not possible. While users solve an 'interaction problem' that
is posed through the 'new media' arts application for their own expressive
purposes, researchers can analyse and collate the results to understand the
solution space. We present a study with the bodyPaint application to address a
design issue that we discovered when assessing the merits of an electronic
patient record system.
Intentional affect: an alternative notion of affective interaction with a
machine
/
Afzal, Shazia
/
Morrison, Cecily
/
Robinson, Peter
Proceedings of the HCI'09 Conference on People and Computers XXIII
2009-09-01
p.370-374
Keywords: affective computing, computer-assisted learning, emotion recognition
© Copyright 2009 Author
Summary: Affective Computing envisages truly effective human-machine interactions as
being affect-sensitive. The field is both motivated and influenced by an
understanding of emotion in an environment, that of person to person, that
differs from its eventual application, person to machine. Analysing data
obtained in a potential application environment -- computer-assisted learning
-- we highlight the limitations of such an understanding and propose an
alternative stance to affect, that of intentional affective interaction.
Co-located group interaction design
Design theater
/
Morrison, Cecily
/
Blackwell, Alan F.
Proceedings of ACM CHI 2008 Conference on Human Factors in Computing Systems
2008-04-05
v.2
p.2587-2590
© Copyright 2008 ACM
Summary: This design theatre experience explores the use of choreographic
improvisation exercises to reflect on the structures of interaction in a
mobile, co-located group. The design technique is motivated by studies of
clinical ward rounds, and applies analytic models from Kendon, Garfinkel and
Hutchins.
Interaction Manifolds: Theory from Experiments
Posters
/
Morrison, Cecily
/
Blackwell, Alan
Proceedings of the HCI'07 Conference on People and Computers XXI
2007-09-03
v.2
p.43
Summary: This poster builds on comparative ethnographic work of a multi-disciplinary
medical team using a paper-based and a computer-based patient record system. It
describes the design and preliminary results of an experiment aimed to help
articulate an analytical construct that would describe the tradeoffs between a
technological setup and a group's ability to negotiate an interaction among
themselves.
Interaction Manifolds: Understanding Behaviour Around a Shareable Interface
Doctoral Consortium
/
Morrison, Cecily
Proceedings of the HCI'07 Conference on People and Computers XXI
2007-09-03
v.2
p.74
Summary: This poster presents a suggestion for how ethnography of shareable
interfaces might be used to inform subsequent design decisions by creating an
analytical construct that we name the interaction manifold. We first describe
and summarize results of our research to date in a medical intensive care unit.
We then propose why and how an analytic construct might prove useful for design
of shareable interfaces.