[1]
Time Series Analysis of Nursing Notes for Mortality Prediction via a State
Transition Topic Model
Session 6A: Time Series and Streams
/
Jo, Yohan
/
Loghmanpour, Natasha
/
Rosé, Carolyn Penstein
Proceedings of the 2015 ACM Conference on Information and Knowledge
Management
2015-10-19
p.1171-1180
© Copyright 2015 ACM
Summary: Accurate mortality prediction is an important task in intensive care units
in order to channel prompt care to patients in the most critical condition and
to reduce nurses' alarm fatigue. Nursing notes carry valuable information in
this regard, but nothing has been reported about the effectiveness of temporal
analysis of nursing notes in mortality prediction tasks.
We propose a time series model that uncovers the temporal dynamics of
patients' underlying states from nursing notes. The effectiveness of this
information in mortality prediction is examined for mortality prediction for
five different time spans ranging from one day to one year. Our experiments
show that the model captures both patient states and their temporal dynamics
that have a strong correlation with patient mortality. The results also show
that incorporating temporal information improves performance in long-term
mortality prediction, but has no significant effect in short-term prediction.
[2]
Interactive Navigation System for the Visually Impaired with Auditory and
Haptic Cues in Crosswalks, Indoors and Urban Areas
Urban Interaction
/
Smith, Tianqi "Tenchi" Gao
/
Rose, Christopher
/
Nolen, Jeffrey "Wayne"
/
Pierce, Daniel
/
Sherman, Alexander
HCI International 2015: 17th International Conference on HCI: Posters'
Extended Abstracts, Part II
2015-08-02
v.5
p.539-545
Keywords: Visually impaired; Navigation; Global positioning system (GPS); Visual
odometry; Pedometry; iPhone application; Dedicated Short Range Communication
(DSRC) radio; Tactile belt; Inertial Motion Unit (IMU)
© Copyright 2015 Springer International Publishing Switzerland
Summary: This Federal Highway sponsored study is aimed at creating an integrated
human-computer system that the visually impaired could use to navigate through
chaotic urban areas, indoors environment, as well as complex crosswalks. The
system incorporates several redundant positioning systems in order to provide a
robust solution to way finding. The main system components include global
positioning system (GPS), visual odometry, pedometry, iPhone, Dedicated Short
Range Communication (DSRC) radios, and tactile belt. The user will wear a
laptop that contains a data processing program that collects data real-time
from the devices and provides navigational feedback to the user's iPhone app
and tactile belt.
[3]
Identifying Latent Study Habits by Mining Learner Behavior Patterns in
Massive Open Online Courses
KM Track Posters
/
Wen, Miaomiao
/
Rose, Carolyn Penstein
Proceedings of the 2014 ACM Conference on Information and Knowledge
Management
2014-11-03
p.1983-1986
© Copyright 2014 ACM
Summary: MOOCs attract diverse users with varying habits. Identifying those patterns
through clickstream analysis could enable more effective personalized support
for student information seeking and learning in that online context. We propose
a novel method to characterize types of sessions in MOOCs by mining the
habitual behaviors of students within individual sessions. We model learning
sessions as a distribution of activities and activity sequences with a topical
N-gram model. The representation offers insights into what groupings of
habitual student behaviors are associated with higher or lower success in the
course. We also investigate how context information, such as time of day or a
user's demographic information, is associated with the types of learning
sessions.
[4]
Constrained Question Recommendation in MOOCs via Submodularity
KM Track Posters
/
Yang, Diyi
/
Shang, Jingbo
/
Rosé, Carolyn Penstein
Proceedings of the 2014 ACM Conference on Information and Knowledge
Management
2014-11-03
p.1987-1990
© Copyright 2014 ACM
Summary: A recent area in which recommender systems have shown their value is in
online discussion forums and question-answer sites. Earlier work in this space
has focused on the problem of matching participants to opportunities but has
not adequately addressed the problem that in these social contexts, multiple
dimensions of constraints must be satisfied, including limitations on capacity
and minimal requirements for expertise. In this work, we propose such a
constrained question recommendation problem with load balance constraints in
discussion forums and use flow based model to generate the optimal solution. In
particular, to address the introduced computation complexity, we investigate
the concept of submodularity of the objective function and propose a specific
submodular method to give an approximated solution. We present experiments
conducted on two Massive Open Online Course (MOOC) discussion forum datasets,
and demonstrate the effectiveness and efficiency of our submodular method in
solving constrained question recommendation tasks.
[5]
Question recommendation with constraints for massive open online courses
Novel setups -- context aware
/
Yang, Diyi
/
Adamson, David
/
Rosé, Carolyn Penstein
Proceedings of the 2014 ACM Conference on Recommender Systems
2014-10-06
p.49-56
© Copyright 2014 ACM
Summary: Massive Open Online Courses (MOOCs) have experienced a recent boom in
interest. Problems students struggle with in the discussion forums, such as
difficultly in finding interesting discussion opportunities or attracting
helpers to address posted problems, provide new opportunities for recommender
systems. In contrast to traditional product recommendation, question
recommendation in discussion forums should simultaneously consider constraints
on both students and questions. These considerations include (1) Load Balancing
-- students should not be over-burdened with too many requests; and (2)
Expertise Matching -- students should not be requested to address problems they
are not capable of addressing. In this work, we formulate a novel constrained
question recommendation problem to address the above considerations. We design
a context-aware matrix factorization model to predict students' preferences
over questions, then build a max cost flow model to manage the constraints.
Experimental results conducted on three MOOC datasets demonstrate that our
method significantly outperforms baseline methods in optimizing overall forum
welfare, and in predicting which specific questions students might be
interested in.
[6]
Learning analytics and machine learning
Workshops
/
Gasevic, Dragan
/
Rose, Carolyn
/
Siemens, George
/
Wolff, Annika
/
Zdrahal, Zdenek
LAK'14: 2014 International Conference on Learning Analytics and Knowledge
2014-03-24
p.287-288
© Copyright 2014 ACM
Summary: Learning analytics (LA) as a field remains in its infancy. Many of the
techniques now prominent from practitioners have been drawn from various
fields, including HCI, statistics, computer science, and learning sciences. In
order for LA to grow and advance as a discipline, two significant challenges
must be met: 1) development of analytics methods and techniques that are native
to the LA discipline, and 2) practitioners in LA to develop algorithms and
models that reflect the social and computational dimensions of analytics. This
workshop introduces researchers in learning analytics to machine learning (ML)
and the opportunities that ML can provide in building next generation analysis
models.
[7]
Effects of social presence and social role on help-seeking and learning
Robot teachers and learners
/
Howley, Iris
/
Kanda, Takayuki
/
Hayashi, Kotaro
/
Rosé, Carolyn
Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot
Interaction
2014-03-03
p.415-422
© Copyright 2014 ACM
Summary: The unique social presence of robots can be leveraged in learning situations
to reduce student evaluation anxiety, while still providing instructional
guidance on multiple levels of communication. Furthermore, social role of the
instructor can also impact the prevalence of evaluation apprehension. In this
study, we examine how human and robot social role affects help-seeking
behaviors and learning outcomes in a one-on-one tutoring setting. Our results
show that help-seeking is a moderator of the significant relationship between
condition and learning, with the "human teacher" condition resulting in
significantly less learning (and marginally less help-seeking) than the "human
assistant" and both robot conditions.
[8]
Triggering effective social support for online groups
/
Kumar, Rohit
/
Rosé, Carolyn P.
ACM Transactions on Interactive Intelligent Systems
2014-01
v.3
n.4
p.24
© Copyright 2014 ACM
Summary: Conversational agent technology is an emerging paradigm for creating a
social environment in online groups that is conducive to effective teamwork.
Prior work has demonstrated advantages in terms of learning gains and
satisfaction scores when groups learning together online have been supported by
conversational agents that employ Balesian social strategies. This prior work
raises two important questions that are addressed in this article. The first
question is one of generality. Specifically, are the positive effects of the
designed support specific to learning contexts? Or are they in evidence in
other collaborative task domains as well? We present a study conducted within a
collaborative decision-making task where we see that the positive effects of
the Balesian social strategies extend to this new context. The second question
is whether it is possible to increase the effectiveness of the Balesian social
strategies by increasing the context sensitivity with which the social
strategies are triggered. To this end, we present technical work that increases
the sensitivity of the triggering. Next, we present a user study that
demonstrates an improvement in performance of the support agent with the new,
more sensitive triggering policy over the baseline approach from prior work.
The technical contribution of this article is that we extend prior work
where such support agents were modeled using a composition of conversational
behaviors integrated within an event-driven framework. Within the present
approach, conversation is orchestrated through context-sensitive triggering of
the composed behaviors. The core effort involved in applying this approach
involves building a set of triggering policies that achieve this orchestration
in a time-sensitive and coherent manner. In line with recent developments in
data-driven approaches for building dialog systems, we present a novel
technique for learning behavior-specific triggering policies, deploying it as
part of our efforts to improve a socially capable conversational tutor agent
that supports collaborative learning.
[9]
Detecting offensive tweets via topical feature discovery over a large scale
Twitter corpus
Information retrieval short paper session
/
Xiang, Guang
/
Fan, Bin
/
Wang, Ling
/
Hong, Jason
/
Rose, Carolyn
Proceedings of the 2012 ACM Conference on Information and Knowledge
Management
2012-10-29
p.1980-1984
© Copyright 2012 ACM
Summary: In this paper, we propose a novel semi-supervised approach for detecting
profanity-related offensive content in Twitter. Our approach exploits
linguistic regularities in profane language via statistical topic modeling on a
huge Twitter corpus, and detects offensive tweets using automatically these
generated features. Our approach performs competitively with a variety of
machine learning (ML) algorithms. For instance, our approach achieves a true
positive rate (TP) of 75.1% over 4029 testing tweets using Logistic Regression,
significantly outperforming the popular keyword matching baseline, which has a
TP of 69.7%, while keeping the false positive rate (FP) at the same level as
the baseline at about 3.77%. Our approach provides an alternative to large
scale hand annotation efforts required by fully supervised learning approaches.
[10]
Understanding participant behavior trajectories in online health support
groups using automatic extraction methods
Behaviour patterns in online communities
/
Wen, Miaomiao
/
Rose, Carolyn Penstein
GROUP'12: International Conference on Supporting Group Work
2012-10-27
p.179-188
© Copyright 2012 ACM
Summary: This paper presents an automatic analysis method that enables efficient
examination of participant behavior trajectories in online communities, which
offers the opportunity to examine behavior over time at a level of granularity
that has previously only been possible in small scale case study analyses. We
provide an empirical validation of its performance. We then illustrate how this
method offers insights into behavior patterns that enable avoiding faulty
oversimplified assumptions about participation, such as that it follows a
consistent trend over time. In particular, we use this method to investigate
the connection between user behavior and distressful cancer events and
demonstrate how this tool could assist in cancer story summarization.
[11]
Discovering habits of effective online support group chatrooms
Methods for understanding & supporting online communities
/
Mayfield, Elijah
/
Wen, Miaomiao
/
Golant, Mitch
/
Rosé, Carolyn Penstein
GROUP'12: International Conference on Supporting Group Work
2012-10-27
p.263-272
© Copyright 2012 ACM
Summary: For users of online support groups, prior research has suggested that a
positive social environment is a key enabler of coping. Typically,
demonstrating such claims about social interaction would be approached through
the lens of sentiment analysis. In this work, we argue instead for a
multifaceted view of emotional state, which incorporates both a static view of
emotion (sentiment) with a dynamic view based on the behaviors present in a
text. We codify this dynamic view through data annotations marking information
sharing, sentiment, and coping efficacy. Through machine learning analysis of
these annotations, we demonstrate that while sentiment predicts a user's stress
at the beginning of a chat, dynamic views of efficacy are stronger indicators
of stress reduction.
[12]
Supporting collaboration in Wikipedia between language communities
Social media
/
Kulkarni, Ranjitha Gurunath
/
Trivedi, Gaurav
/
Suresh, Tushar
/
Wen, Miaomiao
/
Zheng, Zeyu
/
Rose, Carolyn
Proceedings of the 2012 International Conference on Intercultural
Collaboration
2012-03-21
p.47-56
© Copyright 2012 ACM
Summary: This paper describes an application of machine translation technology for
supporting collaboration in Wikipedia. Wikipedia hosts separate language
Wikipedias for hundreds of different languages. While some content is specific
to these different versions of Wikipedia, some topics have pages within
multiple different Wikipedias. Similarly, while some users participate only in
one Wikipedia, we find users who play a bridging role between these
sub-communities and participate in the process of maintaining similar pages in
different Wikipedias. Since these are not the majority of users, a support tool
that allows stretching the effort of these specialized users further by
indicating where their effort is needed could be a tremendous benefit to the
community. An evaluation of the proposed approach demonstrates promise that
such a tool could substantially reduce the effort involved in playing this
bridging role on Wikipedia.
[13]
Computational representation of discourse practices across populations in
task-based dialogue
Intercultural communication
/
Mayfield, Elijah
/
Adamson, David
/
Rudnicky, Alexander
/
Rosé, Carolyn Penstein
Proceedings of the 2012 International Conference on Intercultural
Collaboration
2012-03-21
p.67-76
© Copyright 2012 ACM
Summary: In this work, we employ quantitative methods to describe the discourse
practices observed in a direction giving task. We place a special emphasis on
comparing differences in strategies between two separate populations and
between successful and unsuccessful groups. We isolate differences in these
strategies through several novel representations of discourse practices. We
find that information sharing, instruction giving, and social feedback
strategies are distinct between subpopulations in empirically identifiable
ways.
[14]
Modeling the Rhetoric of Human-Computer Interaction
Voice, Natural Language and Dialogue
/
Howley, Iris K.
/
Rosé, Carolyn Penstein
HCI International 2011: 14th International Conference on Human-Computer
Interaction, Part II: Interaction Techniques and Environments
2011-07-09
v.2
p.341-350
Keywords: computational linguistics; dialogue analysis; usability heuristics
Copyright © 2011 Springer-Verlag
Summary: The emergence of potential new human-computer interaction styles enabled
through technological advancements in artificial intelligence, machine
learning, and computational linguistics makes it increasingly more important to
formalize and evaluate these innovative approaches. In this position paper, we
propose a multi-dimensional conversation analysis framework as a way to expose
and quantify the structure of a variety of new forms of human-computer
interaction. We argue that by leveraging sociolinguistic constructs referred to
as authoritativeness and heteroglossia, we can expose aspects of novel
interaction paradigms that must be evaluated in light of usability heuristics
so that we can approach the future of human-computer interaction in a way that
preserves the usability standards that have shaped the state-of-the-art that is
tried and true.
[15]
PROSPECT: a system for screening candidates for recruitment
Industry track: IR applications
/
Singh, Amit
/
Rose, Catherine
/
Visweswariah, Karthik
/
Chenthamarakshan, Vijil
/
Kambhatla, Nandakishore
Proceedings of the 2010 ACM Conference on Information and Knowledge
Management
2010-10-26
p.659-668
© Copyright 2010 ACM
Summary: Companies often receive thousands of resumes for each job posting and employ
dedicated screeners to short list qualified applicants. In this paper, we
present PROSPECT, a decision support tool to help these screeners shortlist
resumes efficiently. Prospect mines resumes to extract salient aspects of
candidate profiles like skills, experience in each skill, education details and
past experience. Extracted information is presented in the form of facets to
aid recruiters in the task of screening. We also employ Information Retrieval
techniques to rank all applicants for a given job opening. In our experiments
we show that extracted information improves our ranking by 30% there by making
screening task simpler and more efficient.
[16]
INTERNET
Human-Computer Interaction Institute (HCII)
/
Aleven, Vincent
/
Anderson, John
/
Atkeson, Chris
/
Boyarski, Daniel
/
Cassell, Justine
/
Corbett, Albert
/
Dabbish, Laura
/
Date, Jenna
/
Dey, Anind
/
Evenson, Shelley
/
Forlizzi, Jodi
/
Hong, Jason
/
Hudson, Scott
/
John, Bonnie
/
Kam, Matthew
/
Kiesler, Sara
/
Kittur, Aniket
/
Klatzky, Roberta
/
Koedinger, Ken
/
Kraut, Robert
/
Lindqvist, Janne
/
Matsuda, Noboru
/
McLaren, Bruce M.
/
Morris, James
/
Myers, Brad
/
Neuwirth, Christine
/
Paulos, Eric
/
Pavlik, Philip I., Jr.
/
Rosé, Carolyn Penstein
/
Scheines, Richard
/
Siewiorek, Daniel P.
/
Stamper, John
/
Waibel, Alexander
/
Yang, Jie
/
Zimmerman, John
2010-08-26
2001-09-06
1998-05-22
United States, Pennsylvania, Pittsburgh
Carnegie Mellon University
[17]
Investigating the effect of discussion forum interface affordances on
patterns of conversational interactions
Mathletics: markets and modeling
/
Wang, Yi-Chia
/
Joshi, Mahesh
/
Rosé, Carolyn Penstein
Proceedings of ACM CSCW'08 Conference on Computer-Supported Cooperative Work
2008-11-08
p.555-558
© Copyright 2008 ACM
Summary: We investigate how the affordances provided by alternative interfaces for
on-line discussion forums affect the structure of the discourse that unfolds.
In order to investigate this impact, we compare the predictive power of time
related and text similarity related features for identifying parent-child links
between messages. The results from this work using this methodology suggest
that interfaces that make parent-child relationships between messages explicit
and do not constrain the choice of previous messages that users can reply to
allow patterns of conversational behavior that violate the assumptions of
traditional, tree-structured models of discourse where time related and
similarity related features are highly predictive. An implication for future
work is that because there is evidence that interface affordances affect the
form of conversational contributions, techniques that process on-line
communication data may need to be adapted for different communication
interfaces.
[18]
Sharing a single expert among multiple partners
Expert/novice
/
Wong, Jeffrey
/
Oh, Lui Min
/
Ou, Jiazhi
/
Rosé, Carolyn P.
/
Yang, Jie
/
Fussell, Susan R.
Proceedings of ACM CHI 2007 Conference on Human Factors in Computing Systems
2007-04-28
v.1
p.261-270
© Copyright 2007 ACM
Summary: Expertise to assist people on complex tasks is often in short supply. One
solution to this problem is to design systems that allow remote experts to help
multiple people in simultaneously. As a first step towards building such a
system, we studied experts' attention and communication as they assisted two
novices at the same time in a co-located setting. We compared simultaneous
instruction when the novices are being instructed to do the same task or
different tasks. Using machine learning, we attempted to identify speech
markers of upcoming attention shifts that could serve as input to a remote
assistance system.
[19]
Modeling the impact of shared visual information on collaborative reference
Social influence
/
Gergle, Darren
/
Rose, Carolyn P.
/
Kraut, Robert E.
Proceedings of ACM CHI 2007 Conference on Human Factors in Computing Systems
2007-04-28
v.1
p.1543-1552
© Copyright 2007 ACM
Summary: A number of recent studies have demonstrated that groups benefit
considerably from access to shared visual information. This is due, in part, to
the communicative efficiencies provided by the shared visual context. However,
a large gap exists between our current theoretical understanding and our
existing models. We address this gap by developing a computational model that
integrates linguistic cues with visual cues in a way that effectively models
reference during tightly-coupled, task-oriented interactions. The results
demonstrate that an integrated model significantly outperforms existing
language-only and visual-only models. The findings can be used to inform and
augment the development of conversational agents, applications that dynamically
track discourse and collaborative interactions, and dialogue managers for
natural language interfaces.
[20]
Providing support for adaptive scripting in an on-line collaborative
learning environment
End user programming
/
Gweon, Gahgene
/
Rose, Carolyn
/
Carey, Regan
/
Zaiss, Zachary
Proceedings of ACM CHI 2006 Conference on Human Factors in Computing Systems
2006-04-22
v.1
p.251-260
© Copyright 2006 ACM
Best paper nominee: This paper describes a series of three controlled
experiments that explore on-line learning and the potential benefits of
automatic prompting. It shows the importance of a well-structured
infrastructure for supporting on-line collaborative learning and offers
insights into how students can work together effectively in extended
on-line discussions.
Summary: This paper describes results from a series of experimental studies to
explore issues related to structuring productive group dynamics for
collaborative learning using an adaptive support mechanism. The first study
provides evidence in favor of the feasibility of the endeavor by demonstrating
with a tightly controlled study that even without adaptive support, problem
solving in pairs is significantly more effective for learning than problem
solving alone. The results from a second study offer guidelines for strategic
matching of students with learning partners. Furthermore, the results reveal
specific areas for needed support. Based on the results from the second study,
we present the design of an adaptive support mechanism, which we evaluate in a
third study. The results from the third study provide evidence that certain
aspects of our design for adaptive support in the form of strategic prompts are
effective for manipulating student behavior in productive ways and for
supporting learning. These results also motivate specific modifications to the
original design.
[21]
The effect of miscommunication rate on user response preferences
Work-in-progress
/
Ai, Hua
/
Harris, Thomas
/
Rose, Carolyn Penstein
Proceedings of ACM CHI 2006 Conference on Human Factors in Computing Systems
2006-04-22
v.2
p.448-453
© Copyright 2006 ACM
Summary: We report results from a small Wizard-of-Oz study investigating user
responses to miscommunications in speech dialogue systems. We explore the
separate and joint effects of miscommunication rate and system response to
miscommunications on the likelihood that users choose to resort to direct
manipulation, to repeat, or to rephrase. While we predicted that users would be
more likely to resort to direct manipulation as miscommunication rate
increased, our surprising finding was that users were most likely to resort to
direct manipulation where communication success was least predictable, i.e., in
the middle of the range, rather than at either extreme.
[22]
Usable browsers for ontological knowledge acquisition
Work-in-progress
/
Tribble, Alicia
/
Rose, Carolyn
Proceedings of ACM CHI 2006 Conference on Human Factors in Computing Systems
2006-04-22
v.2
p.1451-1456
© Copyright 2006 ACM
Summary: In this paper we compare the usability of several presentation formats for
ontological knowledge of events. The goal is to support further work in
knowledge acquisition from informants who are not necessarily experienced with
knowledge representations. This work investigates the question: How can we
present detailed ontological information to such informants, in a format that
is easy to understand, modify, and augment? We compare three formats: two
commonly-used diagram styles and one lisp-like list of knowledge axioms.
Ongoing work on this topic will expand the investigation into a study of the
role of natural language in knowledge acquisition.
[23]
Interactivity and Expectation: Eliciting Learning Oriented Behavior with
Tutorial Dialogue Systems
Long Papers: Intelligent Interfaces
/
Rose, C. P.
/
Torrey, C.
Proceedings of IFIP INTERACT'05: Human-Computer Interaction
2005-09-12
p.323-336
Summary: We investigate the reasons behind students' different responses to human
versus machine tutors and explore possible solutions that will motivate
students to offer more elaborated responses to computerized tutoring systems,
and ultimately behave in a more "learning oriented" manner. We focus upon two
sets of variables, one surrounding the students' perceptions of tutor qualities
and the other surrounding the conversational dynamics of the dialogues
themselves. We offer recommendations based on our empirical investigations.
[24]
The Necessity of a Meeting Recording and Playback System, and the Benefit of
Topic-Level Annotations to Meeting Browsing
Long Papers: Computer Supported Cooperative Work (CSCW)
/
Banerjee, S.
/
Rose, C.
/
Rudnicky, A. I.
Proceedings of IFIP INTERACT'05: Human-Computer Interaction
2005-09-12
p.643-656
Summary: Much work in the area of Computer Supported Cooperative Work (CSCW) has
targeted the problem of supporting meetings between collaborators who are
non-collocated, enabling meetings to transcend boundaries of space. In this
paper, we explore the beginnings of a proposed solution for allowing meetings
to transcend time as well. The need for such a solution is motivated by a user
survey in which busy professionals are questioned about meetings they have
either missed or forgotten the important details about after the fact. Our
proposed solution allows these professionals to transcend time in a sense by
revisiting a recorded meeting that has been structured for quick retrieval of
sought information. Such a solution supports complete recovery of prior
discussions, allowing needed information to be retrieved quickly, and thus
potentially facilitating the effective continuation of discussions from the
past. We evaluate the proposed solution with a formal user study in which we
measure the impact of the proposed structural annotations on retrieval of
information. The results of the study show that participants took significantly
less time to retrieve the answers when they had access to discourse structure
based annotation than in a control condition in which they had access only to
unannotated video recordings (p < 0.01, effect size 0.94 standard
deviations).
[25]
Supporting Efficient and Reliable Content Analysis Using Automatic Text
Processing Technology
Short Papers: Tools
/
Gweon, G.
/
Rose, C. P.
/
Wittwer, J.
/
Nueckles, M.
Proceedings of IFIP INTERACT'05: Human-Computer Interaction
2005-09-12
p.1112-1115
Summary: Text categorization technology can be used to streamline the process of
content analysis of corpus data. However, while recent results for automatic
corpus analysis show great promise, tools that are currently being used for HCI
research and practice do not make use of it. Here, we empirically evaluate
trade-offs between semi automatic and hand labeling of data in terms of speed,
validity, and reliability of coding in order to assess the usefulness of
incorporating this technology into HCI tools.
