| Leveraging machine-learned detectors of systematic inquiry behavior to estimate and predict transfer of inquiry skill | | BIBAK | Full-Text | 1-39 | |
| Michael A. Sao Pedro; Ryan S. J. de Baker | |||
| We present work toward automatically assessing and estimating science
inquiry skills as middle school students engage in inquiry within a physical
science microworld. Towards accomplishing this goal, we generated
machine-learned models that can detect when students test their articulated
hypotheses, design controlled experiments, and engage in planning behaviors
using two inquiry support tools. Models were trained using labels generated
through a new method of manually hand-coding log files, "text replay tagging".
This approach led to detectors that can automatically and accurately identify
these inquiry skills under student-level cross-validation. The resulting
detectors can be applied at run-time to drive scaffolding intervention. They
can also be leveraged to automatically score all practice attempts, rather than
hand-classifying them, and build models of latent skill proficiency. As part of
this work, we also compared two approaches for doing so, Bayesian
Knowledge-Tracing and an averaging approach that assumes static inquiry skill
level. These approaches were compared on their efficacy at predicting skill
before a student engages in an inquiry activity, predicting performance on a
paper-style multiple choice test of inquiry, and predicting performance on a
transfer task requiring data collection skills. Overall, we found that both
approaches were effective at estimating student skills within the environment.
Additionally, the models' skill estimates were significant predictors of the
two types of inquiry transfer tests. Keywords: Scientific inquiry; Exploratory learning environment assessment; Skill
prediction; Machine-learned models; Microworlds; Behavior detection; Designing
and conducting experiments; Bayesian Knowledge-Tracing | |||
| A PLA-based privacy-enhancing user modeling framework and its evaluation | | BIBAK | Full-Text | 41-82 | |
| Yang Wang; Alfred Kobsa | |||
| Reconciling personalization with privacy has been a continuing interest in
user modeling research. This aim has computational, legal and
behavioral/attitudinal ramifications. We present a dynamic privacy-enhancing
user modeling framework that supports compliance with users' individual privacy
preferences and with the privacy laws and regulations that apply to each user.
The framework is based on a software product line architecture. It dynamically
selects personalization methods during runtime that meet the current privacy
constraints. Since dynamic architectural reconfiguration is typically
resource-intensive, we conducted a performance evaluation with four
implementations of our system that vary two factors. The results demonstrate
that at least one implementation of our approach is technically feasible with
comparatively modest additional resources, even for websites with the highest
traffic today. To gauge user reactions to privacy controls that our framework
enables, we also conducted a controlled experiment that allowed one group of
users to specify privacy preferences and view the resulting effects on employed
personalization methods. We found that users in this treatment group utilized
this feature, deemed it useful, and had fewer privacy concerns as measured by
higher disclosure of their personal data. Keywords: User modeling; Privacy laws; Privacy preferences; Compliance; Product line
architecture; Performance evaluation; User experiment; Disclosure behavior | |||
| Preface to the Special Issue on Personalization in Social Web systems | | BIB | Full-Text | 83-87 | |
| Peter Brusilovsky; David N. Chin | |||
| The evaluation of a social adaptive website for cultural events | | BIBAK | Full-Text | 89-137 | |
| Cristina Gena; Federica Cena; Fabiana Vernero | |||
| In this paper, we present an evaluation of a social adaptive website in the
domain of cultural events, iCITY DSA, which provides information about cultural
resources and events that promote the cultural heritage in the city of Turin.
Using this evaluation, our objective was to investigate the actual usage of a
social adaptive website, in an effort to discover the real behavior of users,
the unforeseen correlations among user actions and the consequent interactive
behavior, the accuracy of both system and social recommendations and their
impact on the users themselves, and the role of tagging in the user modeling
process. The major contributions of the paper are manifold: insights into user
interactions with social adaptive systems; guidelines for future designs;
evaluation of the tagging activity and tag meanings in relation to the
application domain and thus their impact on the representation of the user
model; and a demonstration of how a combination and interplay of evaluation
methodologies (e.g., quantitative and qualitative) can enhance our
comprehension of evaluation data. Keywords: Evaluation; Social adaptive systems; Tag-based user model; Cultural events;
Social recommenders | |||
| A knowledge-tracing model of learning from a social tagging system | | BIBAK | Full-Text | 139-168 | |
| Peter Pirolli; Sanjay Kairam | |||
| We propose a user model to support personalized learning paths through
online material. Our approach is a variant of student modeling using the
computer tutoring concept of knowledge tracing. Knowledge tracing involves
representing the knowledge required to master a domain, and, from traces of
online user behavior, diagnosing user knowledge states as a profile over those
elements. The user model is induced from documents tagged by an expert in a
social tagging system. Tags identified with "expertise" in a domain can be used
to identify a corpus of domain documents. That corpus can be fed to an
automated process that distills a topic model representation characteristic of
the domain. As a learner navigates and reads online material, inferences can be
made about the degree to which topics in the target domain have been learned.
We validate this knowledge tracing approach against data from a social tagging
study. As part of this evaluation, we match the predictions of the
knowledge-tracing model to individual participant responses made to individual
question items used to test domain knowledge. Keywords: Cognitive models; User models; Latent Dirichlet allocation; LDA; Topic
models; SparTag.us; Social tagging; Social web | |||
| Cross-system user modeling and personalization on the Social Web | | BIBAK | Full-Text | 169-209 | |
| Fabian Abel; Eelco Herder; Geert-Jan Houben | |||
| In order to adapt functionality to their individual users, systems need
information about these users. The Social Web provides opportunities to gather
user data from outside the system itself. Aggregated user data may be useful to
address cold-start problems as well as sparse user profiles, but this depends
on the nature of individual user profiles distributed on the Social Web. For
example, does it make sense to re-use Flickr profiles to recommend bookmarks in
Delicious? In this article, we study distributed form-based and tag-based user
profiles, based on a large dataset aggregated from the Social Web. We analyze
the completeness, consistency and replication of form-based profiles, which
users explicitly create by filling out forms at Social Web systems such as
Twitter, Facebook and LinkedIn. We also investigate tag-based profiles, which
result from social tagging activities in systems such as Flickr, Delicious and
StumbleUpon: to what extent do tag-based profiles overlap between different
systems, what are the benefits of aggregating tag-based profiles. Based on
these insights, we developed and evaluated the performance of several
cross-system user modeling strategies in the context of recommender systems.
The evaluation results show that the proposed methods solve the cold-start
problem and improve recommendation quality significantly, even beyond the
cold-start. Keywords: User modeling; Personalization; Social Web; User profiles; Social tagging;
Cross-system user modeling | |||
| Facebook single and cross domain data for recommendation systems | | BIBAK | Full-Text | 211-247 | |
| Bracha Shapira; Lior Rokach | |||
| The emergence of social networks and the vast amount of data that they
contain about their users make them a valuable source for personal information
about users for recommender systems. In this paper we investigate the
feasibility and effectiveness of utilizing existing available data from social
networks for the recommendation process, specifically from Facebook. The data
may replace or enrich explicit user ratings. We extract from Facebook content
published by users on their personal pages about their favorite items and
preferences in the domain of recommendation, and data about preferences related
to other domains to allow cross-domain recommendation. We study several methods
for integrating Facebook data with the recommendation process and compare the
performance of these methods with that of traditional collaborative filtering
that utilizes user ratings. In a field study that we conducted, recommendations
obtained using Facebook data were tested and compared for 95 subjects and their
crawled Facebook friends. Encouraging results show that when data is sparse or
not available for a new user, recommendation results relying solely on Facebook
data are at least equally as accurate as results obtained from user ratings.
The experimental study also indicates that enriching sparse rating data by
adding Facebook data can significantly improve results. Moreover, our findings
highlight the benefits of utilizing cross domain Facebook data to achieve
improvement in recommendation performance. Keywords: Recommender systems; Facebook; Collaborative filtering; Cross-Domain
recommendations; Evaluation | |||
| Exploring social tagging for personalized community recommendations | | BIBAK | Full-Text | 249-285 | |
| Heung-Nam Kim; Abdulmotaleb El Saddik | |||
| Users of social Web sites actively create and join communities as a way to
collectively share their media content and rich experience with diverse groups
of people. In this study we focus on the issue of recommending social
communities (or groups) to individual users. We address specifically the
potential of social tagging for accentuating users' interests and
characterizing communities. We also discuss some unique methods of improving
several techniques that have been adapted for use in the context of community
recommendations: collaborative filtering, a random walk model, a Katz influence
model, a latent semantic model, and a user-centric tag model. We effectively
incorporate social tagging information in each algorithm. We present empirical
evaluations using real datasets from CiteULike and Last.fm. Our experimental
results demonstrate that the different algorithms incorporated with social
tagging offer significant advantages in improving both the recommendation
quality and coverage, and demonstrate their feasibility for community
recommendations in dealing with sparsity-related limitations. Keywords: Community recommendations; Collaborative filtering; Graph-based
recommendation; Latent semantic analysis; Recommender systems; Social tagging | |||
| Adaptive notifications to support knowledge sharing in close-knit virtual communities | | BIBAK | Full-Text | 287-343 | |
| Styliani Kleanthous Loizou; Vania Dimitrova | |||
| Social web-groups where people with common interests and goals communicate,
share resources, and construct knowledge, are becoming a major part of today's
organisational practice. Research has shown that appropriate support for
effective knowledge sharing tailored to the needs of the community is
paramount. This brings a new challenge to user modelling and adaptation, which
requires new techniques for gaining sufficient understanding of a virtual
community (VC) and identifying areas where the community may need support. The
research presented here addresses this challenge presenting a novel
computational approach for community-tailored support underpinned by
organisational psychology and aimed at facilitating the functioning of the
community as a whole (i.e. as an entity). A framework describing how key
community processes -- transactive memory (TM), shared mental models (SMMs),
and cognitive centrality (CCen) -- can be utilised to derive knowledge sharing
patterns from community log data is described. The framework includes two
parts: (i) extraction of a community model that represents the community based
on the key processes identified and (ii) identification of knowledge sharing
behaviour patterns that are used to generate adaptive notifications. Although
the notifications target individual members, they aim to influence individuals'
behaviour in a way that can benefit the functioning of the community as a
whole. A validation study has been performed to examine the effect of
community-adapted notifications on individual members and on the community as a
whole using a close-knit community of researchers sharing references. The study
shows that notification messages can improve members' awareness and perception
of how they relate to other members in the community. Interesting observations
have been made about the linking between the physical and the VC, and how this
may influence members' awareness and knowledge sharing behaviour. Broader
implications for using log data to derive community models based on key
community processes and generating community-adapted notifications are
discussed. Keywords: Community modelling; Adaptive support for knowledge sharing; Virtual
communities | |||
| Creating a model of the dynamics of socio-technical groups | | BIBAK | Full-Text | 345-379 | |
| Sean P. Goggins; Giuseppe Valetto | |||
| Individuals participating in technologically mediated forms of organization
often have difficulty recognizing when groups emerge, and how the groups they
take part in evolve. This paper contributes an analytical framework that
improves awareness of these virtual group dynamics through analysis of
electronic trace data from tasks and interactions carried out by individuals in
systems not explicitly designed for context adaptivity, user modeling or user
personalization. We discuss two distinct cases to which we have applied our
analytical framework. These two cases provide a useful contrast of two
prevalent ways for analyzing social relations starting from electronic trace
data: either artifact-mediated or direct person-to-person interactions. Our
case study integrates electronic trace data analysis with analysis of other,
triangulating data specific to each application. We show how our techniques fit
in a general model of group informatics, which can serve to construct group
context, and be leveraged by future tool development aimed at augmenting
context adaptivity with group context and a social dimension. We describe our
methods, data management strategies and technical architecture to support the
analysis of individual user task context, increased awareness of group
membership, and an integrated view of social, information and coordination
contexts. Keywords: Activity awareness; Group awareness; Virtual groups; Communities of
practice; Networks of practice; Task context | |||
| Personalised Information Retrieval: survey and classification | | BIBAK | Full-Text | 381-443 | |
| M. Rami Ghorab; Dong Zhou; Alexander O'Connor | |||
| Information Retrieval (IR) systems assist users in finding information from
the myriad of information resources available on the Web. A traditional
characteristic of IR systems is that if different users submit the same query,
the system would yield the same list of results, regardless of the user.
Personalised Information Retrieval (PIR) systems take a step further to better
satisfy the user's specific information needs by providing search results that
are not only of relevance to the query but are also of particular relevance to
the user who submitted the query. PIR has thereby attracted increasing research
and commercial attention as information portals aim at achieving user loyalty
by improving their performance in terms of effectiveness and user satisfaction.
In order to provide a personalised service, a PIR system maintains information
about the users and the history of their interactions with the system. This
information is then used to adapt the users' queries or the results so that
information that is more relevant to the users is retrieved and presented. This
survey paper features a critical review of PIR systems, with a focus on
personalised search. The survey provides an insight into the stages involved in
building and evaluating PIR systems, namely: information gathering, information
representation, personalisation execution, and system evaluation. Moreover, the
survey provides an analysis of PIR systems with respect to the scope of
personalisation addressed. The survey proposes a classification of PIR systems
into three scopes: individualised systems, community-based systems, and
aggregate-level systems. Based on the conducted survey, the paper concludes by
highlighting challenges and future research directions in the field of PIR. Keywords: Personalisation; User modelling; User interests; Information Retrieval;
Multilingual Information Retrieval; Adaptive hypermedia; Search history; Query
adaptation; Result adaptation; Evaluation; Survey | |||
| James Chen Annual Award for Best Journal Article | | BIB | Full-Text | 445 | |
| Recommending people to people: the nature of reciprocal recommenders with a case study in online dating | | BIBAK | Full-Text | 447-488 | |
| Luiz Pizzato; Tomasz Rej; Joshua Akehurst | |||
| People-to-people recommenders constitute an important class of recommender
systems. Examples include online dating, where people have the common goal of
finding a partner, and employment websites where one group of users needs to
find a job (employer) and another group needs to find an employee.
People-to-people recommenders differ from the traditional items-to-people
recommenders as they must satisfy both parties; we call this type of
recommender reciprocal. This article is the first to present a comprehensive
view of this important recommender class. We first identify the characteristics
of reciprocal recommenders and compare them with traditional recommenders,
which are widely used in e-commerce websites. We then present a series of
studies and evaluations of a content-based reciprocal recommender in the domain
of online dating. It uses a large dataset from a major online dating website.
We use this case study to illustrate the distinctive requirements of reciprocal
recommenders and highlight important challenges, such as the need to avoid bad
recommendations since they may make users to feel rejected. Our experiments
indicate that, by considering reciprocity, the rate of successful connections
can be significantly improved. They also show that, despite the existence of
rich explicit profiles, the use of implicit profiles provides more effective
recommendations. We conclude with a discussion, linking our work in online
dating to the many other domains that require reciprocal recommenders. Our key
contributions are the recognition of the reciprocal recommender as an important
class of recommender, the identification of its distinctive characteristics and
the exploration of how these impact the recommendation process in an extensive
case study in the domain of online dating. Keywords: Recommender systems; Online dating; Reciprocity | |||
| Real-time rule-based classification of player types in computer games | | BIBAK | Full-Text | 489-526 | |
| Ben Cowley; Darryl Charles; Michaela Black | |||
| The power of using machine learning to improve or investigate the experience
of play is only beginning to be realised. For instance, the experience of play
is a psychological phenomenon, yet common psychological concepts such as the
typology of temperaments have not been widely utilised in game design or
research. An effective player typology provides a model by which we can analyse
player behaviour. We present a real-time classifier of player type, implemented
in the test-bed game Pac-Man. Decision Tree algorithms CART and C5.0 were
trained on labels from the DGD player typology (Bateman and Boon, 21st century
game design, vol. 1, 2005). The classifier is then built by selecting rules
from the Decision Trees using a rule- performance metric, and experimentally
validated. We achieve 70% accuracy in this validation testing. We further
analyse the concept descriptions learned by the Decision Trees. The algorithm
output is examined with respect to a set of hypotheses on player behaviour. A
set of open questions is then posed against the test data obtained from
validation testing, to illustrate the further insights possible from extended
analysis. Keywords: Player typology; Player profiling; Computer games; Decision trees;
Classification; Experimental validation | |||