HCI Bibliography Home | HCI Conferences | LAK Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
LAK Tables of Contents: 1112131415

LAK'12: 2012 International Conference on Learning Analytics and Knowledge

Fullname:Proceedings of the 2nd International Conference on Learning Analytics and Knowledge
Editors:Shane Dawson; Caroline Haythornthwaite; Simon Buckingham Shum; Dragan Gasevic; Rebecca Ferguson
Location:Vancouver, British Columbia, Canada
Dates:2012-Apr-29 to 2012-May-02
Standard No:ISBN: 978-1-4503-1111-3; ACM DL: Table of Contents; hcibib: LAK12
Links:Conference Website
  1. Keynote
  2. Workshop
  3. Panel
  4. Social learning analytics
  5. Adaptive-recommender systems
  6. Analytics for reflective learning
  7. Institutional perspectives
  8. Visual analytics
  9. Educator interventions
  10. Textual analytics & analytics infrastructure
  11. Empirical studies
  12. Educational data mining
  13. Predictive modeling


Networked individualism: how the personalized internet, ubiquitous connectivity, and the turn to social networks can affect learning analytics BIBAFull-Text 1
  Barry Wellman
The Triple Revolution -- the coming together of the turn to social networks, the personalized internet, and accessible mobile connectivity -- has fostered networked individualism. This has implications for learning analytics, in the need to move beyond analyzing bounded groups and aggregates of individuals to taking into account complex, partial networks of social relationships.
Visual analytics in support of education BIBAFull-Text 2-3
  Katy Börner
The amount of data about us and our world is increasing rapidly, and the capability to analyze large data sets -- so-called big data--becomes a key basis of competition, underpinning new waves of productivity growth and innovation. The big data phenomenon is fueled by cheap sensors and high-throughput simulation models, the increasing volume and detail of information captured by enterprises, the rise of multimedia, social media, and the Internet. It exists from social media to cell biology offering unparalleled opportunities to document the inner workings of many complex systems [1]. Research by MGI and McKinsey's Business Technology Office argues that there will be a shortage of talent necessary for organizations to take advantage of big data. "By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions" [2].
   In everyday life, people deal with large amounts of data regularly: online search engines provide access to millions of web sites almost instantly; consumer sites offer literally thousands of purchase options seamlessly; and social media sites let you create and benefit from extensive social networks.
   In bestselling books like Freakonomics, Super Crunchers and The Numerati, authors illuminate how more and more decisions in health care, politics, education, and other sectors utilize big data and data analysis [3]. The texts highlight the growing need for specialists and every-day citizens to be able to understand and interpret data. Whether it is a table of nutritional information, a graph of stock prices, or a chart comparing health care plans, the skills of understanding and interpreting data are necessary to navigate successfully through daily life.
   This talk starts with a review of visual analytics projects that aim to increase our understanding of how people learn, increase the efficacy of learning environments, or support decision making in education [4]. The second part of the talk provides a theoretical framework for the design of effective data analysis workflows and insightful visualizations. It also introduces plug-and-play macroscope tools [5], see also http://cishell.org, that were designed for different research communities and are used by more than 120,000 users from 40+ countries to design and benefit from visualizations of complex data.
   The talk concludes with a discussion of challenges that arise when visual analytics tools are introduced to classrooms and informal science education.
Learning analytics: envisioning a research discipline and a domain of practice BIBAFull-Text 4-8
  George Siemens
Learning analytics are rapidly being implemented in different educational settings, often without the guidance of a research base. Vendors incorporate analytics practices, models, and algorithms from datamining, business intelligence, and the emerging "big data" fields. Researchers, in contrast, have built up a substantial base of techniques for analyzing discourse, social networks, sentiments, predictive models, and in semantic content (i.e., "intelligent" curriculum). In spite of the currently limited knowledge exchange and dialogue between researchers, vendors, and practitioners, existing learning analytics implementations indicate significant potential for generating novel insight into learning and vital educational practices. This paper presents an integrated and holistic vision for advancing learning analytics as a research discipline and a domain of practices. Potential areas of collaboration and overlap are presented with the intent of increasing the impact of analytics on teaching, learning, and the education system.


1st International Workshop on Learning Analytics and Linked Data BIBAFull-Text 9-10
  Hendrik Drachsler; Stefan Dietze; Wolfgang Greller; Mathieu D'Aquin; Jelena Jovanovic; Abelardo Pardo; Wolfgang Reinhardt; Katrien Verbert
The main objective of the 1st International Workshop on Learning Analytics and Linked Data (#LALD2012) is to connect the research efforts on Linked Data and Learning Analytics in order to create visionary ideas and foster synergies between the two young research fields. Therefore, the workshop will collect, explore, and present datasets, technologies and applications for Technology Enhanced Learning (TEL) to discuss Learning Analytics approaches that make use of educational data or Linked Data sources. During the workshop, an overview of available educational datasets and related initiatives will be given. The participants will have the opportunity to present their own research with respect to educational datasets, technologies and applications and discuss major challenges to collect, reuse, and share these datasets.
Connecting levels and methods of analysis in networked learning communities BIBAFull-Text 11-13
  Daniel D. Suthers; H. Ulrich Hoppe; Maarten de Laat; Simon Buckingham Shum
This paper describes the rationale behind a workshop on using data-intensive computational methods of analysis for empirical-analytical studies of collaborative and networked learning, with a particular focus on how learning takes place in the technically-mediated interplay between individual, small group and collective levels of agency. This workshop is primarily designed for researchers interested in empirical-analytical studies using data-intensive computational methods of analysis (including social-network analysis, log-file analysis, data mining, video analysis).
Where learning analytics meets learning design BIBAFull-Text 14-15
  Lori Lockyer; Shane Dawson
The wealth of data available through student management systems and eLearning systems has the potential to provide faculty with important, just-in-time information that may allow them to positively intervene with struggling students and/or enhance the learning experience during the delivery of a course. This information might also facilitate post-delivery review and reflection for faculty who wish to revise course design and content. But to be effective, this data needs to be appropriate to the context or pedagogical intent of the course -- this is where learning analytics meets learning design.
Learning analytics and higher education: ethical perspectives BIBAFull-Text 16-17
  Sharon Slade; Fenella Galpin
Take two students who were enrolled on the same higher education course, both of whom were identified as likely to benefit from additional support and tailoring of their learning experience. Three years later, one student has gone on to gain a good degree and is now making great progress in her career. The other student, whose background and learning needs appeared similar, scraped through the experience, has recently been eased out of her organization and is unemployed. To what extent were decisions taken by their tutors and institution about the design of their learning experiences, responsible for these two very different outcomes?


Building organizational capacity for analytics: panel proposal BIBAFull-Text 18-19
  Donald M. Norris; Linda Baer
The field of analytics is mushrooming. Analytics is perceived as the potential decoder for institutional transformation. Given the mandates for improved assessment, accountability and performance, analytical tools are in high demand. A critical goal is to optimize student success by managing the student pipeline to success, eliminating structural and programmatic impediments to retention and success and by utilizing dynamic query, reporting, intervention and embedded predictive analytics to respond to at-risk behavior. Additional optimization practices are emerging as well: Expanded data mining, early-stage learner relationship management practices, and consideration of employability success. This panel presentation will describe the tools leading-edge institutions are using and what tools vendors are offering. The gap between supply and demand will be the main focus of the session.
Educational data mining meets learning analytics BIBAFull-Text 20
  Ryan S. J. d. Baker; Erik Duval; John Stamper; David Wiley; Simon Buckingham Shum
W This panel is proposed as a means of promoting mutual learning and continued dialogue between the Educational Data Mining and Learning Analytics communities. EDM has been developing as a community for longer than the LAK conference, so what if anything makes the LAK community different, and where is the common ground?
Building a data governance model for learning analytics BIBAFull-Text 21-22
  Sabine Graf; Cindy Ives; Lori Lockyer; Paul Hobson; Doug Clow
This international panel presentation aims to explore and discuss the issues that emerge when an educational institution decides to develop learning analytics initiatives. While learning analytics may provide data that lead to improvements in the quality of teaching and learning design, and therefore has the potential to enhance the overall quality of education, the successful development and implementation of tools and processes for learning analytics are complex and problematic. In this panel, data governance considerations will be discussed from organizational, ethical, learning design, and technical points of view.

Social learning analytics

Social learning analytics: five approaches BIBAFull-Text 23-33
  Rebecca Ferguson; Simon Buckingham Shum
This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The paper introduces the broad rationale for SLA by reviewing some of the key drivers that make social learning so important today. Five forms of SLA are identified, including those which are inherently social, and others which have social dimensions. The paper goes on to describe early work towards implementing these analytics on SocialLearn, an online learning space in use at the UK's Open University, and the challenges that this is raising. This work takes an iterative approach to analytics, encouraging learners to respond to and help to shape not only the analytics but also their associated recommendations.
Modelling learning & performance: a social networks perspective BIBAFull-Text 34-42
  Walter Christian Paredes; Kon Shing Kenneth Chung
Traditional models of learning using a sociological perspective include social learning, situated learning and models of connectivisim and self-efficacy. While these models explain how individuals learn in varying social dimensions, very few studies provide empirical validation of such models and extend them to include group learning and performance. In this exploratory study, we develop a theoretical model based on social learning and social network theories to understand how knowledge professionals engage in learning and performance, both as individuals and as groups. We investigate the association between egocentric network properties (structure, position and tie), 'content richness' in the social learning process and performance. Analysis from data collected using an online eLearning environment shows that rather than performance; social learning is influenced by properties of social network structure (density, inter-group and intra-network communication), relations (tie strength) and position (efficiency). Furthermore, individuals who communicate with others internal rather than external to the group show higher tendencies of social learning. The contribution of this study is therefore two-fold: a theoretical development of a social learning and networks based model for understanding learning and performance; and the construction of a novel metric called 'content richness' as a surrogate measure for social learning. In conclusion, a useful implication of the study is that the model fosters understanding social factors that influence learning and performance in the domain of learning analytics. It also begs the question of whether the relationship between social networks and performance is mediated or moderated by learning and whether assumptions of the model hold true in non-educational domains.
Multi-mediated community structure in a socio-technical network BIBAFull-Text 43-53
  Dan Suthers; Kar-Hai Chu
Digital environments for networked learning and professional networks may not comprise one "community:" identification of clusters of affiliated groups of participants that potentially constitute embedded communities is an empirical matter, and one of interest to managers of large learning and professional networks. Also, these socio-technical networks are typically multi-mediated, in that they offer multiple means of participation, each with their own interactional affordances. Different communities may be using the multiple media in different ways. We have developed an analytic framework for extracting events from log files and representing interaction and affiliations at different granularities as needed for analysis. In this paper we show how bimodal networks of actors and media artifacts can be constructed in which directed arcs relate actors to the artifacts they read, write or edit, and how the resulting graphs can be used to detect community structures that extend across different media. We illustrate these ideas with a study that characterizes community structure within the Tapped In network of educational professionals, and how the associations between members of this network are distributed across media (chat rooms, discussion forums and file sharing).
Challenges and opportunities for learning analytics when formal teaching meets social spaces BIBAFull-Text 54-58
  Nazim Rahman; Jon Dron
Social networking is revolutionizing the world in ways few imagined just a few years ago. The power of social networking technology can also be leveraged to improve education and enhance the instructor and learner experience. Unlike conventional learning management systems, social software environments such as Athabasca Landing provide a persistent space and are flexible enough to support social and learner-led methods of informal, non-formal, and formal learning. Analytics can be used to effectively track and measure personal progress and help uncover extra-curricular factor affecting learner success such as network formation and growth. The paper reports on an attempt to explore this problem through analysis of student behaviour on the Athabasca Landing site within the context of a course. Its findings, explanation, and potential implications are listed. Effects of social learning on learners, based on the learner's behaviour before, during, and after the course are described and discussed. Finally, features of an open source tool created for this analysis, LASSIE is presented.
Network awareness tool -- learning analytics in the workplace: detecting and analyzing informal workplace learning BIBAFull-Text 59-64
  Schreurs Bieke; De Laat Maarten
This paper aims to contribute to the understanding of informal workplace learning in contemporary face-to-face and virtual environments. Informal learning is an important driver for professional development and workplace learning. However powerful informal learning may be, there is a problem when it comes to making it a real asset within organizations: Informal learning activities are mostly invisible to others, sometimes the learners themselves might not even be aware of the learning that occurs. As a consequence informal learning in organizations goes undetected, remains off the radar of HR departments and is therefore hard to asses, manage and value [1]. This problem poses an interesting challenge for the field of Learning Analytics, namely finding ways to capture and analyse traces of (social) informal learning in every day life and work networks. Therefore empirical research and tools are needed that can raise awareness about informal learning activities to make it surface the radar, amplify the benefits of it and strengthen the social relations through which it occurs. In this paper we introduce a tool that aims to facilitate exactly this and we hope to stimulate to widen the discussion on Learning Analytics by expanding the field from a predominantly educational focus to informal and workplace learning. In this paper we will discuss methodologies that Learning Analytics can draw upon to make informal learning more explicit and accessible to analyse and to share amongst professionals.
Cyberlearners and learning resources BIBAFull-Text 65-68
  Leyla Zhuhadar; Rong Yang
The discovery of community structure in real world networks has transformed the way we explore large systems. We propose a visual method to extract communities of cyberlearners in a large interconnected network consisting of cyberlearners and learning resources. The method used is heuristic and is based on visual clustering and a modularity measure. Each cluster of users is considered as a subset of the community of learners sharing a similar domain of interest. Accordingly, a recommender system is proposed to predict and recommend learning resources to cyberlearners within the same community. Experiments on real, dynamic data reveal the structure of community in the network. Our approach used the optimal discovered structure based on the modularity value to design a recommender system.
First steps towards a social learning analytics for online communities of practice for educators BIBAFull-Text 69-72
  Darren Cambridge; Kathleen Perez-Lopez
Learning analytics has the potential to provide actionable insights for managers of online communities of practice. Because the purposes of such communities and the patterns of activity that might further them are diverse, a wider range of methods may be needed than in formal educational settings. This paper describes the proposed learning analytics approach of the U. S. Department of Education's Connected Educators project, and presents preliminary applications of social network analysis to the National Science Teachers Association Learning Center as an illustration.

Adaptive-recommender systems

It's just about learning the multiplication table BIBAFull-Text 73-81
  Martin Schön; Martin Ebner; Georg Kothmeier
One of the first and basic mathematical knowledge of school children is the multiplication table. At the age of 8 to 10 each child has to learn by training step by step, or more scientifically, by using a behavioristic learning concept. Due to this fact it can be mentioned that we know very well about the pedagogical approach, but on the other side there is rather less knowledge about the increase of step-by-step knowledge of the school children.
   In this publication we present some data documenting the fluctuation in the process of acquiring the multiplication tables. We report the development of an algorithm which is able to adapt the given tasks out of a given pool to unknown pupils. For this purpose a web-based application for learning the multiplication table was developed and then tested by children. Afterwards so-called learning curves of each child were drawn and analyzed by the research team as well as teachers carrying out interesting outcomes. Learning itself is maybe not as predictable as we know from pedagogical experiences, it is a very individualized process of the learners themselves.
   It can be summarized that the algorithm itself as well as the learning curves are very useful for studying the learning success. Therefore it can be concluded that learning analytics will become an important step for teachers and learners of tomorrow.
Sherpa: increasing student success with a recommendation engine BIBAFull-Text 82-83
  Robert Bramucci; Jim Gaston
Students flock to online services like Amazon, Pandora and Netflix that offer personalized recommendations, in stark contrast to the "one size fits all" services in higher education. In this session we demonstrate Sherpa, a recommendation engine for courses, information and services that utilizes both human and machine intelligence.
Using an instructional expert to mediate the locus of control in adaptive e-learning systems BIBAFull-Text 84-87
  Christopher A. Brooks; Jim Greer; Carl Gutwin
This paper considers the issue of the locus of control in adaptive e-learning environments from the perspective of a new stakeholder; the instructional expert. With an ever increasing ability to gain insight into learners based on their online activities, instructors and instructional designers are poised to add value to the process of adaptation, a process normally reserved for either systems designers or the end user. This work describes the design of an e-learning system which provides automated analytics information to these experts for consideration, and then leverages the insights these experts have made as the basis for content and feature adaptation.
What to do with actionable intelligence: E²Coach as an intervention engine BIBAFull-Text 88-91
  Tim McKay; Kate Miller; Jared Tritz
In this paper, we describe a new, analytics driven approach to supporting students in large introductory physics courses. For this project, we have assembled data for more than 49,000 physics students at the University of Michigan. For each, we combine an extensive portrait of background and preparation with details of progress through the course and final outcome. This information allows us to construct models predicting student performance with a dispersion of half a letter grade. We explore residuals to this model, conducting structured interviews with students who did better (and worse) than expected, identifying strategies which lead to student success (and failure) at all levels of preparation. This work was done in preparation for the launch of E2Coach: a computer tailored educational coaching project which provides a model for an intervention engine, capable of dealing with actionable information for thousands of students.

Analytics for reflective learning

Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics BIBAFull-Text 92-101
  Simon Buckingham Shum; Ruth Deakin Crick
Theoretical and empirical evidence in the learning sciences substantiates the view that deep engagement in learning is a function of a complex combination of learners' identities, dispositions, values, attitudes and skills. When these are fragile, learners struggle to achieve their potential in conventional assessments, and critically, are not prepared for the novelty and complexity of the challenges they will meet in the workplace, and the many other spheres of life which require personal qualities such as resilience, critical thinking and collaboration skills. To date, the learning analytics research and development communities have not addressed how these complex concepts can be modelled and analysed, and how more traditional social science data analysis can support and be enhanced by learning analytics. We report progress in the design and implementation of learning analytics based on a research validated multidimensional construct termed "learning power". We describe, for the first time, a learning analytics infrastructure for gathering data at scale, managing stakeholder permissions, the range of analytics that it supports from real time summaries to exploratory research, and a particular visual analytic which has been shown to have demonstrable impact on learners. We conclude by summarising the ongoing research and development programme and identifying the challenges of integrating traditional social science research, with learning analytics and modelling.
Exploring reflection in online communities BIBAFull-Text 102-110
  John McAuley; Alexander O'Connor; Dave Lewis
Commons-based Peer Production is the process by which internet communities create media and software artefacts. Learning is integral to the success of these communities as it encourages contribution on an individual level, helps to build and sustain commitment on a group level and provides a means for adaption at an organisational level. While some communities have established ways to support organisational learning -- through a forum or thread reserved for community discussion -- few have investigated how more in-depth visual and analytic interfaces could help formalise this process. In this paper, we explore how social network visualisation can be used to encourage reflection and thus support organisational learning in online communities. We make the following contributions: First, we describe Commons-Based Peer Production, in terms of a socio-technical learning system that includes individual, group and organisational learning. Second, we present a novel visualisation environment that embeds social network visualisation in an asynchronous collaborative architecture. Third, we present results from an evaluation and discuss the potential for visualisation to support the process of organisational reflection in online communities.
Applying quantified self approaches to support reflective learning BIBAFull-Text 111-114
  Verónica Rivera-Pelayo; Valentin Zacharias; Lars Müller; Simone Braun
This paper presents a framework for technical support of reflective learning, derived from a unification of reflective learning theory with a conceptual framework of Quantified Self tools -- tools for collecting personally relevant information for gaining self-knowledge. Reflective learning means returning to and evaluating past experiences in order to promote continuous learning and improve future experiences. Whilst the reflective learning theories do not sufficiently consider technical support, Quantified Self (QS) approaches are rather experimental and the many emergent tools are disconnected from the goals and benefits of their use. This paper brings these two strands into one unified framework that shows how QS approaches can support reflective learning processes on the one hand and how reflective learning can inform the design of new QS tools for informal learning purposes on the other hand.
Learn-B: a social analytics-enabled tool for self-regulated workplace learning BIBAFull-Text 115-119
  Melody Siadaty; Dragan Gaševic; Jelena Jovanovic; Nikola Milikic; Zoran Jeremic; Liaqat Ali; Aleksandar Giljanovic; Marek Hatala
In this design briefing, we introduce the Learn-B environment, our attempt in designing and implementing a research prototype to address some of the challenges inherent in workplace learning: the informal aspect of workplace learning requires knowledge workers to be supported in their self-regulatory learning (SRL) processes, whilst its social nature draws attention to the role of collective in those processes. Moreover, learning at workplace is contextual and on-demand, thus requiring organizations to recognize and motivate the learning and knowledge building activities of their employees, where individual learning goals are harmonized with those of the organization. In particular, we focus on the analytics-based features of Learn-B, illustrate their design and current implementation, and discuss how each of them is hypothesized to target the above challenges.

Institutional perspectives

The pulse of learning analytics understandings and expectations from the stakeholders BIBAFull-Text 120-129
  Hendrik Drachsler; Wolfgang Greller
While there is currently much buzz about the new field of learning analytics [19] and the potential it holds for benefiting teaching and learning, the impression one currently gets is that there is also much uncertainty and hesitation, even extending to scepticism. A clear common understanding and vision for the domain has not yet formed among the educator and research community. To investigate this situation, we distributed a stakeholder survey in September 2011 to an international audience from different sectors of education. The findings provide some further insights into the current level of understanding and expectations toward learning analytics among stakeholders. The survey was scaffolded by a conceptual framework on learning analytics that was developed based on a recent literature review. It divides the domain of learning analytics into six critical dimensions. The preliminary survey among 156 educational practitioners and researchers mostly from the higher education sector reveals substantial uncertainties in learning analytics.
   In this article, we first briefly introduce the learning analytics framework and its six domains that formed the backbone structure to our survey. Afterwards, we describe the method and key results of the learning analytics questionnaire and draw further conclusions for the field in research and practice. The article finishes with plans for future research on the questionnaire and the publication of both data and the questions for others to utilize.
Learning analytics: challenges, paradoxes and opportunities for mega open distance learning institutions BIBAFull-Text 130-133
  Paul Prinsloo; Sharon Slade; Fenella Galpin
Despite all the research on student retention and success since the first conceptual mappings of student success e.g. Spady [12], there have not been equal impacts on the rates of both student success and retention. To realise the potential of learning analytics to impact on student retention and success, mega open distance learning (ODL) institutions face a number of challenges, paradoxes and opportunities.
   For the purpose of this paper we critique a 'closed' view of learning analytics as focusing only on data produced by students' interactions with institutions of higher learning. Students are not the only actors in their learning journeys and it would seem crucial that learning analytics also includes the impacts of all stakeholders on students' learning journeys in order to increase the success of students' learning. As such the notion of 'Thirdspace' as used by cultural, postmodern and identity theorists provide a useful heuristic to map the challenges and opportunities, but also the paradoxes of learning analytics and its potential impact on student success and retention.
   This paper explores some of these challenges, paradoxes and opportunities with reference to two mega ODL institutions namely the Open University in the UK (OU) and the University of South Africa (Unisa). Although these two institutions share a number of characteristics, there are also some major and important differences between them. We explore some of the shared challenges, paradoxes and opportunities learning analytics offer in the context of these two institutions.
The learning analytics cycle: closing the loop effectively BIBAFull-Text 134-138
  Doug Clow
This paper develops Campbell and Oblinger's [4] five-step model of learning analytics (Capture, Report, Predict, Act, Refine) and other theorisations of the field, and draws on broader educational theory (including Kolb and Schön) to articulate an incrementally more developed, explicit and theoretically-grounded Learning Analytics Cycle.
   This cycle conceptualises successful learning analytics work as four linked steps: learners (1) generating data (2) that is used to produce metrics, analytics or visualisations (3). The key step is 'closing the loop' by feeding back this product to learners through one or more interventions (4).
   This paper seeks to begin to place learning analytics practice on a base of established learning theory, and draws several implications from this theory for the improvement of learning analytics projects. These include speeding up or shortening the cycle so feedback happens more quickly, and widening the audience for feedback (in particular, considering learners and teachers as audiences for analytics) so that it can have a larger impact.
Mining academic data to improve college student retention: an open source perspective BIBAFull-Text 139-142
  Eitel J. M. Lauría; Joshua D. Baron; Mallika Devireddy; Venniraiselvi Sundararaju; Sandeep M. Jayaprakash
In this paper we report ongoing research on the Open Academic Analytics Initiative (OAAI), a project aimed at increasing college student retention by performing early detection of academic risk using data mining methods. The paper describes the goals and objectives of the OAAI, and lays out a methodological framework to develop models that can be used to perform inferential queries on student performance using open source course management system data and student academic records. Preliminary results on initial model development using several data mining algorithms for classification are presented.

Visual analytics

Goal-oriented visualizations of activity tracking: a case study with engineering students BIBAFull-Text 143-152
  Jose Luis Santos; Sten Govaerts; Katrien Verbert; Erik Duval
Increasing motivation of students and helping them to reflect on their learning processes is an important driver for learning analytics research. This paper presents our research on the development of a dashboard that enables self-reflection on activities and comparison with peers. We describe evaluation results of four iterations of a design based research methodology that assess the usability, use and usefulness of different visualizations. Lessons learned from the different evaluations performed during each iteration are described. In addition, these evaluations illustrate that the dashboard is a useful tool for students. However, further research is needed to assess the impact on the learning process.
Seeing what the system thinks you know: visualizing evidence in an open learner model BIBAFull-Text 153-157
  Barbara Kump; Christin Seifert; Guenter Beham; Stefanie N. Lindstaedt; Tobias Ley
User knowledge levels in adaptive learning systems can be assessed based on user interactions that are interpreted as Knowledge Indicating Events (KIE). Such an approach makes complex inferences that may be hard to understand for users, and that are not necessarily accurate. We present MyExperiences, an open learner model designed for showing the users the inferences about them, as well as the underlying data. MyExperiences is one of the first open learner models based on tree maps. It constitutes an example of how research into open learner models and information visualization can be combined in an innovative way.
Student success system: risk analytics and data visualization using ensembles of predictive models BIBAFull-Text 158-161
  Alfred Essa; Hanan Ayad
We propose a novel design of a Student Success System (S3), a holistic analytical system for identifying and treating at-risk students. S3 synthesizes several strands of risk analytics: the use of predictive models to identify academically at-risk students, the creation of data visualizations for reaching diagnostic insights, and the application of a case-based approach for managing interventions. Such a system poses numerous design, implementation, and research challenges. In this paper we discuss a core research challenge for designing early warning systems such as S3. We then propose our approach for meeting that challenge. A practical implementation of an student risk early warning system, utilizing predictive models, must meet two design criteria: a) the methodology for generating predictive models must be flexible to allow generalization from one context to another; b) the underlying mechanism of prediction should be easily interpretable by practitioners whose end goal is to design meaningful interventions on behalf of students. Our proposed solution applies an ensemble method for predictive modeling using a strategy of decomposition. Decomposition provides a flexible technique for generating and generalizing predictive models across different contexts. Decomposition into interpretable semantic units, when coupled with data visualizations and case management tools, allows practitioners, such as instructors and advisors, to build a bridge between prediction and intervention.
GLASS: a learning analytics visualization tool BIBAFull-Text 162-163
  Derick Leony; Abelardo Pardo; Luis de la Fuente Valentín; David Sánchez de Castro; Carlos Delgado Kloos
The use of technology in every day tasks enables the possibility to collect large amounts of observations of events taking place in different environments. Most tools are capable of storing a detailed account of the operations executed by users in certain files commonly known as logs. These files can be further analyzed to infer information that is not directly visible such as the most popular applications, times of the day with highest activity, calories burnt after a running session, etc. Graphic visualizations of this data can be used to support this type of analysis as shown in [1]. Visualization can also be applied in the domain of learning experiences to track and analyse the data obtained from both learners and instructors. There are several tools that have been proposed in specific environments such as, for example, in personal learning environments [5], to foster self-reflection and awareness [2], and to support instructors in web-based distance learning [3]. These visualizations need to take into account aspects such as how to access and protect personal data, filter management, multi-user support and availability. In this paper, the web-based visualization platform GLASS (Gradient's Learning Analytics System) is presented. The architecture of the tool has been conceived to support a large number of modular visualizations derived from a common dataset containing a large number of recorded events. The tool was developed following a bottom-up methodology to provide a set of basic operations required by any visualization. The design goal is to provide a highly versatile, modular platform that simplifies the implementation of new visualizations.
   The main functionality elements considered in GLASS are database access, module management, visualization parameters, and the web interface. The platform uses datasets stored using the CAM schema (Contextualized Attention Metadata) [6]. This schema allows to capture events occurring during the use of various computer applications which, in our case, are the tools used by students when working in a learning environment. The process to obtain events from learning environments has been described in [4]. GLASS is able to connect to more than one CAM database, thus allowing access to events obtained in different contexts.
   The tool is extensible through the installation of modules. A module is a structured set of scripts and resources that, given a dataset of events and a set of filters, generates at least one visualization. In order to simplify the development of new modules, the platform provides an API to manage common visualizations settings such as the date range and other typical filters. A visualization may include a simpler version suitable to be displayed in the user's Dashboard, which is the entry page of the application. Figure 1 shows an example of dashboard in GLASS. Additionally, visualizations can be exported as HTML code to be embedded in another website.
   The GLASS architecture consists of four layers: data layer, code base, modules and visualizations, as depicted in Figure 2. The data layer is composed of a set of CAM databases and a database to store the platform parameters. The code base is in charge of the main functionalities of GLASS regarding module and user management and interfaces. Modules must comply with the platform specifications to generate visualizations and the settings that can affect their appearance. Currently, the tool includes a default module that provides two visualizations as shown in Figure 1): a frequency time line of activity events and a bar-chart with grouped bars of events generated by different user groups (e.g. events from students individually, or groups). The default module also serves as an example of how to develop a additional modules.
   Currently, GLASS is able to support new visualizations and is undergoing additional testing in different learning scenarios. Preliminary results obtained from user tests indicate that visualizations need to be very intuitive for both instructors and learners. The current development effort is focused on providing visualizations that show the most-common learners events and the most active learners in a given context. To encourage its use in other institutions, the tool has been released with an open source license and can be obtained from http://glass.mozart.gast.it.uc3m.es. A video demonstrating the tool is available at http://bit.ly/glass-lak12.

Educator interventions

Applying artificial intelligence to the educational data: an example of syllabus quality analysis BIBAFull-Text 164-169
  Denis Smolin; Sergey Butakov
Developing new courses and updating existing ones are routine activities for an educator. The quality of a new or updated course depends on the course structure as well as its individual elements. The syllabus defines the structure and the details of the course, thus contributing to the overall quality of the course. This research proposes a new AI based framework to manage the quality of the syllabus. We apply AI methods to automatically evaluate a syllabus on the basis of such characteristics as validity, usability, and efficiency. We provide user trials to show the advantages of the developed approach against the traditional human-based process of syllabi verification and evaluation.
Educational monitoring tool based on faceted browsing and data portraits BIBAFull-Text 170-178
  David García-Solórzano; Germán Cobo; Eugènia Santamaría; Jose Antonio Morán; Carlos Monzo; Javier Melenchón
Due to the idiosyncrasy of online education, students may become disoriented, frustrated or confused if they do not receive the support, feedback or guidance needed to be successful. To avoid this, the role of teachers is essential. In this regard, instructors should be facilitators who guide students throughout the teaching-learning process and arrange meaningful learner-centered experiences. However, unlike face-to-face classes, teachers have difficulty in monitoring their learners in an online environment, since a lot of learning management systems provide faculty with student tracking data in a poor tabular format that is difficult to understand. In order to overcome this drawback, this paper presents a novel graphical educational monitoring tool based on faceted browsing that helps instructors to gain an insight into their classrooms' performance. Moreover, this tool depicts information of each individual student by using a data portrait. Thanks to this monitoring tool, teachers can, on the one hand, track their students during the teaching-learning process and, on the other, detect potential problems in time.
Exploring qualitative analytics for e-mentoring relationships building in an online social learning environment BIBAFull-Text 179-183
  Haiming Liu; Ronald Macintyre; Rebecca Ferguson
The language of mentoring has become established within the workplace and has gained ground within education. As work based education moves online so we see an increased use of what is termed e-mentoring. In this paper we explore some of the challenges of forming and supporting mentoring relationships virtually, and we explore the solutions afforded by online social learning and Web 2.0. Based on a conceptualization of learning network theory derived from the literature and the qualitative learning analytics, we propose that an e-mentoring relationships is mediated by a connection with or through a person or learning objects. We provide an example to illustrate how this might work.
Bridging the gap from knowledge to action: putting analytics in the hands of academic advisors BIBAFull-Text 184-187
  Steven Lonn; Andrew E. Krumm; R. Joseph Waddington; Stephanie D. Teasley
This paper presents current findings from an ongoing design-based research project aimed at developing an early warning system (EWS) for academic mentors in an undergraduate engineering mentoring program. This paper details our progress in mining Learning Management System data and translating these data into an EWS for academic mentors. We focus on the role of mentors and advisors, and elaborate on their importance in learning analytics-based interventions developed for higher education.

Textual analytics & analytics infrastructure

Using computational methods to discover student science conceptions in interview data BIBAFull-Text 188-197
  Bruce Sherin
A large body of research in the learning sciences has focused on students' commonsense science knowledge -- the everyday knowledge of the natural world that is gained outside of formal instruction. Although researchers studying commonsense science have employed a variety of methods, one-on-one clinical interviews have played a unique and central role. The data that result from these interviews take the form of video recordings, which in turn are often compiled into written transcripts, and coded by human analysts. In my team's work on learning analytics, we draw on this same type of data, but we attempt to automate its analysis. In this paper, I describe the success we have had using extremely simple methods from computational linguistics -- methods that are based on rudimentary vector space models and simple clustering algorithms. These automated analyses are employed in an exploratory mode, as a way to discover student conceptions in the data. The aims of this paper are primarily methodological in nature; I will attempt to show that it is possible to use techniques from computational linguistics to analyze data from commonsense science interviews. As a test bed, I draw on transcripts of a corpus of interviews in which 54 middle school students were asked to explain the seasons.
Deriving group profiles from social media to facilitate the design of simulated environments for learning BIBAFull-Text 198-207
  Ahmad Ammari; Lydia Lau; Vania Dimitrova
Simulated environments for learning are becoming increasingly popular to support experiential learning in complex domains. A key challenge when designing simulated learning environments is how to align the experience in the simulated world with real world experiences. Social media resources provide user-generated content that is rich in digital traces of real world experiences. People comments, tweets, and blog posts in social spaces can reveal interesting aspects of real world situations or can show what particular group of users is interested in or aware of. This paper examines a systematic way to analyze user-generated content in social media resources to provide useful information for learning simulator design. A hybrid framework exploiting Machine Learning and Semantics for social group profiling is presented. The framework has five stages: (1) Retrieval of user-generated content from the social resource (2) Content noise filtration, removing spam, abuse, and content irrelevant to the learning domain; (3) Deriving individual social profiles for the content authors; (4) Clustering of individuals into groups of similar authors; and (5) Deriving group profiles, where interesting concepts suitable for the use in simulated learning systems are extracted from the aggregated content authored by each group. The framework is applied to derive group profiles by mining user comments on YouTube videos. The application is evaluated in an experimental study within the context of learning interpersonal skills in job interviews. The paper discusses how the YouTube-based group profiles can be used to facilitate the design of a job interview skills learning simulator, considering: (1) identifying learning needs based on digital traces of real world experiences; and (2) augmenting learner models in simulators based on group characteristics derived from social media.
The learning registry: building a foundation for learning resource analytics BIBAFull-Text 208-211
  Marie Bienkowski; John Brecht; Jim Klo
We describe our experimentation with the current implementation of a distribution system used to share descriptive and social metadata about learning resources. The Learning Registry, developed and released in a beta version in October 2011, is intended to store and forward learning-resource metadata among a distributed, de-centralized network of nodes. The Learning Registry also accepts social/attention metadata -- data about users of and activity around the learning resource. The Learning Registry open-source community has proposed a schema for sharing social metadata, and has experimented with a number of organizations representing their social metadata using that schema. This paper describes the results and challenges, and the learning-resource analytics applications that will use Learning Registry data as their foundation.

Empirical studies

Monitoring student progress through their written "point of originality" BIBAFull-Text 212-221
  Jóhann Ari Lárusson; Brandon White
This paper describes a new method for the objective evaluation of student work through the identification of original content in writing assignments. Using WordNet as a lexical reference, this process allows instructors to track how key phrases are employed and evolve over the course of a student's writing, and to automatically visualize the point at which the student's language first demonstrates original thought, phrased in their own, original words. The paper presents a case study where the analysis method was evaluated by analyzing co-blogging data from a reading and writing intensive undergraduate course. The evidence shows that the tool can be predictive of students' writing in a manner that correlates with their progress in the course and engagement in the technology-mediated activity. By visualizing otherwise subjective information in a way that is objectively intelligible, the goal is to provide educators with the ability to monitor student investment in concepts from the course syllabus, and to extend or modify the boundaries of the syllabus in anticipation of pre-existing knowledge or trends in interest. A tool of this sort can be of value particularly in larger gateway courses, where the sheer size of the class makes the ongoing evaluation of student progress a daunting if not otherwise impossible task.
Learning analytics for collaborative writing: a prototype and case study BIBAFull-Text 222-225
  Brian J. McNely; Paul Gestwicki; J. Holden Hill; Philip Parli-Horne; Erika Johnson
This paper explores the ways in which participants in writing intensive environments might use learning analytics to make productive interventions during, rather than after, the collaborative construction of written artifacts. Specifically, our work considered how university students learning in a knowledge work model -- one that is collaborative, project-based, and that relies on consistent peer-to-peer interaction and feedback -- might leverage learning analytics as formative assessment to foster metacognition and improve final deliverables. We describe Uatu, a system designed to visualize the real time contribution and edit history of collaboratively written documents. After briefly describing the technical details of this system, we offer initial findings from a fifteen week qualitative case study of 8 computer science students who used Uatu in conjunction with Google Docs while collaborating on a variety of writing and programming tasks. These findings indicate both the challenges and promise of delivering useful metrics for collaborative writing scenarios in academe and industry.
The relationship between educational performance and online access routines: analysis of students' access to an online discussion forum BIBAFull-Text 226-229
  Tariq M. Khan; Fintan Clear; Samira Sadat Sajadi
A study of behaviour patterns associated with students accessing an online discussion forum is presented. Data collected on the frequency of access and the duration of sessions is analysed to establish several categories of learners, which depict the differences among the cohort in terms of participation in social learning. A British business school course for second year undergraduates was studied over two years (i.e. two cohorts) and the results were combined to derive categories of learner types. We conclude that that academic attainment does not appear to be related to student access behaviour necessarily.
Investigating the core group effect in usage of resources with analytics BIBAFull-Text 230-233
  Agathe Merceron
In many educational institutions, face to face as well as on-line teaching is supported by the use of a Learning Management System (LMS). To be able to analyze better data stored by LMS, we have started developing a dedicated tool for this purpose. While analyzing usage data with teachers, we have noticed that the number of students attempting non self-tests decreases during the semester. Teachers were interested in investigating this pattern further to uncover the strategy adopted by students. In this paper, we explain our approach to investigate the core group effect in resources usage: given a set of resources, is a group of students emerging that continuously uses the resources or, on the contrary, are the resources used on an irregular basis by different students? We answer this question checking the confidence of what we call local rules and global rules. We show a case study conducted with our analysis tool as a first step to validate our approach.
Does the length of time off-task matter? BIBAFull-Text 234-237
  Daniel Roberge; Anthony Rojas; Ryan Baker
We investigate the relationship between a student's time off-task and the amount that he or she learns to see whether or not the relationship between time off-task and learning is a more complex model than the traditional linear model typically studied. The data collected is based off of students' interactions with Cognitive Tutor learning software. Analysis suggested that more complex functions did not fit the data significantly better than a linear function. In addition, there was not evidence that the length of a specific pause matters for predicting learning outcomes; e.g. students who make many short pauses do not appear to learn more or less than students who make a smaller number of long pauses. As such, previous theoretical accounts arguing that off-task behavior primarily reduces learning by reducing the amount of time spent learning remain congruent with the current evidence.

Educational data mining

Clustering by usage: higher order co-occurrences of learning objects BIBAFull-Text 238-247
  Katja Niemann; Hans-Christian Schmitz; Uwe Kirschenmann; Martin Wolpers; Anna Schmidt; Tim Krones
In this paper, we introduce a new way of detecting semantic similarities between learning objects by analyzing their usage in a web portal. Our approach does not rely on the content of the learning objects or on the relations between the users and the learning objects but on usage-based relations between the objects themselves. The technique we apply for calculating higher order co-occurrences to create semantically homogenous clusters of data objects is taken from corpus driven lexicology where it is used to cluster words. We expect the members of a higher order co-occurrence class to be similar according to their content and present the evaluations of that assumption using two teaching and learning systems.
Using agglomerative hierarchical clustering to model learner participation profiles in online discussion forums BIBAFull-Text 248-251
  Germán Cobo; David García-Solórzano; Jose Antonio Morán; Eugènia Santamaría; Carlos Monzo; Javier Melenchón
Online discussion forums are a key element in virtual learning environments. The way learners participate in discussion boards can be a very useful source of indicators for teachers to facilitate their tasks. The use of a two-stage analysis strategy based on an agglomerative hierarchical clustering algorithm is proposed in this paper to identify different participation profiles adopted by learners in online discussion forums. Different parameters are used to characterize learners' activity (amount of posts, rhythm, depth of threads, crossed replies, etc). Participation profiles are identified and analyzed in terms of behavior and performance.
Learning analytics and educational data mining: towards communication and collaboration BIBAFull-Text 252-254
  George Siemens; Ryan S. J. d. Baker
Growing interest in data and analytics in education, teaching, and learning raises the priority for increased, high-quality research into the models, methods, technologies, and impact of analytics. Two research communities -- Educational Data Mining (EDM) and Learning Analytics and Knowledge (LAK) have developed separately to address this need. This paper argues for increased and formal communication and collaboration between these communities in order to share research, methods, and tools for data mining and analysis in the service of developing both LAK and EDM fields.

Predictive modeling

Probability estimation and a competence model for rule based e-tutoring systems BIBAFull-Text 255-258
  Diederik M. Roijers; Johan Jeuring; Ad Feelders
In this paper, we present a student model for rule based e-tutoring systems. This model describes both properties of rewrite rules (difficulty and discriminativity) and of students (start competence and learning speed). The model is an extension of the two-parameter logistic ogive function of Item Response Theory. We show that the model can be applied even to relatively small datasets. We gather data from students working on problems in the logic domain, and show that the model estimates of rule difficulty correspond well to expert opinions. We also show that the estimated start competence corresponds well to our expectations based on the previous experience of the students in the logic domain. We point out that this model can be used to inform students about their competence and learning, and teachers about the students and the difficulty and discriminativity of the rules.
Course correction: using analytics to predict course success BIBAFull-Text 259-262
  Rebecca Barber; Mike Sharkey
Predictive analytics techniques applied to a broad swath of student data can aid in timely intervention strategies to help prevent students from failing a course. This paper discusses a predictive analytic model that was created for the University of Phoenix. The purpose of the model is to identify students who are in danger of failing the course in which they are currently enrolled. Within the model's architecture, data from the learning management system (LMS), financial aid system, and student system are combined to calculate a likelihood of any given student failing the current course. The output can be used to prioritize students for intervention and referral to additional resources. The paper includes a discussion of the predictor and statistical tests used, validation procedures, and plans for implementation.
Predicting failure: a case study in co-blogging BIBAFull-Text 263-266
  Bjorn Levi Gunnarsson; Richard Alterman
Monitoring student progress in homework is important but difficult to do. The work in this paper presents a method for monitoring student progress based on their participation. By tracking participation we can successfully create a model that predicts, with very high accuracy, if a student is going to score a low grade on her current assignment before it is completed, thus enabling selective interventions.
Course signals at Purdue: using learning analytics to increase student success BIBAFull-Text 267-270
  Kimberly E. Arnold; Matthew D. Pistilli
In this paper, an early intervention solution for collegiate faculty called Course Signals is discussed. Course Signals was developed to allow instructors the opportunity to employ the power of learner analytics to provide real-time feedback to a student. Course Signals relies not only on grades to predict students' performance, but also demographic characteristics, past academic history, and students' effort as measured by interaction with Blackboard Vista, Purdue's learning management system. The outcome is delivered to the students via a personalized email from the faculty member to each student, as well as a specific color on a stoplight -- traffic signal -- to indicate how each student is doing. The system itself is explained in detail, along with retention and performance outcomes realized since its implementation. In addition, faculty and student perceptions will be shared.