HCI Bibliography Home | HCI Conferences | HCIR Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
HCIR Tables of Contents: 07080910111213

Proceedings of the Workshop on Human-Computer Interaction and Information Retrieval

Fullname:Proceedings of the Workshop on Human-Computer Interaction and Information Retrieval
Editors:Bill Kules; Daniel Tunkelang; Ryen White
Location:Washington, DC
Standard No:hcibib: HCIR09
Links:Conference Website and Proceedings | Papers | Posters | Full Proceedings
Summary:When we held the first HCIR workshop in 2007, the idea of uniting the fields of Human-Computer Interaction (HCI) and Information Retrieval (IR) was a battle cry to move this research area from the fringes of computer science into the mainstream. Two years later, as we organize this third HCIR workshop on the heels of a highly successful HCIR 2008, we see some of the fruits of our labor. Topics like interactive information retrieval and exploratory search are receiving increasing attention, among both academic researchers and industry practitioners.
    But we have only begun this journey. Most of the work in these two fields still stays within their silos, and the efforts to realize more sophisticated models, tools, and evaluation metrics for information seeking are still in their early stages.
    In this year's one-day workshop, we will continue to explore the advances each domain can bring to the other.
  1. Papers
  2. Posters/Demos


Usefulness as the Criterion for Evaluation of Interactive Information Retrieval BIBA 1-4
  Michael Cole; Jingjing Liu; Nicholas Belkin; Ralf Bierig; Jacek Gwizdka; Chang Liu; Jun Zhang; Xiangmin Zhang
The purpose of an information retrieval (IR) system is to help users accomplish a task. IR system evaluation should consider both task success and the value of support given over the entire information seeking episode. Relevance-based measurements fail to address these requirements. In this paper, usefulness is proposed as a basis for IR evaluation.
Modeling Searcher Frustration BIBA 5-8
  Henry Feild; James Allan
When search engine users have trouble finding what they are looking for, they become frustrated. In a pilot study, we found that 36% of queries submitted end with users being moderately to extremely frustrated. By modeling searcher frustration, search engines can predict the current state of user frustration, tailor the search experience to help the user find what they are looking for, and avert them from switching to another search engine. Among other observations, we found that across the fifteen users and six tasks in our study, frustration follows a law of conservation: a frustrated user tends to stay frustrated and a non-frustrated user tends to stay not frustrated. Future work includes extracting features from the query log data and readings from three physical sensors collected during the study to predict searcher frustration.
Query Suggestions as Idea Tactics for Information Search BIBA 9-12
  Diane Kelly
This paper explores the thesis that query suggestions function as a type of idea tactic. A further thesis of this paper is that query suggestions are most useful for open-ended search tasks that require the searcher to explore and learn about a particular topic, and in situations where topics are difficult. Results are presented from two studies that examined people's use of query suggestions while searching for open-ended search topics and how usage varied according to topic difficulty.
I Come Not to Bury Cranfield, but to Praise It BIBA 13-16
  Ellen Voorhees
Much information retrieval research is currently performed using test collections, a methodology introduced by Cleverdon and his colleagues in the Cranfield tests [4] and further refined in evaluation exercises such as TREC (http: //trec.nist.gov/). A test collection is (purposely) a stark abstraction of real user search tasks that models only a few of the variables that affect search behavior and was explicitly designed to minimize individual searcher effects. Nonetheless, I argue that Cranfield-style experimentation is critical to the study of interactive (user-in-the-loop) retrieval for at least two reasons. First, research using test collections identifies good retrieval technology, allowing expensive user testing to be reserved for the most promising avenues. Second, meta-analysis of the Cranfield methodology can inform the development of new research abstractions that make different trade-offs among realism, experimental power, and cost.
   This paper is a condensation of an earlier paper in which I made similar arguments [12]. The arguments stem from my experience with building and validating test collections as the manager of the TREC project, a role that clearly marks me as a Cranfield advocate. The next section provides a brief recap of the Cranfield methodology as currently practiced, the following section examines the immense impact of user variability on retrieval experiments, and the final section describes two prior TREC efforts to develop evaluation paradigms in the space between test collections and full interactive experiments. The paper does not contain a proposal for a new abstraction for testing interactive retrieval systems; that is very much an open research problem. Instead, my hope is that the paper clarifies some of the issues that must be addressed by such an abstraction.
Search Tasks and Their Role in Studies of Search Behaviors BIBA 17-21
  Barbara Wildemuth; Luanne Freund
In experimental studies of search behaviors and evaluations of information retrieval systems, researchers generally assign search tasks to the subjects to perform. Since it can be expected that the tasks themselves will influence search behaviors and performance, we need to be able to construct tasks having particular attributes, knowing that our study findings can then be generalized to all search tasks having those attributes. In this paper, we report on an ongoing analysis of the search tasks that have been used in experimental search studies. We review a number of typologies of search tasks currently in use (complex vs. simple, specific vs. general, exploratory vs. lookup, and navigational vs. informational) and make recommendations for designing search tasks for use in future studies.


Visual Interaction for Personalized Information Retrieval BIBA 22-25
  Jae-wook Ahn; Peter Brusilovsky
There are two promising answers to the classic information overload problem: (1) personalized search and (2) exploratory search. Personalized search stresses more on the algorithmic side and the exploratory search pays more attention on the user interface to help users to achieve better search results. We believe that by combining these two approaches, we can provide users with a better solution than the old ranked list based interaction mechanism of personalized search. This paper proposes to incorporate interactive visualization into personalized search. We extended a well known visualization method called VIBE (Visual Information Browsing Environment) to visualize user models and then incorporated it into the personalized search framework. We expect our approach will be able to help users to better explore the information space and locate relevant information more efficiently. Here, we showed the concept and the potential of this adaptive visualization method. We also introduced a new prototypical personalized search system and its interaction model implementing the adaptive visualization idea.
PuppyIR: Designing an Open Source Framework for Interactive Information Services for Children BIBA 26-29
  Leif Azzopardi; Richard Glassey; Mounia Lalmas; Tamara Polajnar; Ian Ruthven
One of the main aims of the PuppyIR project is to provide an open source framework for the development of Interactive Information Retrieval Services. The main focus of the project is directed towards developing such services for children, which introduces a number of novel and challenging issues to address (such as language development, security, moderation, etc).
   In this poster paper, we outline the preliminary high-level design of the open source framework. The framework uses a layered architecture to minimize dependencies between the user-side concerns of interaction and presentation, and the system-side concerns of aggregating content from multiple sources and processing information appropriately. Each layer will consist of a series of interchangeable components, which can be interconnected to form a complete service. To facilitate the construction of diverse information services, a dataflow language is proposed to enable the assembly of the components in an intuitive and visual manner. One of the design goals of the architecture, and ultimate measures of success, is to provide a "lego" style building block environment in which researchers and developers of any age can build their own information service.
   This poster provides the starting point for the design of the framework and aims to seek comments, feedback and suggestions from the community in order to improve and refine the architecture.
Designing an Interactive Automatic Document Classification System BIBA 30-33
  Kirk Baker
In this paper we report on a series of completed and ongoing experiments that involve the integration of fully automatic document classification techniques into an existing manually-oriented document retrieval system. We take our primary findings as positions on the design of an interactive document classifier and retrieval tool.
Graphic User Interface for Content and Structure Queries in XML Retrieval BIBA 34-37
  Luis M. de Campos; Juan M. Fernández-Luna; Juan F. Huete; Carlos J. Martín-Dancausa
Structured Information Retrieval works with documents internally organised around a well defined structure, typically XML documents. In this research field, documents are not retrieved as a whole, but only those most specific relevant parts of the documents are delivered to the users. Lots of models have been developed to deal with the new dimension of the internal organization. From the point of view of the users, the document structure should be an added value in order to retrieve relevant material, because they are able to specify structural hints, in the form of the types of elements to be retrieved and as restrictions over some elements. There are several ways to query a system specifying content and structure queries (natural and artificial languages), but few of them rely on graphic user interfaces, supporting the users to create queries that fulfil more accurately their information needs. In this paper, we present a graphic user interface with the aim of formulating these types of queries, where the users only have to state what they wish to retrieve and structural restrictions about it.
The HCI Browser Tool for Studying Web Search Behavior BIBA 38-41
  Robert Capra
In this paper, we introduce the HCI Browser, a Mozilla Firefox extension designed to support studies of Web search behaviors. The HCI Browser presents tasks to the user, collects browser event data as the user searches for information, records answers that are found, and administers pre- and post-task questionnaires. The HCI Browser is a configurable tool that HCI and IR researchers can use to conduct studies and gather data about users' Web information seeking behaviors. It is especially well suited for "batch mode" laboratory studies in which multiple participants complete a study at the same time, but work independently. The HCI Browser is open-source software and is available for download at: http://ils.unc.edu/hcibrowser
Improving Search-Driven Development with Collaborative Information Retrieval Techniques BIBA 42-45
  Juan M. Fernández-Luna; Luis M. de Campos; Juan F. Huete; Carlos J. Martin-Dancausa
Software developers frequently spend time searching for information, generally source-code. In the last few years this habit has increased the community's interest to improve it and some are staring to refer to as Search-Driven Development (SDD). In this work we examine the SDD as a collaborative and commonplace task. However, current integrated development environments (IDEs) do not include information retrieval systems with support for explicit collaboration among developers with shared technical information need. We then introduce PosseSrc, a prototype outside the IDEs that enables teams of remote developers to collaborate in real time during the search sessions. PosseSrc improve the SDD by supporting several modern state-of-the-art collaborative information retrieval (CIR) techniques such as session persistence, division of labor, sharing of knowledge and group awareness.
A visualization interface for interactive search refinement BIBA 46-49
  Fernando Figueira Filho; João Porto de Albuquerque; André Resende; Paulo Lício de Geus; Gary Olson
It is common practice nowadays to find, assess and explore the Web by groping scattered information presented through many search results. Browsing interfaces and query suggestion techniques attempt to guide the user by providing term recommendations and query phrases. In this paper, we introduce the browsing interface of Kolline, a community search engine under development. Two case studies are described and two distinct web browsing interfaces are analyzed. Based on this analysis, we present a new browsing interface, describing our design decisions and providing directions for future work.
Cognitive Dimensions Analysis of Interfaces for Information Seeking BIBA 50-53
  Gene Golovchinsky
Cognitive Dimensions is a framework for analyzing human-computer interaction. It is used for meta-analysis, that is, for talking about characteristics of systems without getting bogged down in details of a particular implementation. In this paper, I discuss some of the dimensions of this theory and how they can be applied to analyze information seeking interfaces. The goal of this analysis is to introduce a useful vocabulary that practitioners and researchers can use to describe systems, and to guide interface design toward more usable and useful systems.
Cognitive Load and Web Search Tasks BIBA 54-57
  Jacek Gwizdka
Assessing cognitive load on web search is useful for characterizing search system features, search tasks and task stages with respect to their demands on the searcher's mental effort. It is also helpful in examining how individual differences among searchers (e.g. cognitive abilities) affect the search process and its outcomes. We discuss assessment of cognitive load from the perspective of primary and secondary task performance. Our discussion is illustrated by results from a controlled web search study (N=48). No relationship was found between objective task difficulty and performance on the secondary task. There was, however, a significant relationship between search task stages and performance on the secondary task.
Visualising Digital Video Libraries for TV Broadcasting Industry: A User-Centred Approach BIBA 58-61
  Mieke Haesen; Jan Meskens; Karin Coninx
Finding a suitable video fragment in a vast video archive is mostly a complex task. Even professional users have to skim many hours of stored video data before they find the desired content. In this paper, we present a user-centred software engineering approach that is employed to create a novel news video explorer for TV broadcasting industry. This approach helps to ensure the balance between the technological progress in the field of information retrieval on the one hand and the needs and goals of the end users on the other hand.
Log Based Analysis of How Faceted and Text Based Searching Interact in a Library Catalog Interface BIBA 62-65
  Bradley Hemminger; Xi Niu; Cory Lown
Faceted based search is an increasingly common part of search interfaces. This study examines the use of a library catalog search interface which supports but text searching and faceted based searching. Log analysis is performed of library catalog search records to analyze how and when faceted based searching is used in conjunction with text based searching. The logs are from the Triangle Research Libraries Network, which all use an Endeca based catalog search system. Results show that faceted based search is used much less frequently than text searching, and the usage clusters into certain categories of search behaviors.
Freebase Cubed: Text-based Collection Queries for Large, Richly Interconnected Data Sets BIBA 66-69
  David Huynh
Any large data set such as Freebase that contains a large number of types and properties accumulated over actual use rather than fixed at design time poses challenges to designing easy-to-use faceted browsers. This is because the faceted browser cannot be tuned with domain knowledge at design time, but must operate in a generic manner, and thus become unwieldy.
   In this work, we propose that support for a particular kind of text-based queries can let users perform faceted browsing and set-based browsing operations on such data sets with the ease and familiarity of conventional keyword search. For example, the text query "German car companies founders" can replace the actions of filtering all Freebase data by type to "company", by industry to "car", and by country to "Germany", and then pivoting to those companies' founders. From there, the user can perform faceted browsing actions to refine the already narrow collection further. We describe an algorithm for parsing these collection queries and demonstrate an implementation that works on Freebase.
System Controlled Assistance for Improving Search Performance BIBA 70-73
  Bernard Jansen
This position paper outlines the concept of system assistance as a method to improve searching performance. I present an investigation concerning the effects of user-controlled versus system-controlled assistance on searching performance using a within subjects, counterbalanced empirical evaluation. Forty-three subjects interacted with two fully functional, information retrieval systems offering searching assistance based on implicit feedback. The systems were identical in all respects except that one offered searching assistance via a help link, and the other offered system-controlled support at specified points during the search progress based on patterns of searcher interactions. The evaluation used the W2G Text REtrieval Conference document collection with six topics. Research results indicate that offering system-controlled assistance based on patterns of implicit feedback can improve searching performance based on user selected relevant documents, with an approximately 30% performance increase overall. I discuss the implications for the design of future searching systems with assistance that is based on user implicit feedback patterns.
Designing for Enterprise Search in a Global Organization BIBA 74-77
  Maria Johansson; Lina Westerling
Enterprise Search is used by organizations to capitalize on their internal knowledge by providing quick access to all internal information, helping users re-finding and discovering new information, as well as creating the necessary conditions for collaboration across organizational and geographical boundaries. In this large organization a search application was created to meet these goals. This paper focuses on the main design concepts of the second release of the search application, and how these were affected by experiences gained throughout the project. This design focused on simplicity and discoverability. Preliminary results show that the design is usable and that users find it easier to find the information they are looking for. A general increase in user satisfaction is also established.
Cultural Differences in Information Behavior BIBA 78-81
  Anita Komlodi; Karoly Hercegfi
With the availability of online translation services and the large amount of English-language content on the Web, more and more global users come in contact with content that was not created in their own language or culture. While some sites make efforts to localize their user interfaces and content, many simply translate content and use the same user interface. This is in direct contrast with findings that different cultures approach knowledge, information, and interaction with information in different ways. This paper will describe work in progress to study some national cultural differences in information behavior and the problems users face while interacting with information that was created in a language and culture different from their own.
Adapting an Information Visualization Tool for Mobile Information Retrieval BIBA 82-86
  Sherry Koshman; Jae-wook Ahn
The application of information visualization (infoviz) tools to mobile devices for information retrieval (IR) is uncommon. This has been attributed to the complex challenges related to mobile devices including the technical restrictions upon generating a small screen graphical visual representation for abstract information. This paper reports on a work in progress on the basic interface design and creation of MVIBE (Mobile VIBE), a new mobile version of VIBE (Visual Information Browsing Environment), which is an information visualization tool developed for information retrieval. MVIBE was developed and tested on the Apple, Linux, and Windows mobile platforms. User feedback was obtained and some of the reported challenges are common to mobile technology and others to general information visualization. At this early stage, the overall question is: can mobile devices be effective for generating a viable visualization of search results? The paper concludes with observations gained during the adaptation process, recommendations for the next phase of Mobile VIBE development, and future design considerations for developing information visualization interfaces on mobile devices.
A Theoretical Framework for Subjective Relevance BIBA 87-90
  Katrina Muller; Diane Kelly
This paper explicitly models subjective relevance by deconstructing its elements. We outline the various dimensions of subjective relevance, considering internal and external factors as well as interactions. We employ a utility framework for modeling, both conceptually and mathematically, subjective relevance and its multiple dimensions, aspects and interactions.
Query Reuse in Exploratory Search Tasks BIBA 91-94
  Chirag Shah; Gary Marchionini
In this paper, we present a number of observations and analyses from a user study. The study involved 84 subjects working on two different exploratory tasks for two sessions, which were one to two weeks apart. We found that a large portion of queries consisted of repetition of previously used query by the same user. There was also a high amount of overlap among the queries of different users for a given task, thus confirming the assumption that people tend to express their information request in the same/similar way for the same information need.
Towards Timed Predictions of Human Performance for Interactive Information Retrieval Evaluation BIBA 95-98
  Mark Smucker
Today's popular retrieval metrics are largely divorced from any notion of a user interface or a user model. These metrics such as mean average precision produce measures of ranked results quality rather than predictions of human performance. Using GOMS, we modify the Cranfield-style of evaluation to create a new evaluation method that makes testable predictions of human performance. While not yet validated by user studies, we demonstrate using our evaluation method that such an evaluation technique gives information retrieval researchers the ability to understand how changes in the interface or in the underlying retrieval algorithm impact user performance. Future work should be directed to the creation and validation of evaluation methods that predict user performance and incorporate explicit user interfaces and user models.
Text-To-Query: Suggesting Structured Analytics to Illustrate Textual Content BIB i
  Raphael Thollot; Marie-Aude Aufaure
The Information Availability Problem BIBA 99-101
  Daniel Tunkelang
In recent years, library and information scientists, particularly those concerned with interactive information retrieval, have complained that the information retrieval community -- both researchers and practitioners -- overemphasizes precision as a performance measure. More precisely, the IR community favors measures that emphasize precision in the top-ranked results, either explicitly (e.g., p@10) or implicitly (e.g., average precision, DCG). This essay advocates the study of the information availability problem, a general information seeking problem ill-served by today's models, evaluation measures, and tools. It defines the problem, proposes evaluation criteria for it, and explores how current and future tools could address it. Finally, it considers a testing approach based on the "games with a purpose" framework.
Exploratory Search Over Temporal Event Sequences: Novel Requirements, Operations, and a Process Model BIBA 102-105
  Taowei Wang; Krist Wongsuphasawat; Catherine Plaisant; Ben Shneiderman
Developing a detailed requirement analysis facilitates the building of interactive visualization systems that support exploratory analysis of multiple temporal event sequences. We discuss our experiences with collaborators in several domains on how they have used our systems and present a process model for exploratory search as the generalization of our experiences. This process model is intended as an outline of high-level analysis activities, and we hope can be a useful model for future and ongoing exploratory search tools.
Keyword Search: Quite Exploratory Actually BIBA 106-108
  Max Wilson
This short position paper describes some evidence found that counters the argument that there are better ways to support exploratory search than keyword search. Instead, this paper suggests that keyword search actually provides people with the freedom to search in relation to their own current state of understanding, rather than in the terms controlled by a search system. The challenge for future exploratory search systems, therefore, may be to maintain and enhance such freedoms.
Using Twitter to Assess Information Needs: Early Results BIBA 109-112
  Max Wilson
Information needs tell us why search terms are used, helping to disambiguate, for example, what exactly people are looking for with queries such as 'Orange' or 'Java'. It is hard to understand goals and motivations, however, from the keywords entered into search engines alone. This paper discusses the pilot analysis of 180,000 tweets, containing search-related terms, to try and understand how people describe their own needs and goals. The early analysis shows that some terms academically associated with searching behaviours were infrequently used by twitter users, and that the use of terminology varied depending on the subject of search. The results also show that specific topics of searching tasks can be identified directly within tweets. Future analysis of the still on-going 5-month study will constitute more formal text analytical methods and try to build a corpus of real search tasks.
Integrating User-generated Content Description to Search Interface Design BIBA 113-119
  Kyunghye Yoon
In this paper, the ideas discussed will focus on the integration of user tags into information search and interface design. There are two propositions: 1) user-created tagging is a valuable source of user's personal views and annotations that can augment the content description of information resources; 2) information search can be viewed as seeking meaning in information use and need. It is suggested to draw user meaning from the tag data by employing the topic and comment as two dimensions of linguistic meaning and to represent the meaning as a simple semantic relation that can be used for clustering the search results to supplement the traditional topic-based information matching. The sample analysis was done with the user tagging data positioned in the Delicious site to identify the semantic relations of linguistic meaning.
Ambiguity and Context-Aware Query Reformulation BIBA 120-122
  Hui Zhang
In this position paper, we suggest that query ambiguity is a major challenge for IR and there is space of improvement for existing approaches. Thus, we propose a novel disambiguation approach that constructs word meanings based on context mining from user sessions in search engine query log. Our preliminary result makes us believe that it is a promising direction. We also discuss how a search interface benefits from this approach in supporting faceted and exploratory search by context-based query reformulation.