HCI Bibliography Home | HCI Conferences | ESP Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
ESP Tables of Contents: 868791939697

Empirical Studies of Programmers: Fourth Workshop

Fullname:Empirical Studies of Programmers: Fourth Workshop
Editors:Jurgen Koenemann-Belliveau; Thomas G. Moher; Scott P. Robertson
Location:New Brunswick, NJ
Dates:1991-Dec-07 to 1991-Dec-09
Publisher:Ablex Publishing Corporation
Standard No:ISBN 0-89391-857-1 (paper); 0-89391-856-3 (cloth); hcibib: ESP91
  1. Panels
  2. Papers


The Best of ESP BIB 1-2
  Stu Zweben; Bill Curtis; Ben Shneiderman
Tutors and Environments for Novice Programmers BIB 3-4
  Wayne D. Gray; John R. Anderson; Brian Reiser; Elliot Soloway; James C. Spohrer


Reasoning from a Schema and from an Analog in Software Code Reuse BIBA 5-22
  Francoise Detienne
The activity of design involves the decomposition of problems into subproblems and the development and evaluation of solutions. In many cases, solution development is not done from scratch. Designers often evoke and adapt solutions developed in the past. These solutions may come from an internal source, i.e. the memory of the designers, and/or from an external source. The goal of this paper is to analyse the characteristics of the cognitive mechanisms, the knowledge and the representations involved in the code reuse activity performed by experienced programmers. More generally, the focus is the control structure of the reuse activity. Data collected in an experiment in which programmers had to design programs are analyzed. Two code reuse situations are distinguished depending on whether or not the processes involved in reuse start before the elaboration of what acts as a source-solution. Our analysis highlights the use of reasoning from a schema and from an analog in the code reuse activity.
A Feasibility Study of Early Class Hierarchy Construction in Object-Oriented Development BIBA 23-35
  Joseph L. Dvorak; Thomas G. Moher
In an observational study, five professional programmers were asked to design object-oriented class hierarchies based on lists of properties similar to those which might be extracted from a software project requirements document. Differences in domain experience resulted in qualitative differences in their approaches to the problems and substantially impacted inter-subject agreement on the structure of the resultant hierarchies. Performances differences are interpreted in the context of categorization theory, and implications are drawn for object-oriented software development.
Analyzing Distributed Cognition in Software Teams: A Case Study of Team Programming During Perfective Software Maintenance BIBA 36-64
  Nick V. Flor; Edwin L. Hutchins
This paper introduces a new approach -- Distributed Cognition -- to viewing collaborative activities and analyzes a pair of programmers performing a perfective software maintenance task in accordance with this approach. Distributed Cognition takes as its unit of analysis a complex cognitive system: collections of individuals and artifacts that participate in the performance of a task. The external structures exchanged by the agents of complex cognitive systems comprise its "mental" state and unlike individual cognition, where mental states are inaccessible, these states are observable and available for direct analysis. Through an analysis of these structures, their trajectories through the system, and their transformations, it will be demonstrated that complex cognitive systems engaged in software development tasks possess cognitive properties distinct from those of individual programmers. These properties are important for the system's successful completion of the task, yet they are ignored in studies of individual programmers. Studies of system level cognitive properties, when combined with existing research on the cognitive properties of individual programmers, should lead to a more comprehensive understanding of the process of software development. Before this integration can be made, however, these system level properties must be uncovered. The following research is a step in this direction.
An Investigation of Procedure and Variable Names as Beacons during Program Comprehension BIBA 65-81
  Edward M. Gellenbeck; Curtis R. Cook
Beacons are defined as sets of key features that typically indicate the presence of a particular data structure or operation. It has been claimed that programmers use them to help comprehend an unfamiliar program. However, despite their importance, beacons are not well-defined and very few have been identified. The present study investigated the importance of procedure and variable names as beacons during comprehension of short Pascal procedures. Ninety-six college seniors and graduate computer science students studied versions of a binary search and sorting procedures for one minute each and then provided a written description of the functions of each procedure. It was found that both meaningful procedure and variable names served as beacons to high-level comprehension. However, a strong code beacon, like the swap operation in a sorting routine, was found to be relied on more than the procedure name for high-level comprehension.
Does Signaling Help Professional Programmers Read and Understand Computer Programs? BIBA 82-98
  Edward M. Gellenbeck; Curtis R. Cook
Signaling refers to the addition of non-content information to a text in order to emphasize certain ideas and/or clarify the organization. There is increased interest in using typographic signaling, such as boldface type, different font sizes and ruled lines, to format computer program source code listings. However, little evidence exists to show that this type of formatting makes programs easier to read and understand. The present study reports on a controlled experiment which investigated the effectiveness of typographic signaling, header comments before modules, and mnemonic module names as ways to aid comprehension. We found that while all three factors helped experienced programmers confirm hypotheses about a 913-line program, only the header comments and mnemonic names helped the programmers locate material in the program. We also found only modest improvements in comprehension when typographic signaling was used with header comments and mnemonic names.
Use of Structure Editing Tools by Novice Programmers BIBA 99-120
  Dennis R. Goldenson; Bing Jyun Wang
Ever since the advent of the Cornell Program Synthesizer, syntax directed structure editing has been the subject of much controversy in the programming systems and computer science education communities. In previous work we have demonstrated striking performance quality differences attributable to use of structure editing environments by novice programmers. Here we examine more closely evidence about the process by which students actually build and test their programs using various structure editor based features and optional tools. Data obtained by logging user initiated editing events reveal that students using Carnegie Mellon's GENIE Pascal environment do in fact make good use of structure editing functionality. However noticeable optional tool use pattern differences also exist, with implications for user interface design, curriculum and teacher preparation.
Comprehensibility of Visual and Textual Programs: A Test of Superlativism Against the 'Match-Mismatch' Conjecture BIBA 121-146
  T. R. G. Green; M. Petre; R. K. E. Bellamy
The response-time study reported here compares readability of textual and graphical programming notations. The two issues addressed are relative comprehensibility of low-level micro-structure, and higher-level reasoning -- advantages have been claimed for visual programming languages at both levels. The study contrasts two major hypotheses: Superlativism (graphical is naturally best) and information accessibility (the structure of any given VPL, in combination with the reader's experience, will mean that certain tasks are easy and others are hard). Short conditional structures were expressed in four notations: textual or graphical crossed with sequential or circumstantial. Experienced programmers performed question-answering, same-different judgment, and identification of programs presented tachistoscopically. All were applied to both the visual and the textual forms. The clear overall result was that the graphical programs took longer than the textual ones. The reading strategies employed by the subjects varied considerably: broader experience led to more flexible performance. The less experienced subjects were unable to exploit the 'secondary notation' of the graphical representations which would have assisted comprehension -- it appears that what a programmer sees is largely a matter of training.
How Experts Infer Novice Programmer Expertise: A Protocol Analysis of LISP Code Evaluation BIBA 147-155
  Thomas Mastaglio; John Rieman
We present the results of a protocol study of human experts analyzing code produced by other programmers. The study focused on how experts infer the knowledge and expertise levels of anonymous programmers from examining their LISP code. We were particularly interested in what aspects of those programs our experts used in their inference process. Those aspects which triggered a verbal response were collected and are referred to as "cues". Further analysis determined three distinct categories of cues: syntactic, code semantics, and problem semantics. We determined that of these categories the first two are amenable to acquisition by a computer system designed to serve as a knowledge-based programmer's assistant or critic. The third category however, requires knowledge beyond state of the art artificial intelligence techniques.
Transfer of Training Between Programming Subtasks: Is Knowledge Really Use Specific? BIBA 156-176
  Nancy Pennington; Robert Nicolich
The dominant theory of transfer of training is a theory of "common elements" based on Anderson's ACT* theory of skill acquisition (Singley & Anderson, 1989). In this theory, the knowledge acquired while learning a skill is encapsulated in procedures called production rules. Transfer between tasks is predicted to occur to the extent that the two tasks share production rules or "common elements." This leads to a principle of "use specificity of knowledge" which makes the strong statement that knowledge acquired in the practice of one subskill (such as writing a computer program) will not transfer to performance in a related subskill (such as understanding a computer program), even though the two subskills rest on a shared declarative knowledge base (such as definitions of programming language instructions) (McKendree & Anderson, 1987).
   Our research provides a test of the ACT* predictions of transfer and the use specificity principle, when considering transfer between two subtasks within the acquisition of computer programming skill. First we provide detailed a priori transfer predictions based on a task analysis and production system simulation model of two programming subtasks: the evaluation and generation of LISP instructions. Next, we present preliminary results from an empirical study of training and transfer between these two subtasks. Comparisons between empirical results and simulation predictions reveal that there is substantially more transfer between subtasks than was predicted. These results call into question either the principle of use specificity of knowledge or the ability to make a priori predictions within the ACT* framework. Current research is aimed at disentangling these two explanations.
What Do Expert Programmers Communicate by Means of Descriptive Commenting? BIBA 177-195
  R. Douglas Riecken; Jurgen Koenemann-Belliveau; Scott P. Robertson
When generating source code, programmers are required to communicate a set of tasks to be performed such that they are interpreted correctly by both a computer and other programmers. We report a study in which expert programmers improved the clarity of a program in an attempt to communicate specific program knowledge to novice programmers. We examined descriptive commenting and addition of white space as methods of communication applied by experts. We discuss plausible factors which motivate the use of these different methods of communication. The results of this study provide insight by which to improve code-based communication between expert and novice programmers.
Psychological Design Rationale for an Intelligent Tutoring System for Smalltalk BIBA 196-209
  Mark K. Singley; John M. Carroll; Sherman R. Alpert
Although a descriptive psychology of programming is maturing, it is unclear whether it is cumulating in such a way as to have any obvious impact on the design of programming languages or environments. In the present work, we consider an alternative framework for bringing to bear empirical observations of user behavior on the design of programming artifacts. Specifically, we construct a psychological design rationale for an intelligent tutoring system for Smalltalk based on observations of users working with Smalltalk. We observed six persons of varying levels of expertise undertake an introductory yet representative project in Smalltalk: to create a window that transforms and displays all of its keyboard input in upper case. Our intention is to explicate and strengthen the link between empirical observations and design decisions.
An Empirical Study of the Effects of Design/Documentation Formats on Expert System Modifiability BIBA 210-226
  Kathleen M. Swigger; Robert P. Brazile
This study evaluates two different design/documentation formats to determine which is most effective for modifying expert systems. In a previous study, the authors examined the effectiveness of using a design/documentation format to modify a small expert system (Swigger & Brazile, 1989). The current study extends that research by asking the same question in the context of a larger, more complex expert system which was built for a procedural domain. During the exercise, programmers were given standard documentation (i.e. listings, program specifications, etc.) as well as two different design/documentation formats. Measures relating to modification time and number of errors were calculated for each group of programmers. Results indicate that subjects who used a Petri net design format to make modifications were able to complete the task in less time than programmers who used an Entity-Relationship (ER) design/documentation format. These and other findings are discussed with respect to their relationship to software and knowledge engineering.
Strategy Selection and Change in PASCAL Programming BIBA 227-238
  Quanfeng Wu; John R. Anderson
Three experiments were conducted to investigate iterative or looping strategy selection and change, especially between the "while-do" and the "repeat-until" looping constructs, in PASCAL programming. The results from the first experiment, in which subjects were free to choose between the two looping alternatives, indicated that most PASCAL programmers were quite sensitive to problem types and adaptable in choosing appropriate looping strategies. In other two experiments subjects were either forced or primed to use one of the two looping strategies. These experiments revealed that subjects were quite tenacious in using the appropriate strategy and their performance deteriorated when they were forced to use a different strategy. Finally, some implications of our findings to programming language designing and learning were explored.