HCI Bibliography Home | HCI Conferences | ESP Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
ESP Tables of Contents: 868791939697

Empirical Studies of Programmers: Second Workshop

Fullname:Empirical Studies of Programmers: Second Workshop
Editors:Gary M. Olson; Sylvia Sheppard; Elliot Soloway
Location:Washington, DC
Dates:1987-Dec-07 to 1987-Dec-08
Publisher:Ablex Publishing Corporation
Standard No:ISBN 0-89391-461-4 (cloth) 0-89391-462-2 (ppk); HD 8039 D37 E46; hcibib: ESP87
Programming and Algebra Word Problems: A Failure to Transfer BIBA 1-13
  Gary M. Olson; Richard Catrambone; Elliot Soloway
Prior work has suggested that learning to program may provide students with skills that help them in algebra. However, this work was only preliminary. An extensive experiment was conducted in order to examine the contribution of programming to students' algebra word problem performance. Students taking an introductory programming course in Pascal were compared to a control group of students (taking an introductory statistics course) with a similar mathematics background. Subjects were tested on algebra word problems at the beginning of the semester and either one week later or at the end of the semester (12 weeks later). Subjects performance on the algebra word problems improved from the first test to the second. However, contrary to expectations, the programming students did not improve more than the control subjects. In addition, those subjects who took the second test one week after the first test improved more than subjects who took the second test at the end of the semester. The results suggest that programming does not provide general benefits that transfer to algebra word problems, but that there is specific transfer due to practicing algebra problems.
Understanding Procedures as Objects BIBA 14-32
  Michael Eisenberg; Mitchel Resnick; Franklyn Turbak
Programming languages that treat procedures as "object-like" entities (for example, allowing procedures to be passed as arguments to other procedures) offer major advantages in semantic power and syntactic elegance. In this paper, we examine how novice programmers appropriate the idea of procedures as objects. Based on a series of structured interviews with students in the introductory computer-science course at MIT, we develop a model of the students' ontology of procedures. We conclude that many students view procedures as inherently active entities, with few "object-like" properties. We speculate on the implications of these results for the design and teaching of languages that treat procedures as objects.
Mental Representations of Programs for Student and Professional Programmers BIBA 33-46
  Robert W. Holt; Deborah A. Boehm-Davis; Alan C. Schultz
This research examined programmers' cognitive representations of software. In this study, student and professional programmers were asked to make either simple or complex modifications to three different programs that had been generated using each of three different design methodologies: in-line code, functional decomposition, and a form of object-oriented design. The programmers' mental models of the programs they had studied were elicited and then scored in several different ways. The results suggest that problem structure, problem type, and ease of modification may affect the mental models formed. Specifically, the data suggest that while the mental models of professional programmers were affected primarily by modification difficulty, the mental models of student programmers were primarily affected by the structure and content of the programs. Performance differences between the two groups of programmers were small because the experience variables which were most strongly related to performance were nearly equal in the two groups, and the experience variables which were very different between the two groups were not related to performance. Across the two groups, the primary aspect of the mental model which was correlated with performance variables was the width or breadth of the mental model structure. The implications of the results for the application of program design methodologies are discussed.
Communication Breakdowns and Boundary Spanning Activities on Large Programming Projects BIBA 47-64
  Herb Krasner; Bill Curtis; Neil Iscoe
Detailed interviews were conducted with personnel from 19 large software development projects to better understand team and project level problems to be addressed in MCC's research on software design environments. This paper reports observations of project communication phenomena that help bridge the gap between our understanding of programming-in-the-small and programming-in-the-large. We describe 1) the typical communications breakdowns in large programming projects, 2) the cultural and environmental differences that create barriers to effective intergroup communications, and 3) the boundary spanning activities that coordinate five crucial topical networks of communication. The identification of these processes and breakdowns provide a basis for more effective project coordination, including the use of tools for computer supported collaborative software design.
Breakdowns and Processes During the Early Activities of Software Design by Professionals BIBA 65-82
  Raymonde Guindon; Herb Krasner; Bill Curtis
This chapter summarizes some of the main breakdowns (of difficulties) occurring early in the software design process when professional designers work on a problem of realistic complexity. One class of breakdowns is caused by lack of knowledge and another class is caused by cognitive limitations. A third class of breakdowns is caused by a combination of these two factors. The main breakdowns observed are: 1) lack of specialized design schemas; 2) lack of a meta-schema about the design process leading to poor allocation of resources to the various design activities; 3) poor prioritization of issues leading to poor selection of alternative solutions; 4) difficulty in considering all the stated or inferred constraints in defining a solution; 5) difficulty in performing mental simulations with many steps or test cases; 6) difficulty in keeping track and returning to subproblems whose solution has been postponed; and 7) difficulty in expanding or merging solutions from individual subproblems to form a complete solution. We have also observed serendipitous design and the process of understanding and elaborating the requirements through exploration of the designer's mental model of the problem environment. This study provides many observations of breakdowns and design behaviors not reported in previous studies and necessary prior to developing a model of the cognitive activities during software design. This study also provides critical information to guide the design of tools and methodologies to improve the efficiency of software designers.
A Methodology for Studying Software Design Teams: An Investigation of Conflict Behaviors in the Requirements Definition Phase BIBA 83-99
  Diane B. Walz; Joyce J. Elam; Herb Krasner; Bill Curtis
This paper presents a methodology which the authors have developed for the analysis of the processes involved in designing large-scale computer-based systems. This methodology is based upon a characterization of the design process which 1) recognizes the diversity of team members' underlying conceptualizations, 2) emphasizes the transformation of abstract goals into concrete systems, and 3) distinguishes between those breakdowns in the design process which are a part of the design function, and those which are the results of the group process itself (within the design context). Essentially, the methodology traces interaction behaviors within design group meetings and relates them, hierarchically, to the issues under discussion. The paper also presents empirical results and a preliminary evaluation of the methodology from an analysis of the group aspect (specifically, conflict behaviors) within a series of requirements definition meetings from an actual development project.
Comprehension Strategies in Programming BIBA 100-113
  Nancy Pennington
This report focuses on differences in comprehension strategies between programmers who attain high and low levels of program comprehension. Comprehension data and program summaries are presented for 40 professional programmers who studied and modified a moderate length program. Illustrations from detailed think-aloud protocol analyses are presented for selected subjects who displayed distinctive comprehension strategies. The results show that programmers attaining high levels of comprehension tend to think about both the program world and the domain world to which the program applies while studying the program. We call this a cross-referencing strategy and contrast it with strategies in which programmers focus on program objects and events or on domain objects and events, but not both.
Graphical vs. Textual Representation: An Empirical Study of Novices' Program Comprehension BIBA 114-131
  Nancy Cunniff; Robert P. Taylor
This paper reports a study which demonstrates that certain fundamental aspects of computer programs are comprehended more quickly and more accurately when represented graphically than when represented in textual form. It is one in a series of reports on an on-going investigation into the influence of graphical representation on program comprehension and debugging being undertaken at Teachers College. It assumes that program comprehension is both definable and measurable and that the aspects of comprehension measured in this study are important components of novices' programming skill. This particular study involved 23 novice programmers who had learned to program in two languages, one graphically and the other textually represented. It investigated the speed and accuracy with which the subjects responded to comprehension questions about eight short program segments, each represented during the experimental session in both forms. The findings indicate that, for the comprehension measures considered, almost all of the subjects comprehended the graphically represented program segments both more rapidly and more accurately than their textually represented equivalents.
Parsing and Gnisrap: A Model of Device Use BIBA 132-146
  T. R. G. Green; R. K. E. Bellamy; J. M. Parker
This paper introduces a model of coding which highlights features of the device, task, interaction medium and user knowledge that are important in determining the ease of use of a programming support environment. The model has been implemented in Prolog and applied to the domain of expert coding; where it is used to explain results from an observational study of expert coding behavior. Although the model is still at an early stage of development, it clearly shows the need to build device languages and support environments which complement each other, in the light of users' tasks and knowledge structures. Methods for achieving this aim are discussed.
Improving Children's Debugging Skills BIBA 147-171
  Sharon McCoy Carver; Sally Clarke Risinger
Expert programmers use a wide variety of high-level problem-solving skills to deal effectively with diverse programming tasks. The hope that children might develop some of these high-level skills has inspired many parents and educators to advocate the inclusion of computer programming, usually Logo, in the school curriculum. Despite the intuitive appeal of this campaign and the vote of confidence many school districts have given in the form of computer equipment, researchers have found strikingly little evidence that children are developing any general problem-solving skills form their programming experiences. Our thesis is that children can indeed develop such skills if educators specify the skill components and teach them explicitly. Carver and Klahr (1) specified the components of debugging skill in the form of a computer simulation model, and Carver (2) designed a debugging curriculum to teach the specific components directly. We gave this explicit debugging instruction to 18 sixth graders in the context of a 25 hour Logo list-processing course over a four month period while 17 of their classmates were in study hall. After instruction, the Logo students' debugging speed and efficiency improved because they learned to narrow their search for bugs. We also found that the children who acquired effective debugging skills in the Logo course improved more on transfer tasks involving debugging written instructions than students who did not take the Logo course. We suggest that using a detailed model of a problem-solving skill as the basis for explicit instruction could be an effective methodology for fostering other high-level programming skills such as problem decomposition, procedurality, or planning.
An Analysis of the On-Line Debugging Process BIBA 172-184
  Murthi Nanja; Curtis R. Cook
This paper reports the results of a protocol study that compared the debugging process of expert, intermediate, and novice student programmers. Subjects used a microcomputer to debug a Pascal program with three semantic and three logic errors. The following performance differences were identified:
   (1) Experts employed a comprehension approach in which they first attempted to understand the program and then used this knowledge for finding the bugs. Intermediates and novices employed an isolation approach in which they immediately attempted to identify candidate bug locations by searching the output for clues, recalling similar bugs, and testing program states.
   (2) Experts corrected multiple errors before verifying the corrections while intermediates and novices corrected and verified single errors. Intermediates and novices corrected the semantic errors first and then the logic errors while experts corrected both semantic and logic errors at the same time.
   (3) Experts were the fastest and most successful in correcting all of the errors, modified fewer statements, and did not introduce more errors. Most of the novice did not correct all of the errors, made very extensive modifications and introduced many new errors. Most intermediates corrected all of the errors but they made considerable modifications and introduced several new errors.
Change-Episodes in Coding: When and How Do Programmers Change Their Code? BIBA 185-197
  Wayne D. Gray; John R. Anderson
Any change in a programmer's code or intentions while coding constitutes a change-episode. Change-episodes include error detection and correction (including false positives) as well as stylistic, and tactical changes. In this study we examine change-episodes to determine what they can add to the study of the cognition of programming. We argue that change-episodes occur most often for constructs that allow the most variability (with variability defined by the language, the task, and the programmer's history). We predict and find that those constructs that are involved in the most change-episodes are those for which much planning is needed during coding. Similarly, we discuss two ways in which a goal can be changed in a change-episode. One involves relatively minor editing of a goal's subgoals, suggesting that much planning is local to the current goal. The other entails major transformations in the goal's structure. Finally, we find that change-episodes are initiated in one of three very distinct circumstances: as an interrupt to coding, a tag-along to another change-episode, or a byproduct of symbolic execution. Our findings support the distinction between inherent and planning subgoals (2,3) and the distinction between progressive and evaluative problem-solving activities (6).
Advancing the Study of Programming with Computer-Aided Protocol Analysis BIBA 198-216
  Carolanne Fisher
The study of computer programming in all its complexity requires the use of data gathering and analysis techniques equal to the task. Protocol analysis is ideally suited to both the development and testing of theories in this emerging domain. Protocol analysis is, however, a difficult, time-consuming, and largely ill-defined task that can become formidable if the data set is particularly large or the patterns in it large, complex, or infrequent -- as is the case with computer programming data. To reduce the difficulty and to increase the reliability and objectivity of conducting protocol studies and to extend the range of possible analyses, I have developed a multipurpose computer-aided protocol analysis system. In this paper, I use a set of protocol data gathered from expert programmers developing sizable, real-world programs as a context in which to describe the system and demonstrate the detailed analysis it makes possible.
Strategies in Programming Programmable Controllers: A Field Study on a Professional Programmer BIBA 217-230
  Willemien Visser
One of the questions raised at the end of the First Workshop on Empirical Programmers (see 17), and which subsequently became the title of a Future Directions paper, was "By the way, did anyone study any real programmers?" (7). Our answer is "Yes." It is our wish in presenting this paper to contribute to the understanding of some aspects of "programming-in-the-large," in particular those concerning the specific strategies that the programmer uses.
   A professional programmer constructing a program that was to control an automatic machine tool installation was observed full time for four weeks in his daily work.
   In this paper, we chose to focus on the strategies used, under the hypothesis that they differ, at least partially, from those observed to date in most novice, student programmers working on artificial, limited problems.
   We observed some strategies already known to be at work in "programming-in-the small": planning, top-down and bottom-up processing, schema-guided information processing. However, other strategies seem indeed to be characteristic of programming in a work context: the frequent use of example programs, the importance of analogical reasoning, and the search for homogeneity (for comprehension and maintenance reasons). Finally, the opportunistic nature of the activity we observed also seems to be a characteristic of real programming activity.
A Cognitive Analysis of a Code Inspection BIBA 231-247
  Stanley Letovsky; Jeannine Pinto; Robin Lampert; Elliot Soloway
In this paper we describe a cognitive analysis of a design and code inspection. This formal inspection process, as designed and carried out at IBM, has shown itself to be effective for improving the quality of produced code. Our intent in studying this process was to understand what goes on during a code inspection and why. After analyzing one video-taped, 65 minute inspection in detail, we suggest that the participants in the inspection were attempting to achieve three main goals (clarity, correctness, and consistency), by executing three corresponding behaviors (design reconstruction, mental simulation, document cross-checking). These categories accounted for 89% of the duration of the code inspection. The major overall finding is that, surprisingly, considerable resources (time and personnel) were spent in establishing the relationship between the various software documents, i.e., the requirements, the design, and the code.
Can Principles of Cognition Lower the Barriers to Programming? BIBA 248-263
  Clayton Lewis; Gary M. Olson
We analyze the barriers to programming faced by the intelligent, task-oriented non-programmer, such as a working scientist. Part of the analysis is to examine a success story in making programming easier, the spreadsheet. Generalizations about programming extracted from this example are related to principles of cognition. Two general tactics for making programming easier for our target audience are presented: programming by modification, and reform of the primitives of programming languages. Two examples of programming tools that fit this vision are briefly presented, CMU Tutor and NoPumpG.