HCI Bibliography Home | HCI Conferences | ESP Archive | Detailed Records | RefWorks | EndNote | Hide Abstracts
ESP Tables of Contents: 868791939697

Empirical Studies of Programmers: Seventh Workshop

Fullname:Empirical Studies of Programmers: Seventh Workshop
Editors:Susan Wiedenbeck; Jean Scholtz
Location:Alexandria, Virginia
Dates:1997-Oct-24 to 1997-Oct-26
Standard No:ISBN 0-89791-992-0; ACM DL: Table of Contents hcibib: ESP97
An Experimental Evaluation of Quality Guidelines on the Maintainability of Object-Oriented Design Documents BIBAKPDF 1-19
  Lionel C. Briand; Christian Bunse; John W. Daly
This paper presents a controlled experiment focusing on the impact of applying quality design principles such as the ones provided by Coad and Yourdon on the maintainability of object-oriented design documents.
   Results, which repeat the findings of a previous study, strongly suggest that such design principles have a beneficial effect on the maintainability of object-oriented design documents. It is argued that object-oriented design documents are sensitive to poor design practices because the cognitive complexity introduced becomes increasingly unmanageable. However, as our ability to generalise these results is limited, they should be considered as preliminary, i.e., it is very likely that they can only be generalised to programmers with little object-oriented training and programming experience. Such programmers can, however, be commonly found on maintenance projects. As well as additional research, external replications of this study are required to confirm the results and achieve confidence in these findings.
Keywords: Design documents, Experiment, Maintainability, Object-oriented, Replication
A Bug's Eye View of Immediate Visual Feedback in Direct-Manipulation Programming Systems BIBAKPDF 20-41
  Curtis Cook; Margaret Burnett; Derrick Boom
Immediate visual feedback is becoming a common feature in direct-manipulation programming systems of all kinds -- from demonstrational macro builders to spreadsheet packages to visual programming languages featuring direct manipulation. But does immediate visual feedback actually help in the domain of programming? We previously reported on an empirical study to determine whether the inclusion of immediate visual feedback into a direct-manipulation programming system helps with one particular task: debugging. In that study, subjects debugged programs with and without immediate visual feedback. We found that although immediate visual feedback did not significantly help with debugging in general, it did significantly help with debugging in some circumstances.
   In this paper, we follow up on those results, looking at attributes of the bugs themselves to see if they help to determine the circumstances in which feedback helps with debugging. We analyze how particular bugs and collections of bugs grouped by error type related to subjects' debugging abilities with and without immediate visual feedback, which we term the "which" questions; how bugs' position on the screen related to subjects' debugging abilities with and without immediate visual feedback, termed the "where" questions; and whether the presence or absence of immediate visual feedback affected the speed and order in which bugs were corrected, termed the "when" questions. The results show that a bug's error type and screen position were together a strong predictor of whether feedback would aid in identifying and correcting it, and that these two factors also significantly influenced how feedback affected the speed and order in which the bugs were corrected.
Keywords: Direct manipulation, Debugging, End-user programming, Spreadsheets, Visual programming languages, Liveness, Empirical study
Programming Complex Objects in Spreadsheets: An Empirical Study Comparing Textual Formula Entry with Direct Manipulation and Gestures BIBAKPDF 42-68
  Herkimer J. Gottfried; Margaret M. Burnett
Spreadsheets have traditionally provided strong support for computations that use simple types such as numbers and strings, while providing little or no support for programming with more complex objects such as colored circles or user-defined types. In expanding spreadsheets to support complex objects, we wanted to learn whether programs using complex objects could be constructed more effectively by defining formulas textually or through the use of graphical techniques such as direct manipulation and gestures. In this paper we present an empirical study comparing these two techniques for constructing spreadsheet programs that use complex objects. The results show that programmers can use graphical techniques to program complex objects faster and with fewer errors than with the textual methods traditional of spreadsheets.
Keywords: Spreadsheets, Gestures, Direct manipulation, Empirical study, Visual programming, Forms/3
How is the Designer Different from the User? -- Focusing on a Software Development Methodology BIBAKPDF 69-90
  Jinwoo Kim; Jungpil Hahn; F. Javier Lerch
The main objective of this study is to uncover the differences in the programming behavior between methodology designers and methodology users. We conducted an experiment with methodology designers who have invented one of the major object-oriented methodologies and programmers who have used the methodology for their projects. Concurrent verbal protocols were analyzed based on a theoretical framework which views programming as search in four problem spaces: representation, rule, instance, and paradigm. In programming, the main problem spaces are the representation and the rule spaces, while the paradigm and instance spaces are the supporting spaces. The results of the experiment show that differences in the supporting space produced different search behavior in the main problem spaces, which in turn resulted in different final programs and performance.
Keywords: Designer, User, Multiple problem spaces
Using a Fine-Grained Comparative Evaluation Technique to Understand and Design Software Visualization Tools BIBAKPDF 91-108
  Paul Mulholland
Software Visualization can be defined as the use of graphical and textual formalisms to describe the execution of computer programs. A large amount of Software Visualization technology has been developed to support computer science education, using a range of interface techniques. Far less effort has been devoted to evaluating the technology. As a result, it is unclear how effective Software Visualization tools are, either for students or professional programmers. Even more worrying, it is doubtful whether lessons are being learnt in successive designs of Software Visualization tools, or whether the application of new technologies (e.g. 3D animation and the internet) has become the primary goal, rather than the true goal of making computer programs easier to understand. To counter this problem the study reported here used protocol analysis to develop a fine-grained account of user behaviour, identifying (i) information access from the display, (ii) the use of comprehension strategies, and (iii) misunderstandings of the visualization and execution. The results were able to motivate future deigns which in turn could be compared and improved. The approach is compared to other evaluation techniques which aim to inform design. Finally, the generalizability of the approach is considered.
Keywords: Evaluation, Design, Software visualization, Software comprehension, Prolog
A Glimpse of Expert Programmers' Mental Imagery BIBAKPDF 109-123
  Marian Petre; Alan F. Blackwell
There is widespread anecdotal evidence that expert programmers make use of visual mental images when they are designing programs. In this study, expert programmers were directly questioned regarding the nature of their mental representations while they were engaged in a design task. This investigative technique was used with the explicit intention of eliciting introspective reports of mental imagery. The resulting transcripts displayed a considerable number of common elements. These suggest that software design shares many characteristics of more concrete design disciplines. They also provide promising areas for further investigation of software development support tools and design strategies.
Keywords: Mental representation, Mental imagery, Expert programmer behaviour
An Empirical Study of Novice Program Comprehension in the Imperative and Object-Oriented Styles BIBAKPDF 124-139
  Vennila Ramalingam; Susan Wiedenbeck
The objective of this study was to determine whether the mental representation of object-oriented programs differs from imperative programs for novice programmers. In our study novices who had little or no previous programming experience studied and answered questions about three imperative and three object-oriented programs. The questions targeted information categories making up the program model and the domain model representations of the programs. It was found that there was a sharp contrast between the mental representations of the imperative and object-oriented programs. While the comprehension of the imperative programs was better overall than that of the object-oriented programs, the mental representations of the imperative programs focused on program-level knowledge. On the other hand, the mental representations of the object-oriented programs focused more strongly on domain-level knowledge. The results tend to support the view that language notations differ in how well they support the extraction of various kinds of information.
Keywords: Object-oriented programming, Imperative programming, Novices, Mental representation, Program model, Domain model, Program comprehension
A Study of End-User Programming for Geographic Information Systems BIBAKPDF 140-156
  Carol Traynor; Marian G. Williams
This paper presents an empirical study of a programming by demonstration language for a geographic information system (GIS). The long-term goal of the project is to enable non-technical end users to exercise the capabilities of a GIS without having to learn the technical concepts that are embedded in most traditional GIS interfaces (Traynor & Williams, 1995). The programming by demonstration language is an extension of the Pursuit language introduced by Modugno for file management in the Macintosh Finder (Modugno, Corbett & Myers, 1996). The extensions permit the display of textual information in tables and of cartographic information on a map. The purpose of the preliminary study reported here was to determine whether programmers could read, edit, and create programs in the programming by demonstration language. Subjects' performance on the program comprehension tasks and the editing of simple programs was error free. Errors in the editing of more complex programs and in the program creation tasks indicate that some of the language constructs may need to be redesigned. Subjects' opinions of the programming by demonstration language were generally positive, as indicated by post-test questionnaires. We conclude that programming by demonstration is a promising approach for a GIS interface.
Keywords: End-user programming, Programming by demonstration, Visual language, Geographic information systems, Empirical study
Program Understanding Behavior During Debugging of Large Scale Software BIBAKPDF 157-179
  Anneliese von Mayrhauser; A. Marie Vans
This paper reports on a software understanding experiment during corrective maintenance of large-scale software. Participants were professional software maintenance engineers. The paper reports on the general understanding process, the types of actions programmers preferred during the debugging task, and the level of abstraction at which they were working. The results of the observation are also interpreted in terms of the information needs of these software engineers during the debugging task.
Keywords: Program comprehension, Software maintenance, Debugging
Visual Programming: The Outlook from Academia and Industry BIBAKPDF 180-208
  K. N. Whitley; Alan F. Blackwell
This paper presents three surveys of beliefs about the cognitive effects of visual programming. One survey examined the visual programming literature for the opinions of academic researchers. The second and third surveys gathered the opinions of the users of programming languages: the second survey questioned professional programmers attending a trade show and the third was a world-wide survey of LabVIEW programmers conducted via the Internet. The first survey revealed speculative, optimistic views of visual programming amongst visual programming researchers. The second found that a broad sample of professional programmers were the most skeptical of the three groups. The third indicates that LabVIEW programmers are confident that the visual programming provided by LabVIEW is beneficial. Taken together, these surveys illustrate the wide range of unanswered questions about the cognitive effects of visual programming languages. In doing so, this research hopes to assist in the process of establishing a more rigorous scientific investigation of these cognitive effects.
Keywords: Visual programming, Questionnaire, Opinion, Survey
A Study of Program Entry Time Predictions for Application-Specific Visual and Textual Languages BIBAKPDF 209-223
  Marian G. Williams; J. Nicholas Buehler
Creating and editing a computer program involves creative design work, but also involves the mechanical work of entering the code. Thus, program entry time needs to be taken into account in comparative studies of program creation and editing tasks using textual and graphical languages. We present a study of program entry time for application-specific graphical and textual languages with equivalent functionality. First, typical program entry tasks were modeled, and time predictions were calculated from the models. Then a small empirical study was performed to check the validity of the models. There was a high positive correlation (r=.927, p < .005) between observed execution times and predicted times. In addition, there was a significant difference (p < .05) between the execution times for the graphical and textual conditions for each task, and the difference was always in the direction predicted by the models. Finally, the prediction model was fine-tuned to produce even greater correlation with observed results. This study suggests that our upcoming study of learning outcomes in time-limited training situations, which will use the graphical and textual languages reported on here, does not have a systematic bias against either language in the effort required for program entry. It also provides evidence for the usefulness of keystroke level modeling for comparison of program entry tasks and suggests that related kinds of models may be useful for comparing the performance of other kinds of programming tasks.
Keywords: Visual language, Textual language, Program entry, Keystroke Level Model
What Do We Know about Programming? BIBAKPDF 224-232
  Watts S. Humphrey
This brief paper summarizes an invited talk I gave at the 7th Workshop on Empirical Studies of Programmers, in Alexandria, VA, on October 24-26, 1997. The paper describes the methods of the Personal Software Process (PSP) and shows how PSP data can be used to illuminate some questions about programming work. The paper concludes with two additional questions that need to be addressed as the software business grows and matures. To keep this paper at a reasonable size, I only include selections of the PSP data I showed in the talk.
Keywords: Discipline, Education, Personal Software Process, PSP, Programming, Quality
Problems versus Solutions: The Role of the Application Domain in Software BIBAKPDF 233-240
  Iris Vessey
The computing field has, to date, focused almost exclusively on application-independent approaches. While a great deal of progress has been made, there is, now, general agreement that further such progress is becoming increasingly difficult, and that it is vital to begin considering application domain-specific solution approaches that address the particular problem at hand.
   There are currently signs that many computing communities are, in fact, taking an application domain-specific approach to their fields. This analysis will address the status of several of those communities (for example, domain analysis (reuse), architecture, frameworks, patterns, method engineering, among others) to assess the progress that is being made and to stimulate reflection on the use of application domain-specific approaches in the software field. The analysis will show that, while some progress is being made, much more research is needed before the field can say it has made worthwhile inroads into the domain-focused approach.
Keywords: Application, Application domain, Problem, Solution, Methods, Taxonomies