Proceedings of the Workshop on Empirical Studies of Software 
Maintenance, pages 11-12, Washington, D.C., U.S.A., 
16 November, 1998.


Characterizing How Software Maintainers Work: Cultures and Habits

Susan Elliott Sim
Department of Computer Science
University of Toronto
simsuz@cs.utoronto.ca

Position Statement
It is important to know the "state of practice" when improving processes 
and tools for software maintenance.  In order to modify processes, positive 
and negative aspects need to be identified, so they can be enhanced or 
eliminated appropriately.  Tools to assist maintainers need to be designed 
to fit with how they work.  We have employed both case study and survey 
methods to characterize existing practices in software maintenance.  These 
two methods are complementary and can be used to build an understanding of 
software maintenance practice.

Case study methods are appropriate for exploratory studies to develop a 
model or theory of a particular phenomenon.  They have been used a great deal 
in empirical studies of software maintenance.  We have used this method in a 
study of how newcomers, or software immigrants, to a project learn about the 
software system and become productive members of a team [Sim98a].  Seven 
patterns of commonality were found across four cases.  Some of the patterns 
found were:

+ software immigrants relied on mentors and peers for information 
  because there was little documentation available;
+ initial tasks were isolated problems that were assigned no earlier 
  than two weeks after beginning work; and
+ mentors passed on low-level information about the software system as 
  needed by the software immigrant to complete the assigned task.

Subsequent study at other industrial sites support many of the patterns.  
Indeed, it is a testament to the strength of case study methods that we were 
able to build such a robust theory with four cases.  The next step would be 
to test this theory with many subjects from different sites.  Survey methods 
can be used to determine the prevalence and validity of the model.

Survey methods can be applied to gather data from a large number of subjects.  
This method is better suited to social rather than psychological phenomenon.  
A common application of surveys is in the polling of political opinions.  By 
drawing a representative sample from a population, and asking a series of 
questions, pollsters are able to make more general statements, for instance, 
about the popularity of a candidate.  In order to generalize the results of a 
survey, the sample must represent the population being studied along relevant 
dimensions.  Over the years pollsters have determined the relevant dimensions 
for political issues which include age, gender, and socio-economic status.  
Some dimensions that not relevant to creating a representative sample are shoe 
size, handedness, and apartment number.

In order to apply survey methods effectively in software maintenance, we need 
to characterize the population that we wish to generalize the results to, and 
the relevant dimensions for creating a representative sample.  It is in 
characterizing the population, that an understanding of the magnitude of the 
software maintenance challenge plays a role.  What are the dimensions of 
variability in the population of software maintainers?  Which of them are 
relevant to the research question being examined?  

Some of the dimensions of variability in the population of software maintenance 
are size, type, and age of the software system, programming language used in 
implementation, size of maintenance team, education level and prior experience 
of maintainers.  Applied to the question of surveying software immigrants, not 
all of these dimensions are relevant.  Although our understanding is still 
incomplete, we know that beyond a minimum level dimensions such as size of 
system and education level does not affect the amount of time a software 
immigrant requires to become a productive team member.

Despite our current lack of knowledge about many dimensions for sampling, it 
is still possible to use survey methods in exploratory studies as we have done 
in a survey of source code searching habits [Sim98b].  However, to exploit the 
full power of survey methods, we need to greatly increase of understanding of 
the state of practice in software maintenance.  

References
[Sim98a]
   S.E. Sim and R.C. Holt.  The Ramp-Up Problem in Software Projects: A Case 
   Study of How Software Immigrants Naturalize.  Proceedings of the 20th 
   International Conference on Software Engineering, pages 361-370, Kyoto, 
   Japan, 1998.

[Sim98b]
   S.E. Sim, C.L.A. Clarke, and R.C. Holt.  Archetypal Source Code Searches: 
   A Survey of Software Developers and Maintainers.  International Workshop 
   on Program Comprehension, forthcoming, Ischia, Italy, 1998.

Background
Susan Elliott Sim is a doctoral student in the Department of Computer Science 
at the University of Toronto.  She has a background in psychology, sociology, 
and computer science.  Her research has focussed on empirical studies of 
software maintainers and the development of program comprehension tools based 
on these studies.  Her research interests also include source code analysis, 
and research methodology.  

Session Preference
I would like to participate in either session 1 or session 2.