reducing search space with no other features: dec ... - Semantic Scholar
Simulating Global Software Evolution Processes by Combining Simple Models: an Initial Study Paul Wernick and Tracy Hall Centre for Empirical Software Process Research Department of Computer Science University of Hertfordshire College Lane, Hatfield, Hertfordshire AL10 9AB, England tel. ++1707 286323/284782; fax ++1707 284303 {p.d.wernick, t.hall}@herts.ac.uk Abstract A number of studies of the long-term evolution of commercial software products over many releases have established a consistent pattern for growth in the size of the systems examined. This shows a trend towards a progressive slowdown in growth over time. The work described here forms the first step in developing a simulation of the combined effects of the causes of this trend. A number of simple System Dynamics simulations, each capturing the causal structure and behaviour of one of a number of causes believed to contribute to the observed behaviour, and its calibration to follow the observed trend, is presented. These models are then combined into a single simulation model reflecting the effect in combination of these causes. Possible causes considered here are: the reduction over time in both the conceptual space which can be searched for new uses of a system and the design spaces constraining developers’ ability to modify their systems, the effects of the degradation of the structure of a software system as is it is evolved in unanticipated directions, and the reducing coverage of developers’ knowledge of the structure of the system. The combined model incorporates all of these causal mechanisms and introduces a relative weighting for each. The long-term aim of this work is to produce a simulation embodying those possible causes which are found to be supported by actual evidence. Calibration of this model to reflect the relative importance of each cause in a real-world situation should then enable it to reproduce the actual software system size trend, so assisting in identifying those causes which affect most significantly the growth trend of that system.
Keywords software evolution, system dynamics, global software process, simulation, modularity
1. Introduction Studies of the long-term evolution of a number of commercial software products over many releases have established that there is a degree of consistency in the growth trend observed in their size. This trend shows a progressive falling-off over time of growth in the size of the system,
meaning that the system has relatively fewer new functions added as it grows older. The trend has been observed in a variety of types of software product evolved in differing technical environments [1, 2, 3]. Simulation-based research conducted so far into the global software process 1 [4] to determine possible reasons for this consistency has considered one possible cause, viz. the degradation of high-level system structure over time due to repeated changes [5]. However, that work does not necessarily deny the existence of a larger number of causes, each of which may contribute to the observed trend. The work described here forms the first step in the development of a System Dynamics (SD) simulation reflecting the structure, parameters and effects in combination of the causes of software evolution trends. The possible causes considered in the first model presented here are the reduction over time in both the conceptual space which can be searched for new uses of a system and the design spaces constraining developers’ ability to modify their systems, the effects of the degradation of the structure of a software system as is it is evolved in unanticipated directions, and the reducing coverage over time of developers’ knowledge of the structure of the system. The approach taken is to produce a simple simulation for each of these causes, calibrate it to follow the trend expected from such a mechanism, and then to combine these models into a single model reflecting all the causal mechanisms presented here and introducing a relative weighting for each. This approach has the advantage of exposing each of the proposed causes and its simulation to examination both in isolation and in the context of the other possible causes. The combined model has been calibrated as an initial proof of concept against typical software evolution trends, using parameter values taken in part from a previous simulation model [6]. The approach and the models presented here are intended in the first instance as a tool to improve understanding of the causal mechanisms underlying software evolution processes. Future development of this model will therefore include the development of additional simple models to simulate more causes as these are identified and the incorporation of these 1
This process comprises both the technical process by which software is produced, and the mechanisms and activities surrounding and controlling this technical process.
into the combined model. Calibration of this model to reflect the relative importance of each reason in a real-world situation should then enable it to reproduce the actual software system size trend. With this instrument it may thereby be possible to identify those causes which affect most significantly growth trends in an actual system within an individual situation. In the longer term, it may be possible to devise a simulation reflecting both the causes and the policies which can be used to affect the results of such processes; an example of a simulation of the effects of a policy decision is given in Section 4.2.1 below. By helping managers understand the effects of policy decisions on product and process metrics, this model may then form the basis of initiatives to improve the performance of specific software evolution processes. The rest of this paper is structured as follows. The approach taken is outlined in Section 2. The structure, parameters and calibration of a number of simple models are described in Sections 2.2, 3 and 4. A description of how these simple models have been combined into the final model, and the further calibration work needed to support this appears in Section 5. A discussion of how to use this model in the real world is contained in Section 6, the next steps envisaged for this research are in Section 7, and final conclusions of the work to date are presented in Section 8.
2. Outline of the Approach 2.1. The Approach Described This section describes the way in which System Dynamics has been employed, why this approach was taken, and the expected benefits of the approach. The approach is based on the view that the trends observed in software size over time are the result of a number of causes, each of which contributes to the trend to a degree which may vary over time. Thus, the causal mechanisms of the actual trend cannot be captured in a model which either reflects only one of these causes or which fails to allow the degree of contribution of each cause to be varied over time. The approach has been to produce a simulation of the causal mechanisms underlying the behaviour of a global software process which is made up of a number of sub-models. Each of these sub-models is intended to capture both the mechanism and the dynamic behaviour resulting arising from one possible cause in as simple a manner as possible, and is initially constructed as a stand-alone SD model using the Vensim simulation environment [7]. The simple sub-models are then combined into a single model, which thus captures the behaviour resulting from the identified causes of enhanced software size acting in combination. As part of this combination, the contribution of each causal sub-model to the simulated growth trend is weighted by a factor (which forms a parameter to the model), to enable the simulation builder to reflect the degree to which that cause affects the overall software evolution trend. By changing these weightings over simulated time, the changes in the relative degree of importance of the different causes over long periods of time can be reflected in the combined
simulation. This can include situations in which the changes in the values of the weightings over time are non-linear. Causes included in the simulation but which are believed to have no effect on the simulated evolutionary environment at any particular time can be eliminated from the calculation by setting their weightings to zero. The advantages of this approach include: each simple model is quick and easy to construct, understand, calibrate (by having fewer parameters requiring data collection exercises) and modify; the combined model will also be likely to be simpler in structure than a model constructed using techniques currently employed for developing simulations of software processes (see, for example, [8]; once the process of adding more causes to the combined model is systematised (see Section 5 below), it will also be simple to reflect new understanding about the global software process when compared with the development of a new SD model; taking an approach which as simply as possible simulates a believed cause and reproduces the expected software evolution trend arising from that cause results in a model which is easier to understand and to reason about than the combined model which is necessarily more complex in structure and behaviours. The ease of understanding and reasoning about the simple models will make it easier for people not experienced in simulation techniques, such as software developers, marketing people, etc. who are involved in the global software process to understand how the combined model’s overall structure and behaviours arise and can be influenced; it is easier using a model constructed as described here to understand the effect on software evolution of different causes individually and in combination, for example examining whether under a particular set of circumstances anti-regressive activities [5] are worth undertaking in the light of their relative effect on long-term system growth; the approach described here both simplifies the breaking down of some causes, which have previously been treated as monolithic, of observed software system size trends into combinations of lower-level causes, such as the hierarchy of filling of conceptual spaces described in Section 4 below. It also allows interactions between causal mechanisms, such as the effect of anti-regressive activities [5] on analysis and design activities, to be examined; the combined model, with each cause weighted relative to the others, can reflect circumstances in which the causes change in relative importance over time, e.g. increased loss of system architectural integrity and falling behind developer knowledge as the system ages, or lessening as anti-regressive work such as refactoring [9] is undertaken; and by adding new causal mechanisms which reflect possible future behaviours in the global software process and/or by changing the weightings of the different causes, it will be possible to investigate ‘what if’ situations such as the possible effects of decisions to make major changes to a
software system. However, care will need to be exercised in conducting and interpreting the results of such experimental simulation activities, since research suggests that when major changes are made to a system the trend in software product size may itself change significantly [10] (cf. [6]). Some of these reflect the benefits of a modular approach to simulation model development [11].
2.2. Basis of Model Calibration At this stage, the calibration of the models has been on the basis of proof-of-concept, rather than by reference to specific real-world software evolution processes. One aim is to demonstrate that, when provided with input parameter values which are reasonable on the basis of past experience, each simple model can represent expected trends based on the mechanism it represents. Since the simple models each only capture some part of the actual global process behaviour, they should not be expected individually to capture all of the detail in real-world software system evolution trends. However, the ability of each simple model to replicate the expected behaviour adds to the confidence in its reasonableness. The other aim of the calibration work has been to ensure that the combined model can reflect the widely-observed evolution trend referred to above. The approach to calibration has therefore been to use typical values derived from previous SD simulations of global software processes [12, 6] where possible for model input parameters. For example, the relationships between the time delays for the need for changes to be recognised and acted upon in the reducing spaces models (see Section 4.1 below) are derived from values used in the simulation of VME evolution [6]. At this early stage in the development of these models, and in the absence of a real-world process against which to calibrate the models, other parameters such as the EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR2 shown in both Figure 3 and Figure 5 have been have been calibrated to enable the model to fit the typical evolutionary trend, as in earlier work [6]. The simulated timeframe for the models is similarly abstracted, but is intended to represent real elapsed time (as against release sequence numbers) over some considerable period of time, typically several years or more. The output value of the simulations which has been used to test the reasonable operation of the simulation is the size of the system, which is shown to follow the typical trends of reducing growth noted previously. Again, this output is a generalisation of the observed trend, rather than representing actual values for the evolution of a real-world software system. The unit of measure of system ‘size’ is not defined in the models as a specific product metric, but is intended to represent some suitable measure related to the physical size of the system. In previous research into software evolution trends, this system size metric has been measured in terms of relative values over time [12], numbers of programming units 2
This constant parameter value is a scaling factor from the square of the current system size to the effect of growing system size on the effectiveness of developers’ work; see Section 4.2.1 below.
in the form of modules [6], object-oriented classes [2] or programs [1], and non-comment source code lines [13]. Generally, all of these measures have been seen by the researchers concerned as analogues of some measure of the ability of the system to do different types of useful work, i.e. the functional power of the system. This is captured in generalised form in the system size outputs of the models presented here. To summarise, the proof-of-concept calibration of these models is not against any specific evolution processes. However this calibration has demonstrated that the general trends in output behaviour which can be produced by these models reflect real-world observations.
3. The Simple Models Introduced 3.1. Introduction Each of the models presented below captures and simulates in a simple manner the result of one possible cause of the evolutionary trends described above, whilst abstracting all other features. The models reflect specific examples of two broad hypotheses as to the cause of the observed trend in software product size: a reduction over time in the possible changes which might be made to the system, each represented by the size of a conceptual space which has to be searched to find new possible or useful system enhancements; and the increasing size of the existing software system over time making it more difficult to change and/or to understand the effects of proposed changes.
3.2. Common Features of the Simple Models In order to simplify the process of combining the simple models, each was constructed around a common conceptual core, which has been augmented as necessary to represent the causal mechanisms underlying each identified influence on the behaviour of the software evolution process. This represents a flow of completed new system enhancements into a level representing the size of the system. This common core is shown in Figure 1. Its structure is similar to that of previous software evolution process simulations [12, 6] in which an SD third-order time delay function simulated the multistage software production process. Here the structure abstracts only the implementation of the software from an existing design. It represents using a third-order delay a flow of work being done on the system and the reflection of this work in system size and usable system functionality. The implementation, testing and preparation for release of new system functions are thus all abstracted into a single delay function represented by the valve in the diagram.
Implementation rate of implementation Figure 1. Common core structure of simple software evolution models
4. The Simple Models Described 4.1. Simulating Reducing Search Spaces There are three conceptual spaces within which a software-based system may be considered to be designed, implemented and fielded. The first of these is the user application space, i.e. the real-world environment within which the system is used. Evidence for this real-world problem space behaving as if it were of finite size arises from a previous software evolution study [6], in which the potential for use of a system in a new physical environment of itself resulted in an increase in demand for changes to that system. In a manner analogous to that real-world system user space, it is reasonable to suggest that there also exists an analysis model space, created by the software systems’ analysts as a conceptual model reflecting the analysts’ understanding of the real world. This analysis space is in turn used to bound the design of the system and its interactions with its environment, creating the design model space, which encompasses the high-level design of the software artefact itself, including the architecture and other design elements and which places limitations on how the system can be evolved without major restructuring. The implementation (code) of the system is seen here as an output of the design process; the obstacles to evolution are common to both, and the code is therefore only modelled as a simulation output. The rationale for the effect of a progressively reducing space on analysis and/or design options is that each explicit or implicit design decision made to incorporate some element in a model (or to leave it out) reduces the options available to make further changes. This viewpoint was the driver of a successful calibration of effort against actual values in a previous SD model of a real-world software process [12]. The size of each space places a limitation on the possibilities for the system’s use or capabilities (cf. [14: p.27]). These spaces are also effectively nested; the analysis space, within which the analysts consider what the system can or should do, will be bounded within the real-world possibilities, and the design space similarly bounded within the analysis space [14, p.28]. Since each of these spaces is (or behaves as if it were) finite, it is meaningful to speak of any one space being ‘filled up’ by the current system to the extent that further additions become increasingly difficult to identify
or make. It may however be possible to reverse this slowdown in the ability to add to the system by expanding the space by, for example, identifying new uses, analysis options and/or design possibilities for the system. Figure 2 shows an SD model which combines the effects of changes in all three of these spaces on system growth, and which reflects in addition the interactions between changes in the model spaces and the delays in these interactions. The model represents the real-world possibilities for the system to be useful (with changes if necessary) as a level called ‘Undiscovered Domain’, indicating that the developers have not yet been told or realised that the system could be useful in more situations than is currently the case. This level is increased by a feedback effect which reflects the FEAST hypothesis [15] that feedback from the release of new system functionality will of itself cause additional change demand. Here, the changes in the system create more possibilities for the system to be used if it is subjected to further change. The level is also increased by events in the outside world, such as changes in tax legislation for a payroll system. The level of undiscovered changes is reduced by the developers being told by customers, or otherwise discovering, that the system needs changes. This discovery takes time, which is reflected in a time delay in the model, and increases the level of unfulfilled analysis work demand (‘Unfilled Analysis Space’ in the model). Work done on turning analysis work demanded into designs for changes to the system reduce this level; the time taken to perform this work is reflected in a parameter to a standard SD delay function. The model must now reflect the design activity, which is simulated with a structure close to that of the analysis modelling. An effect of the analysis work being performed is the addition of work to be done to an Unfilled Design Space level parallelling the Unfilled Analysis Space; the amount of design work added is the analysis work done multiplied by some constant factor. In a manner exactly analogous to that of the analysis simulation, work done in designing the system adds to implementation work to be done. Finally, the implementation work, which again takes time, adds to the size of the actual system code. At each stage, a discovery factor determines the proportion of elements which might possibly progress to the next stage; this reflects the observation that not all useful changes will be detected by developers or users, that some of the change requests from users are not actioned for reasons often involving development management organisation decisions,
and so on through the process. These parameters are set to increase the likelihood of a change being implemented as it passes through the process, reflecting for example the higher probability that a change which has completed the design stage will be implemented and incorporated into the system code. As noted above, the increased functionality which all of the changes have finally reflected in the system will result in a feedback-driven pressure for more system changes (which is represented in the model by a figure made up of the increase in system size multiplied by a constant factor), thus restarting the cycle. The output trend shown here represents a situation in which the size of the system in source and object code terms is not ultimately constrained, and can rise to meet the needs of the users; this is typical for commercial and business systems. The model would need to be modified for situations in which
the output software system size is constrained, for example by a fixed physical memory size [12]. In that case, once the memory of the system was filled, no further growth in physical size would be possible. Were it still to be possible to add to the system’s functional power, for example by reducing the size of other parts of the code to make space for new functions, this might require the system size represented here to be denominated in some measure other than that of the physical code. The structure and outputs of the combined reducing space model are shown in Figure 2. The output graph of system size over time, shows, after an initial period for the dynamics behaviour to settle down, a broad trend reflecting the characteristic decrease in rate of growth of the system; the ripple in this trend is due to the interactions of the various delay times in the flows in the model.
DOMAIN DISCOVERY FACTOR
EXOGENOUS EVENTS
rate of addition
Undiscovered Domain
rate of discovery DOMAIN DISCOVERY DELAY
MULTIPLIER FACTOR
analysis space increase
Unfilled Analysis Space
ANALYSIS DISCOVERY ACCEPTANCE FACTOR
design space increase
analysis space filling
Unfilled Design Space
ANALYSIS DISCOVERY DELAY
design space filling DESIGN FILLING DELAY
DESIGN DISCOVERY ACCEPTANCE FACTOR
implementation work to do
To Be Implemented
Implementation rate of implementation
INITIAL IMPLEMENTATION RATE
IMPLEMENTATION DELAY
Implementation 200 150
100
50
0 0
25
50
75
100 125 150 Time (Month)
175
200
225
250
Implementation : combinedspace3 with ddf2
Figure 2. Reducing search spaces
4.2. Reducing Efficiency due to Growing Product Size The models described in this section reflect the broad hypothesis that the increasing size of a software system and changes in unanticipated directions will over time act to reduce the ability of developers to modify that system. This decline may be due to the increasing complexity of the system as its structure is degraded by efforts to make changes unanticipated when the system architecture was designed and/or a decrease in coverage of developers’ knowledge of the system components, their composition and interactions. 4.2.1. Effects of increasing system complexity. As a software product is evolved over time, some of the changes made to it will inevitably not have been identified at the time the architecture of the software was designed. These changes may result in a decay in the system architecture as its abstractions become less relevant and/or its encapsulations need to be infringed. New changes also have to be fitted into an existing system structure, and as the system grows there is more existing structure into which each new change needs to be fitted, thereby constraining the freedom to make these changes (cf. [15]). The model shown in Figure 3 reflects the result of these influences. In the model, a value which represents the efficiency of implementation effort (Efficiency Of Implementation) is reduced by implementation activities. One mechanism which has been suggested to address this problem is to employ software developers on tasks specifically intended to maintain the structure of the system,
for example by reducing coupling and cleaning up a layered architecture, or by restructuring object-oriented classes and their interactions in ‘refactoring’ exercises [9]. To contrast with ‘progressive’ system enhancement work, this is referred to as ‘anti-regressive’ activity. The effect of anti-regressive activity (or its lack) on system evolvability has been explored previously [5]; the simplified model presented here includes a structure intended to represent its effect. Staff effort (STAFF, in units representing numbers of people) is initially directed to adding system functionality. If some of this effort is redirected from adding new functionality to anti-regressive effort, less implementation work is done but the efficiency of future implementation work is increased. In the model, the ANTI-REGRESSIVE POLICY parameter is used to represent management policy in terms of the proportion of effort devoted to anti-regressive work The model output shows the effect on system growth as simulated in the model if there is no anti-regressive activity (line antireg0). It also shows the greater growth in system size later life made possible by changing the staff allocation policy towards anti-regressive work starting at simulated time (month) 50, employing 25% of the effort applied to the project (line antirg25). This result parallels that found by [5] in their simulation-based work i.e. that long-term system growth is increased. However, the model presented here is deliberately much simplified compared with theirs. In both cases, the change is a result of the simulation of a deliberate management decision to redeploy effort from one activity to another.
EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR
ANTI-REGRESSIVE FACTOR
improvement in efficiency
Efficiency of Implementation
deterioration in efficiency
Implementation rate of implementation ANTI-REGRESSIVE POLICY
STAFF
<Time>
240 180 120 60 0 0
20
40
60
80 100 120 Time (Month)
140
160
180
200
Implementation : antireg0 Implementation : antirg25
Figure 3. Effect of growing system complexity 4.2.2.
Potential effects of decreasing developer knowledge of a system. In order to be able to change a system, its developers need access to knowledge about that system. As the system grows, the amount of knowledge needed to support further changes also grows, but at a faster rate, as each new element of the system needs to be examined in the context of all of the existing system. If the developers’ knowledge does not grow at this rate, it can fall behind that needed to support further changes. In practice, however, developers typically assume that the parts of the system which they believe not to be affected by a change are actually not so affected. The reduction in knowledge coverage may over time result in a slowing down of changes to the system, as knowledge needs to be gained before work can start or errors made due to
insufficient knowledge have to be rectified. In addition, some of the knowledge needed may only be available within the system code [16: p.284], and this information may need to be reconstructed by an extended examination of that code before substantive work on the change can start. These factors may lead to a less efficient software evolution process. A model which simulates this mechanism, and its effect on product size output, is shown in Figure 4. Here, the input resource available for system evolution is represented by some number of people (RESOURCE). The ability of this resource to produce changes to the system is modified by the degree of knowledge held on the system (proportion of system about which knowledge is held) scaled by a constant value to reflect average throughput, and particularly the degree to which the developer productivity is influenced by
knowledge about the system being enhanced (KNOWLEDGE PRODUCTIVITY FACTOR). As work is performed to increase the system size, the knowledge of developers about that system also grows, but at a slower rate determined by the PROPORTION OF REQUIRED KNOWELDGE GAINED parameter. The square of the knowledge concerning the system divided by the square of the system size forms an input to the calculation of the effectiveness of effort applied to further
increase the system size; the use of the squared values is based on the inverse square factor used in Turski’s model of software evolution [17] to reflect the effects of growing complexity of the software system as it is evolved. As a result of the simulated diminishing knowledge of the system, all else being equal the growth in the size of the system progressively falls away over time as is shown in the model output.
System Size
growth in system size effective effort
KNOWLEDGE PRODUCTIVITY FACTOR
RESOURCE
proportion of system about which knowledge held
Knowledge about System
growth in knowledge about system
PROPORTION OF REQUIRED KNOWLEDGE GAINED
240 180
120
60
0 0
24
48
72
96 120 144 Time (Month)
168
192
216
240
System Size : knowloss square2
Figure 4. Decreasing developer knowledge of system
5. Combining the Simple Models Having described a number of hypothesised causes of software evolution trends in the form of simulations, it is now possible to combine these into a single model. In addition to the simple causal structures, this combined model needs to reflect interactions between the theories, and to support investigations into the relative weighting of each cause on the
overall system evolution trend. Interactions between policies, such as conflicting demands on a restricted number of available staff, also need to be considered at this stage and further policies to be simulated, to reflect the effect of explicit or implicit management decisions relating to this trade-off. A first example of combining the simple models described above, and the simulated output for size of implementation, are shown in Figure 5. For this initial study exploring the
feasibility of the approach, the combination of the simple models into the combined model was performed in a straightforward manner. A new model containing the core function of the simulation (the system implementation; Figure 1) was created. The structures and functions in each simple model surrounding and driving the implementation process were then added around this core, producing a single model combining all of the structure of the simple models, and the equations of the core structure modified to reflect the combined effect of all of the causes. The space-filling structure (Figure 2) modifying the rate at which new functionality was added as an input to the system implementation. The reducing efficiency (Figure 3) and knowledge gap (Figure 4) aspects were added so as to act to reflect the interactions between their causal mechanism, which required some revisions to their structure as they were combined. In adding the simple models to the final combined model, changes were necessary to those input parameters to the simple model structures which enabled the trend to be followed, in order to bring the separately-developed simple models into comparability with each other. Initial values also needed to be harmonised. In order to support calibration of the combined model to reflect the relative importance of the different causes for real-world software evolution processes, a weighting parameter was added to the contribution of each simple model to the combined effect. These weightings are shown in italics in the model diagram (Figure 5) to distinguish them from other model input parameters. The model output of system size, which as noted in Section 2.2 above is shown as a generalised proof of concept, is shown in Figure 5. The causal weightings have been set to values for all of the causes included in the model which allow the implementation size output to follow the expected trend, as a demonstration that this structure can reproduce actual results. To provide an initial default calibration, policy-related model parameters have been set to represent broadly neutral values; the degree of anti-regressive activity has been set to nil, and no exogenous events have been simulated. The simulated trend shows that the varying sub-trend superimposed on the typical falling-off trend of the space-filling model has been smoothed by the effects of other elements of the model, but that the overall behaviour of falling-off in growth is still retained. This combined model, together with its calibration and output behaviours, are the subject of on-going empirical evaluation.
6. Using the Model in the Real World This section describes a process by which it is intended to use the approach set out in this paper to develop models which simulate the causes of evolution trends of specific software systems. The intention at this stage will be to develop a simulation which can replicate the broad trend in the output of a software evolution process over time for explanatory purposes, rather than one which is able to reproduce accurately all of the detail in the changes in system size over time. The latter would be likely to require a more
detailed and complex model whilst providing no additional explanatory power. The process of developing a model such as that described here for a real-world situation is as follows: identify candidate causes with the help of process and product experts, including marketing people, technical people and customers if available; this should be done to produce the broadest possible range of candidate causes; simulate each candidate cause by reusing or building a simple SD model, and check by an estimate-based calibration that it can reflect the behaviour anticipated by the experts; combine the simple models into a combined model using the mechanism described above; calibrate the full model. Possible approaches to calibrating the model input parameters include: ¾ the trend in product size metrics for software releases can be obtained by examining release-specific data from sources such as binary distributions, product build lists, CASE tool records etc.; ¾ weightings: the relative effects of the different causes, and the changes in these over time, can be based on developers’ and other experts’ opinion; this approach has worked in the calibration of previous SD global software process models [12]. It would also be necessary to identify points in the software evolution history at which changes have occurred in the behaviour of the global process as shown in the evolution trend (cf. [10]), and consider whether these may have been caused by changes in the weightings of the different causes or by exogenous events; ¾ trends in rates of filling of and increase in real-world search space can be determined by examining current/past trends in receipt of change requests from customers and users, augmented by marketing and system analysis expert opinion to analyse this trend and anticipate future changes to it; a similar approach could be adopted for analysis and design space trends. It may also be possible to use an indirect way of determining these trends, such as the rate of increase in work needed to identify a change, analyse the necessary requirements and design a solution; ¾ discovery and acceptance factors: the analysis-to-design and design-to-implementation factors can be calculated from records of numbers of change requests received, passed to design and actually implemented; equivalent factors for the real-world space will need to be estimated by process and product experts, possibly from marketing and other departments not necessarily directly involved in technical aspects of software development; ¾ effort expended on specific activities which may alter the behaviour of the process without directly contributing to size increases, such as the ANTI-REGRESSIVE POLICY parameter referred to
above, may be derived by examining source code check-in records, work schedules, project management documentation and timesheet/activity record data; the effects of this work on process behaviour may be determined by looking at specific instances of the work and its results or by consulting process and product experts; ¾ factors, such as the anti-regressive and efficiency factors, will need to be calibrated for model fit, on the basis of actual or believed trends agreed with process experts; and draw conclusion about the nature and relative importance of the causal mechanisms underlying the evolution of the software product under consideration. Many of the calibration mechanisms to be employed will inevitably be subjective, rather than being based on hard data from proposed process or product metrics collection programmes. However, experience suggests that the collection of these subjective metrics has itself resulted in developers and process owners thinking about their systems in a different way from previously, and has thus resulted in learning, in addition to producing estimates which have supported model behaviours reflecting actual or anticipated real-world outcomes [12]. As noted above, the calibration of the models presented here has been against the broad trends observed in such processes in the past. However, the benefits of such models can extend beyond the ability to simulate the behaviour of actual processes. Even without the calibration of models developed using this approach, the development of the models will in itself allow insights to be gained as to the nature of the actual processes underlying the behaviour of software evolution processes, and the interactions between the effects of the simple causes which can be identified (cf. [18]).
7. Next steps In order to ground the broad hypotheses and the individual factors described in this paper, it will be necessary to: identify with the assistance of practitioners and theorists further candidate causes/factors underlying the observed behaviour; simulate each in the simple manner exemplified above; gather evidence in the real world for or against the causal mechanisms hypothesised for each factor, by calibrating each individual model in real-world or empirical environments; update the combined model to incorporate elements representing those hypothesised causes found to be supported by observed phenomena or empirical studies; and modify the initial hypotheses concerning the weightings of and interactions between the factors, initially on the basis of discussions with process experts and later by calibrating the combined model formed of validated factor sub-models against real-world software product trends.
Other work which can be undertaken using the approach adopted here to develop a combined model includes examining whether the approach, and the models developed to implement it, help explain software evolution trends which do not follow the typical trend described above. Were a model to capture all of the significant causal mechanisms controlling software evolution processes, it should then be able to reproduce trends in software product growth from different evolutionary environments, such as open source. An example of an atypical trend is in the evolutionary growth of the Linux Kernel, which shows exponential growth [13]. The current state of the process of combining simple models is not yet systematised. However, it is able to show the relationship between simple models and the combined model diagram. It may be possible in future to improve the process of combining the models, for example, by using the Vensim array facility to produce a combined model which expresses the common core process variables as arrays, and calculates the combined effect by summing or multiplying the array elements as necessary. This approach would make it easier to add new causal sub-models.
8. Conclusion The approach described here to simulating the high-level structures and behaviour of long-term software evolution processes has enabled the development of a simulation tool which will be capable of reflecting the differing contributions of the different causes of software evolution, adding explanatory power to the growing contribution of simulation to the understanding of software evolution (cf. [12, 6, 5]). It also accomplishes a step in the development of simple but comprehensive models of these processes. It builds on a software evolution simulation which incorporated specific details of one of the evolution-related policy mechanisms [5] to describe an environment into which new causal mechanisms will be able to be added as their existence, causal structure, and relative effects are identified.
References [1] Gall H, Jazayeri M, Klösch RR and Trausmuth G, “Software Evolution Observations Based on Product Release History”, Proc. ICSM 1997, 160–166, 1997. [2] Mattsson M and Bosch J, “Observations on the Evolution of an Industrial OO Framework”; Proc. ICSM 1999. [3] Lehman MM, Ramil JF and Wernick P, “Metrics-Based Process Modeling With Illustrations From The FEAST/1 Project”, in Bustard D, Kawalek P and Norris M (eds.). Systems Modelling for Business Process Improvement, Artech House, Norwood, MA, 147–170, 2000. [4] Lehman MM, “Feedback in the Software Evolution Process”, Information and Software Technology, 38 (11), 681–686, 1996. [5] Kahen G, Lehman MM, Ramil JF and Wernick P, “An Approach to System Dynamics Modelling in the Investigation of Policies for E-type Software Evolution”, J. Systems and Software, 59 (3), 271–281, 15 December 2001. [6] Chatters BW, Lehman MM, Ramil JF and Wernick P, “Modelling A Software Evolution Process”, Software Process: Improvement and Practice, 5 (2–3), 91–102, 2000.
[14] Schach SR and Tomer A, “A Maintenance-oriented Approach to Software Construction”, J. Software Maintenance, 12, 25–45, 2000. [15] Lehman MM, Perry DE and Ramil JF, “On Evidence Supporting the FEAST Hypothesis and the Laws of Software Evolution”, Proc. Metrics ’98, Bethesda, Maryland, 84–88, 20-21 Nov. 1998. [16] Visaggio G, “Ageing of a Data-intensive Legacy System: Symptoms and Remedies”; J Softw. Maint. and Evol., 13, 281–308, 2001. [17] Turski, WM, “Reference Model for Smooth Growth of Software Systems”, IEEE Trans. on Software Engineering, 22 (8), 599–600, 1996. [18] Coyle G, “Qualitative and Quantitative Modelling in System Dynamics: Some Research Questions”, Systems Dynamics Review, 16 (3), 225–244, 2000.
[7] Ventana Systems, Inc., www.vensim.com, accessed 3 February 2003. [8] Abdel-Hamid TK and Madnick, SE, Software Project Dynamics – An Integrated Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1991. [9] Fowler M, Refactoring, Addison-Wesley, Boston, MA, 2000. [10] Lehman MM and Ramil JF, “An Overview of Some Lessons Learnt in FEAST”, Proc. WESS '02, Montreal, 2nd Oct 2002. [11] Ahmed R, Hall T and Wernick P, “A Proposed Framework for Evaluating Software Process Simulation Models”, Proc. ProSim 2003, 2003. [12] Wernick P and Lehman MM, “Software Process Dynamic Modelling for FEAST/1”, J. Systems and Software, 46 (2/3), 193–202, 1999. [13] Godfrey MW and Tu Q, “Evolution in Open Source Software: A Case Study”, Proc. ICSM 2000, 2000.
DOMAIN DISCOVERY DELAY
EXOGENOUS EVENTS
rate of addition
MULTIPLIER FACTOR
analysis space increase UNDISCOVERED DOMAIN WEIGHTING FACTOR
DOMAIN DISCOVERY FACTOR
Undiscovered Domain
rate of discovery
Unfilled Analysis Space
ANALYSIS DISCOVERY DELAY
analysis space filling
ANALYSIS DISCOVERY ACCEPTANCE FACTOR
Unfilled Design Space
design spac e increase UNFILLED ANALYSIS SPACE WEIGHTING FACTOR
DESIGN FILLING DELAY
design space filling
DESIGN DISCOVERY ACCEPTANCE FACTOR
To Be Implemented
implementation work to do
Implementation rate of implementation
IMPLEMENTATIO N DELAY
UNFILLED DESIGN SPACE WEIGHTING FACTOR KNOWLEDGE PRODUCTIVITY WEIGHTING
proportion of system about which knowledge is held
effective effort
KNOWLEDGE PRODUCTIVITY FACTOR
RESOURCE
growth in knowledge about system PROPORTION OF REQUIRED KNOWLEDGE GAINED
Knowledge About System
efficiency of implementation factor
EFFICIENCY OF IMPLEMENTATION WEIGHTING
ANTI-REGRESSIVE POLICY
Efficiency of Implementation improvement in efficiency
deterioration in efficiency
ANTI-REGRESSIVE FACTOR
EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR
75
50
25
0 0
24
48
72
96 120 144 Time (Month)
168
192
216
Implementation : space and size
Figure 5. The combined model and its output
240
Keywords software evolution, system dynamics, global software process, simulation, modularity
1. Introduction Studies of the long-term evolution of a number of commercial software products over many releases have established that there is a degree of consistency in the growth trend observed in their size. This trend shows a progressive falling-off over time of growth in the size of the system,
meaning that the system has relatively fewer new functions added as it grows older. The trend has been observed in a variety of types of software product evolved in differing technical environments [1, 2, 3]. Simulation-based research conducted so far into the global software process 1 [4] to determine possible reasons for this consistency has considered one possible cause, viz. the degradation of high-level system structure over time due to repeated changes [5]. However, that work does not necessarily deny the existence of a larger number of causes, each of which may contribute to the observed trend. The work described here forms the first step in the development of a System Dynamics (SD) simulation reflecting the structure, parameters and effects in combination of the causes of software evolution trends. The possible causes considered in the first model presented here are the reduction over time in both the conceptual space which can be searched for new uses of a system and the design spaces constraining developers’ ability to modify their systems, the effects of the degradation of the structure of a software system as is it is evolved in unanticipated directions, and the reducing coverage over time of developers’ knowledge of the structure of the system. The approach taken is to produce a simple simulation for each of these causes, calibrate it to follow the trend expected from such a mechanism, and then to combine these models into a single model reflecting all the causal mechanisms presented here and introducing a relative weighting for each. This approach has the advantage of exposing each of the proposed causes and its simulation to examination both in isolation and in the context of the other possible causes. The combined model has been calibrated as an initial proof of concept against typical software evolution trends, using parameter values taken in part from a previous simulation model [6]. The approach and the models presented here are intended in the first instance as a tool to improve understanding of the causal mechanisms underlying software evolution processes. Future development of this model will therefore include the development of additional simple models to simulate more causes as these are identified and the incorporation of these 1
This process comprises both the technical process by which software is produced, and the mechanisms and activities surrounding and controlling this technical process.
into the combined model. Calibration of this model to reflect the relative importance of each reason in a real-world situation should then enable it to reproduce the actual software system size trend. With this instrument it may thereby be possible to identify those causes which affect most significantly growth trends in an actual system within an individual situation. In the longer term, it may be possible to devise a simulation reflecting both the causes and the policies which can be used to affect the results of such processes; an example of a simulation of the effects of a policy decision is given in Section 4.2.1 below. By helping managers understand the effects of policy decisions on product and process metrics, this model may then form the basis of initiatives to improve the performance of specific software evolution processes. The rest of this paper is structured as follows. The approach taken is outlined in Section 2. The structure, parameters and calibration of a number of simple models are described in Sections 2.2, 3 and 4. A description of how these simple models have been combined into the final model, and the further calibration work needed to support this appears in Section 5. A discussion of how to use this model in the real world is contained in Section 6, the next steps envisaged for this research are in Section 7, and final conclusions of the work to date are presented in Section 8.
2. Outline of the Approach 2.1. The Approach Described This section describes the way in which System Dynamics has been employed, why this approach was taken, and the expected benefits of the approach. The approach is based on the view that the trends observed in software size over time are the result of a number of causes, each of which contributes to the trend to a degree which may vary over time. Thus, the causal mechanisms of the actual trend cannot be captured in a model which either reflects only one of these causes or which fails to allow the degree of contribution of each cause to be varied over time. The approach has been to produce a simulation of the causal mechanisms underlying the behaviour of a global software process which is made up of a number of sub-models. Each of these sub-models is intended to capture both the mechanism and the dynamic behaviour resulting arising from one possible cause in as simple a manner as possible, and is initially constructed as a stand-alone SD model using the Vensim simulation environment [7]. The simple sub-models are then combined into a single model, which thus captures the behaviour resulting from the identified causes of enhanced software size acting in combination. As part of this combination, the contribution of each causal sub-model to the simulated growth trend is weighted by a factor (which forms a parameter to the model), to enable the simulation builder to reflect the degree to which that cause affects the overall software evolution trend. By changing these weightings over simulated time, the changes in the relative degree of importance of the different causes over long periods of time can be reflected in the combined
simulation. This can include situations in which the changes in the values of the weightings over time are non-linear. Causes included in the simulation but which are believed to have no effect on the simulated evolutionary environment at any particular time can be eliminated from the calculation by setting their weightings to zero. The advantages of this approach include: each simple model is quick and easy to construct, understand, calibrate (by having fewer parameters requiring data collection exercises) and modify; the combined model will also be likely to be simpler in structure than a model constructed using techniques currently employed for developing simulations of software processes (see, for example, [8]; once the process of adding more causes to the combined model is systematised (see Section 5 below), it will also be simple to reflect new understanding about the global software process when compared with the development of a new SD model; taking an approach which as simply as possible simulates a believed cause and reproduces the expected software evolution trend arising from that cause results in a model which is easier to understand and to reason about than the combined model which is necessarily more complex in structure and behaviours. The ease of understanding and reasoning about the simple models will make it easier for people not experienced in simulation techniques, such as software developers, marketing people, etc. who are involved in the global software process to understand how the combined model’s overall structure and behaviours arise and can be influenced; it is easier using a model constructed as described here to understand the effect on software evolution of different causes individually and in combination, for example examining whether under a particular set of circumstances anti-regressive activities [5] are worth undertaking in the light of their relative effect on long-term system growth; the approach described here both simplifies the breaking down of some causes, which have previously been treated as monolithic, of observed software system size trends into combinations of lower-level causes, such as the hierarchy of filling of conceptual spaces described in Section 4 below. It also allows interactions between causal mechanisms, such as the effect of anti-regressive activities [5] on analysis and design activities, to be examined; the combined model, with each cause weighted relative to the others, can reflect circumstances in which the causes change in relative importance over time, e.g. increased loss of system architectural integrity and falling behind developer knowledge as the system ages, or lessening as anti-regressive work such as refactoring [9] is undertaken; and by adding new causal mechanisms which reflect possible future behaviours in the global software process and/or by changing the weightings of the different causes, it will be possible to investigate ‘what if’ situations such as the possible effects of decisions to make major changes to a
software system. However, care will need to be exercised in conducting and interpreting the results of such experimental simulation activities, since research suggests that when major changes are made to a system the trend in software product size may itself change significantly [10] (cf. [6]). Some of these reflect the benefits of a modular approach to simulation model development [11].
2.2. Basis of Model Calibration At this stage, the calibration of the models has been on the basis of proof-of-concept, rather than by reference to specific real-world software evolution processes. One aim is to demonstrate that, when provided with input parameter values which are reasonable on the basis of past experience, each simple model can represent expected trends based on the mechanism it represents. Since the simple models each only capture some part of the actual global process behaviour, they should not be expected individually to capture all of the detail in real-world software system evolution trends. However, the ability of each simple model to replicate the expected behaviour adds to the confidence in its reasonableness. The other aim of the calibration work has been to ensure that the combined model can reflect the widely-observed evolution trend referred to above. The approach to calibration has therefore been to use typical values derived from previous SD simulations of global software processes [12, 6] where possible for model input parameters. For example, the relationships between the time delays for the need for changes to be recognised and acted upon in the reducing spaces models (see Section 4.1 below) are derived from values used in the simulation of VME evolution [6]. At this early stage in the development of these models, and in the absence of a real-world process against which to calibrate the models, other parameters such as the EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR2 shown in both Figure 3 and Figure 5 have been have been calibrated to enable the model to fit the typical evolutionary trend, as in earlier work [6]. The simulated timeframe for the models is similarly abstracted, but is intended to represent real elapsed time (as against release sequence numbers) over some considerable period of time, typically several years or more. The output value of the simulations which has been used to test the reasonable operation of the simulation is the size of the system, which is shown to follow the typical trends of reducing growth noted previously. Again, this output is a generalisation of the observed trend, rather than representing actual values for the evolution of a real-world software system. The unit of measure of system ‘size’ is not defined in the models as a specific product metric, but is intended to represent some suitable measure related to the physical size of the system. In previous research into software evolution trends, this system size metric has been measured in terms of relative values over time [12], numbers of programming units 2
This constant parameter value is a scaling factor from the square of the current system size to the effect of growing system size on the effectiveness of developers’ work; see Section 4.2.1 below.
in the form of modules [6], object-oriented classes [2] or programs [1], and non-comment source code lines [13]. Generally, all of these measures have been seen by the researchers concerned as analogues of some measure of the ability of the system to do different types of useful work, i.e. the functional power of the system. This is captured in generalised form in the system size outputs of the models presented here. To summarise, the proof-of-concept calibration of these models is not against any specific evolution processes. However this calibration has demonstrated that the general trends in output behaviour which can be produced by these models reflect real-world observations.
3. The Simple Models Introduced 3.1. Introduction Each of the models presented below captures and simulates in a simple manner the result of one possible cause of the evolutionary trends described above, whilst abstracting all other features. The models reflect specific examples of two broad hypotheses as to the cause of the observed trend in software product size: a reduction over time in the possible changes which might be made to the system, each represented by the size of a conceptual space which has to be searched to find new possible or useful system enhancements; and the increasing size of the existing software system over time making it more difficult to change and/or to understand the effects of proposed changes.
3.2. Common Features of the Simple Models In order to simplify the process of combining the simple models, each was constructed around a common conceptual core, which has been augmented as necessary to represent the causal mechanisms underlying each identified influence on the behaviour of the software evolution process. This represents a flow of completed new system enhancements into a level representing the size of the system. This common core is shown in Figure 1. Its structure is similar to that of previous software evolution process simulations [12, 6] in which an SD third-order time delay function simulated the multistage software production process. Here the structure abstracts only the implementation of the software from an existing design. It represents using a third-order delay a flow of work being done on the system and the reflection of this work in system size and usable system functionality. The implementation, testing and preparation for release of new system functions are thus all abstracted into a single delay function represented by the valve in the diagram.
Implementation rate of implementation Figure 1. Common core structure of simple software evolution models
4. The Simple Models Described 4.1. Simulating Reducing Search Spaces There are three conceptual spaces within which a software-based system may be considered to be designed, implemented and fielded. The first of these is the user application space, i.e. the real-world environment within which the system is used. Evidence for this real-world problem space behaving as if it were of finite size arises from a previous software evolution study [6], in which the potential for use of a system in a new physical environment of itself resulted in an increase in demand for changes to that system. In a manner analogous to that real-world system user space, it is reasonable to suggest that there also exists an analysis model space, created by the software systems’ analysts as a conceptual model reflecting the analysts’ understanding of the real world. This analysis space is in turn used to bound the design of the system and its interactions with its environment, creating the design model space, which encompasses the high-level design of the software artefact itself, including the architecture and other design elements and which places limitations on how the system can be evolved without major restructuring. The implementation (code) of the system is seen here as an output of the design process; the obstacles to evolution are common to both, and the code is therefore only modelled as a simulation output. The rationale for the effect of a progressively reducing space on analysis and/or design options is that each explicit or implicit design decision made to incorporate some element in a model (or to leave it out) reduces the options available to make further changes. This viewpoint was the driver of a successful calibration of effort against actual values in a previous SD model of a real-world software process [12]. The size of each space places a limitation on the possibilities for the system’s use or capabilities (cf. [14: p.27]). These spaces are also effectively nested; the analysis space, within which the analysts consider what the system can or should do, will be bounded within the real-world possibilities, and the design space similarly bounded within the analysis space [14, p.28]. Since each of these spaces is (or behaves as if it were) finite, it is meaningful to speak of any one space being ‘filled up’ by the current system to the extent that further additions become increasingly difficult to identify
or make. It may however be possible to reverse this slowdown in the ability to add to the system by expanding the space by, for example, identifying new uses, analysis options and/or design possibilities for the system. Figure 2 shows an SD model which combines the effects of changes in all three of these spaces on system growth, and which reflects in addition the interactions between changes in the model spaces and the delays in these interactions. The model represents the real-world possibilities for the system to be useful (with changes if necessary) as a level called ‘Undiscovered Domain’, indicating that the developers have not yet been told or realised that the system could be useful in more situations than is currently the case. This level is increased by a feedback effect which reflects the FEAST hypothesis [15] that feedback from the release of new system functionality will of itself cause additional change demand. Here, the changes in the system create more possibilities for the system to be used if it is subjected to further change. The level is also increased by events in the outside world, such as changes in tax legislation for a payroll system. The level of undiscovered changes is reduced by the developers being told by customers, or otherwise discovering, that the system needs changes. This discovery takes time, which is reflected in a time delay in the model, and increases the level of unfulfilled analysis work demand (‘Unfilled Analysis Space’ in the model). Work done on turning analysis work demanded into designs for changes to the system reduce this level; the time taken to perform this work is reflected in a parameter to a standard SD delay function. The model must now reflect the design activity, which is simulated with a structure close to that of the analysis modelling. An effect of the analysis work being performed is the addition of work to be done to an Unfilled Design Space level parallelling the Unfilled Analysis Space; the amount of design work added is the analysis work done multiplied by some constant factor. In a manner exactly analogous to that of the analysis simulation, work done in designing the system adds to implementation work to be done. Finally, the implementation work, which again takes time, adds to the size of the actual system code. At each stage, a discovery factor determines the proportion of elements which might possibly progress to the next stage; this reflects the observation that not all useful changes will be detected by developers or users, that some of the change requests from users are not actioned for reasons often involving development management organisation decisions,
and so on through the process. These parameters are set to increase the likelihood of a change being implemented as it passes through the process, reflecting for example the higher probability that a change which has completed the design stage will be implemented and incorporated into the system code. As noted above, the increased functionality which all of the changes have finally reflected in the system will result in a feedback-driven pressure for more system changes (which is represented in the model by a figure made up of the increase in system size multiplied by a constant factor), thus restarting the cycle. The output trend shown here represents a situation in which the size of the system in source and object code terms is not ultimately constrained, and can rise to meet the needs of the users; this is typical for commercial and business systems. The model would need to be modified for situations in which
the output software system size is constrained, for example by a fixed physical memory size [12]. In that case, once the memory of the system was filled, no further growth in physical size would be possible. Were it still to be possible to add to the system’s functional power, for example by reducing the size of other parts of the code to make space for new functions, this might require the system size represented here to be denominated in some measure other than that of the physical code. The structure and outputs of the combined reducing space model are shown in Figure 2. The output graph of system size over time, shows, after an initial period for the dynamics behaviour to settle down, a broad trend reflecting the characteristic decrease in rate of growth of the system; the ripple in this trend is due to the interactions of the various delay times in the flows in the model.
DOMAIN DISCOVERY FACTOR
EXOGENOUS EVENTS
rate of addition
Undiscovered Domain
rate of discovery DOMAIN DISCOVERY DELAY
MULTIPLIER FACTOR
analysis space increase
Unfilled Analysis Space
ANALYSIS DISCOVERY ACCEPTANCE FACTOR
design space increase
analysis space filling
Unfilled Design Space
ANALYSIS DISCOVERY DELAY
design space filling DESIGN FILLING DELAY
DESIGN DISCOVERY ACCEPTANCE FACTOR
implementation work to do
To Be Implemented
Implementation rate of implementation
INITIAL IMPLEMENTATION RATE
IMPLEMENTATION DELAY
Implementation 200 150
100
50
0 0
25
50
75
100 125 150 Time (Month)
175
200
225
250
Implementation : combinedspace3 with ddf2
Figure 2. Reducing search spaces
4.2. Reducing Efficiency due to Growing Product Size The models described in this section reflect the broad hypothesis that the increasing size of a software system and changes in unanticipated directions will over time act to reduce the ability of developers to modify that system. This decline may be due to the increasing complexity of the system as its structure is degraded by efforts to make changes unanticipated when the system architecture was designed and/or a decrease in coverage of developers’ knowledge of the system components, their composition and interactions. 4.2.1. Effects of increasing system complexity. As a software product is evolved over time, some of the changes made to it will inevitably not have been identified at the time the architecture of the software was designed. These changes may result in a decay in the system architecture as its abstractions become less relevant and/or its encapsulations need to be infringed. New changes also have to be fitted into an existing system structure, and as the system grows there is more existing structure into which each new change needs to be fitted, thereby constraining the freedom to make these changes (cf. [15]). The model shown in Figure 3 reflects the result of these influences. In the model, a value which represents the efficiency of implementation effort (Efficiency Of Implementation) is reduced by implementation activities. One mechanism which has been suggested to address this problem is to employ software developers on tasks specifically intended to maintain the structure of the system,
for example by reducing coupling and cleaning up a layered architecture, or by restructuring object-oriented classes and their interactions in ‘refactoring’ exercises [9]. To contrast with ‘progressive’ system enhancement work, this is referred to as ‘anti-regressive’ activity. The effect of anti-regressive activity (or its lack) on system evolvability has been explored previously [5]; the simplified model presented here includes a structure intended to represent its effect. Staff effort (STAFF, in units representing numbers of people) is initially directed to adding system functionality. If some of this effort is redirected from adding new functionality to anti-regressive effort, less implementation work is done but the efficiency of future implementation work is increased. In the model, the ANTI-REGRESSIVE POLICY parameter is used to represent management policy in terms of the proportion of effort devoted to anti-regressive work The model output shows the effect on system growth as simulated in the model if there is no anti-regressive activity (line antireg0). It also shows the greater growth in system size later life made possible by changing the staff allocation policy towards anti-regressive work starting at simulated time (month) 50, employing 25% of the effort applied to the project (line antirg25). This result parallels that found by [5] in their simulation-based work i.e. that long-term system growth is increased. However, the model presented here is deliberately much simplified compared with theirs. In both cases, the change is a result of the simulation of a deliberate management decision to redeploy effort from one activity to another.
EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR
ANTI-REGRESSIVE FACTOR
improvement in efficiency
Efficiency of Implementation
deterioration in efficiency
Implementation rate of implementation ANTI-REGRESSIVE POLICY
STAFF
<Time>
240 180 120 60 0 0
20
40
60
80 100 120 Time (Month)
140
160
180
200
Implementation : antireg0 Implementation : antirg25
Figure 3. Effect of growing system complexity 4.2.2.
Potential effects of decreasing developer knowledge of a system. In order to be able to change a system, its developers need access to knowledge about that system. As the system grows, the amount of knowledge needed to support further changes also grows, but at a faster rate, as each new element of the system needs to be examined in the context of all of the existing system. If the developers’ knowledge does not grow at this rate, it can fall behind that needed to support further changes. In practice, however, developers typically assume that the parts of the system which they believe not to be affected by a change are actually not so affected. The reduction in knowledge coverage may over time result in a slowing down of changes to the system, as knowledge needs to be gained before work can start or errors made due to
insufficient knowledge have to be rectified. In addition, some of the knowledge needed may only be available within the system code [16: p.284], and this information may need to be reconstructed by an extended examination of that code before substantive work on the change can start. These factors may lead to a less efficient software evolution process. A model which simulates this mechanism, and its effect on product size output, is shown in Figure 4. Here, the input resource available for system evolution is represented by some number of people (RESOURCE). The ability of this resource to produce changes to the system is modified by the degree of knowledge held on the system (proportion of system about which knowledge is held) scaled by a constant value to reflect average throughput, and particularly the degree to which the developer productivity is influenced by
knowledge about the system being enhanced (KNOWLEDGE PRODUCTIVITY FACTOR). As work is performed to increase the system size, the knowledge of developers about that system also grows, but at a slower rate determined by the PROPORTION OF REQUIRED KNOWELDGE GAINED parameter. The square of the knowledge concerning the system divided by the square of the system size forms an input to the calculation of the effectiveness of effort applied to further
increase the system size; the use of the squared values is based on the inverse square factor used in Turski’s model of software evolution [17] to reflect the effects of growing complexity of the software system as it is evolved. As a result of the simulated diminishing knowledge of the system, all else being equal the growth in the size of the system progressively falls away over time as is shown in the model output.
System Size
growth in system size effective effort
KNOWLEDGE PRODUCTIVITY FACTOR
RESOURCE
proportion of system about which knowledge held
Knowledge about System
growth in knowledge about system
PROPORTION OF REQUIRED KNOWLEDGE GAINED
240 180
120
60
0 0
24
48
72
96 120 144 Time (Month)
168
192
216
240
System Size : knowloss square2
Figure 4. Decreasing developer knowledge of system
5. Combining the Simple Models Having described a number of hypothesised causes of software evolution trends in the form of simulations, it is now possible to combine these into a single model. In addition to the simple causal structures, this combined model needs to reflect interactions between the theories, and to support investigations into the relative weighting of each cause on the
overall system evolution trend. Interactions between policies, such as conflicting demands on a restricted number of available staff, also need to be considered at this stage and further policies to be simulated, to reflect the effect of explicit or implicit management decisions relating to this trade-off. A first example of combining the simple models described above, and the simulated output for size of implementation, are shown in Figure 5. For this initial study exploring the
feasibility of the approach, the combination of the simple models into the combined model was performed in a straightforward manner. A new model containing the core function of the simulation (the system implementation; Figure 1) was created. The structures and functions in each simple model surrounding and driving the implementation process were then added around this core, producing a single model combining all of the structure of the simple models, and the equations of the core structure modified to reflect the combined effect of all of the causes. The space-filling structure (Figure 2) modifying the rate at which new functionality was added as an input to the system implementation. The reducing efficiency (Figure 3) and knowledge gap (Figure 4) aspects were added so as to act to reflect the interactions between their causal mechanism, which required some revisions to their structure as they were combined. In adding the simple models to the final combined model, changes were necessary to those input parameters to the simple model structures which enabled the trend to be followed, in order to bring the separately-developed simple models into comparability with each other. Initial values also needed to be harmonised. In order to support calibration of the combined model to reflect the relative importance of the different causes for real-world software evolution processes, a weighting parameter was added to the contribution of each simple model to the combined effect. These weightings are shown in italics in the model diagram (Figure 5) to distinguish them from other model input parameters. The model output of system size, which as noted in Section 2.2 above is shown as a generalised proof of concept, is shown in Figure 5. The causal weightings have been set to values for all of the causes included in the model which allow the implementation size output to follow the expected trend, as a demonstration that this structure can reproduce actual results. To provide an initial default calibration, policy-related model parameters have been set to represent broadly neutral values; the degree of anti-regressive activity has been set to nil, and no exogenous events have been simulated. The simulated trend shows that the varying sub-trend superimposed on the typical falling-off trend of the space-filling model has been smoothed by the effects of other elements of the model, but that the overall behaviour of falling-off in growth is still retained. This combined model, together with its calibration and output behaviours, are the subject of on-going empirical evaluation.
6. Using the Model in the Real World This section describes a process by which it is intended to use the approach set out in this paper to develop models which simulate the causes of evolution trends of specific software systems. The intention at this stage will be to develop a simulation which can replicate the broad trend in the output of a software evolution process over time for explanatory purposes, rather than one which is able to reproduce accurately all of the detail in the changes in system size over time. The latter would be likely to require a more
detailed and complex model whilst providing no additional explanatory power. The process of developing a model such as that described here for a real-world situation is as follows: identify candidate causes with the help of process and product experts, including marketing people, technical people and customers if available; this should be done to produce the broadest possible range of candidate causes; simulate each candidate cause by reusing or building a simple SD model, and check by an estimate-based calibration that it can reflect the behaviour anticipated by the experts; combine the simple models into a combined model using the mechanism described above; calibrate the full model. Possible approaches to calibrating the model input parameters include: ¾ the trend in product size metrics for software releases can be obtained by examining release-specific data from sources such as binary distributions, product build lists, CASE tool records etc.; ¾ weightings: the relative effects of the different causes, and the changes in these over time, can be based on developers’ and other experts’ opinion; this approach has worked in the calibration of previous SD global software process models [12]. It would also be necessary to identify points in the software evolution history at which changes have occurred in the behaviour of the global process as shown in the evolution trend (cf. [10]), and consider whether these may have been caused by changes in the weightings of the different causes or by exogenous events; ¾ trends in rates of filling of and increase in real-world search space can be determined by examining current/past trends in receipt of change requests from customers and users, augmented by marketing and system analysis expert opinion to analyse this trend and anticipate future changes to it; a similar approach could be adopted for analysis and design space trends. It may also be possible to use an indirect way of determining these trends, such as the rate of increase in work needed to identify a change, analyse the necessary requirements and design a solution; ¾ discovery and acceptance factors: the analysis-to-design and design-to-implementation factors can be calculated from records of numbers of change requests received, passed to design and actually implemented; equivalent factors for the real-world space will need to be estimated by process and product experts, possibly from marketing and other departments not necessarily directly involved in technical aspects of software development; ¾ effort expended on specific activities which may alter the behaviour of the process without directly contributing to size increases, such as the ANTI-REGRESSIVE POLICY parameter referred to
above, may be derived by examining source code check-in records, work schedules, project management documentation and timesheet/activity record data; the effects of this work on process behaviour may be determined by looking at specific instances of the work and its results or by consulting process and product experts; ¾ factors, such as the anti-regressive and efficiency factors, will need to be calibrated for model fit, on the basis of actual or believed trends agreed with process experts; and draw conclusion about the nature and relative importance of the causal mechanisms underlying the evolution of the software product under consideration. Many of the calibration mechanisms to be employed will inevitably be subjective, rather than being based on hard data from proposed process or product metrics collection programmes. However, experience suggests that the collection of these subjective metrics has itself resulted in developers and process owners thinking about their systems in a different way from previously, and has thus resulted in learning, in addition to producing estimates which have supported model behaviours reflecting actual or anticipated real-world outcomes [12]. As noted above, the calibration of the models presented here has been against the broad trends observed in such processes in the past. However, the benefits of such models can extend beyond the ability to simulate the behaviour of actual processes. Even without the calibration of models developed using this approach, the development of the models will in itself allow insights to be gained as to the nature of the actual processes underlying the behaviour of software evolution processes, and the interactions between the effects of the simple causes which can be identified (cf. [18]).
7. Next steps In order to ground the broad hypotheses and the individual factors described in this paper, it will be necessary to: identify with the assistance of practitioners and theorists further candidate causes/factors underlying the observed behaviour; simulate each in the simple manner exemplified above; gather evidence in the real world for or against the causal mechanisms hypothesised for each factor, by calibrating each individual model in real-world or empirical environments; update the combined model to incorporate elements representing those hypothesised causes found to be supported by observed phenomena or empirical studies; and modify the initial hypotheses concerning the weightings of and interactions between the factors, initially on the basis of discussions with process experts and later by calibrating the combined model formed of validated factor sub-models against real-world software product trends.
Other work which can be undertaken using the approach adopted here to develop a combined model includes examining whether the approach, and the models developed to implement it, help explain software evolution trends which do not follow the typical trend described above. Were a model to capture all of the significant causal mechanisms controlling software evolution processes, it should then be able to reproduce trends in software product growth from different evolutionary environments, such as open source. An example of an atypical trend is in the evolutionary growth of the Linux Kernel, which shows exponential growth [13]. The current state of the process of combining simple models is not yet systematised. However, it is able to show the relationship between simple models and the combined model diagram. It may be possible in future to improve the process of combining the models, for example, by using the Vensim array facility to produce a combined model which expresses the common core process variables as arrays, and calculates the combined effect by summing or multiplying the array elements as necessary. This approach would make it easier to add new causal sub-models.
8. Conclusion The approach described here to simulating the high-level structures and behaviour of long-term software evolution processes has enabled the development of a simulation tool which will be capable of reflecting the differing contributions of the different causes of software evolution, adding explanatory power to the growing contribution of simulation to the understanding of software evolution (cf. [12, 6, 5]). It also accomplishes a step in the development of simple but comprehensive models of these processes. It builds on a software evolution simulation which incorporated specific details of one of the evolution-related policy mechanisms [5] to describe an environment into which new causal mechanisms will be able to be added as their existence, causal structure, and relative effects are identified.
References [1] Gall H, Jazayeri M, Klösch RR and Trausmuth G, “Software Evolution Observations Based on Product Release History”, Proc. ICSM 1997, 160–166, 1997. [2] Mattsson M and Bosch J, “Observations on the Evolution of an Industrial OO Framework”; Proc. ICSM 1999. [3] Lehman MM, Ramil JF and Wernick P, “Metrics-Based Process Modeling With Illustrations From The FEAST/1 Project”, in Bustard D, Kawalek P and Norris M (eds.). Systems Modelling for Business Process Improvement, Artech House, Norwood, MA, 147–170, 2000. [4] Lehman MM, “Feedback in the Software Evolution Process”, Information and Software Technology, 38 (11), 681–686, 1996. [5] Kahen G, Lehman MM, Ramil JF and Wernick P, “An Approach to System Dynamics Modelling in the Investigation of Policies for E-type Software Evolution”, J. Systems and Software, 59 (3), 271–281, 15 December 2001. [6] Chatters BW, Lehman MM, Ramil JF and Wernick P, “Modelling A Software Evolution Process”, Software Process: Improvement and Practice, 5 (2–3), 91–102, 2000.
[14] Schach SR and Tomer A, “A Maintenance-oriented Approach to Software Construction”, J. Software Maintenance, 12, 25–45, 2000. [15] Lehman MM, Perry DE and Ramil JF, “On Evidence Supporting the FEAST Hypothesis and the Laws of Software Evolution”, Proc. Metrics ’98, Bethesda, Maryland, 84–88, 20-21 Nov. 1998. [16] Visaggio G, “Ageing of a Data-intensive Legacy System: Symptoms and Remedies”; J Softw. Maint. and Evol., 13, 281–308, 2001. [17] Turski, WM, “Reference Model for Smooth Growth of Software Systems”, IEEE Trans. on Software Engineering, 22 (8), 599–600, 1996. [18] Coyle G, “Qualitative and Quantitative Modelling in System Dynamics: Some Research Questions”, Systems Dynamics Review, 16 (3), 225–244, 2000.
[7] Ventana Systems, Inc., www.vensim.com, accessed 3 February 2003. [8] Abdel-Hamid TK and Madnick, SE, Software Project Dynamics – An Integrated Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1991. [9] Fowler M, Refactoring, Addison-Wesley, Boston, MA, 2000. [10] Lehman MM and Ramil JF, “An Overview of Some Lessons Learnt in FEAST”, Proc. WESS '02, Montreal, 2nd Oct 2002. [11] Ahmed R, Hall T and Wernick P, “A Proposed Framework for Evaluating Software Process Simulation Models”, Proc. ProSim 2003, 2003. [12] Wernick P and Lehman MM, “Software Process Dynamic Modelling for FEAST/1”, J. Systems and Software, 46 (2/3), 193–202, 1999. [13] Godfrey MW and Tu Q, “Evolution in Open Source Software: A Case Study”, Proc. ICSM 2000, 2000.
DOMAIN DISCOVERY DELAY
EXOGENOUS EVENTS
rate of addition
MULTIPLIER FACTOR
analysis space increase UNDISCOVERED DOMAIN WEIGHTING FACTOR
DOMAIN DISCOVERY FACTOR
Undiscovered Domain
rate of discovery
Unfilled Analysis Space
ANALYSIS DISCOVERY DELAY
analysis space filling
ANALYSIS DISCOVERY ACCEPTANCE FACTOR
Unfilled Design Space
design spac e increase UNFILLED ANALYSIS SPACE WEIGHTING FACTOR
DESIGN FILLING DELAY
design space filling
DESIGN DISCOVERY ACCEPTANCE FACTOR
To Be Implemented
implementation work to do
Implementation rate of implementation
IMPLEMENTATIO N DELAY
UNFILLED DESIGN SPACE WEIGHTING FACTOR KNOWLEDGE PRODUCTIVITY WEIGHTING
proportion of system about which knowledge is held
effective effort
KNOWLEDGE PRODUCTIVITY FACTOR
RESOURCE
growth in knowledge about system PROPORTION OF REQUIRED KNOWLEDGE GAINED
Knowledge About System
efficiency of implementation factor
EFFICIENCY OF IMPLEMENTATION WEIGHTING
ANTI-REGRESSIVE POLICY
Efficiency of Implementation improvement in efficiency
deterioration in efficiency
ANTI-REGRESSIVE FACTOR
EFFICIENCY DETERIORATION INVERSE SQUARE FACTOR
75
50
25
0 0
24
48
72
96 120 144 Time (Month)
168
192
216
Implementation : space and size
Figure 5. The combined model and its output
240
