Estimating Effort and Cost in Software Projects ... - PublicationsList.org

Report 5 Downloads 69 Views
Estimating Effort and Cost in Software Projects − ISBSG A Multi-Organizational Project Data Repository for Project Estimation And Benchmarking Briefing Centre for Software Process Technologies University of Ulster Northern Ireland 18 September 2003 Pierre Bourque École de technologie supérieure Montreal, Canada

Cumulative Concern Value

109

100 93

S/ w Pr oc re wo es sA rk dh er en ce Pr oj ec tP lan ni De En ng ve su lop rin g ing qu Re ali ty qu M ire an m ag en ing ts Re qu ire m en ts M ain Te ta in am in sc gs om /w m u Te nic ch ati no on log yc ha ng es M an ag ing fau lts

Es tim ati ng Ta Ri sk sk s m an ag em en t Pr od uc tiv W Pr ity or o j ec kp tT ra rac cti se kin co g ns ist en cy

Benchmarking the NI software industry Concern measures for Engineering Aspects

120 106

99 91 88 85 84 83

80

Centre for Software Process Technologies

81 79 74 69 65

60 49

40

20

0

Engineering Aspect

2

Ice Bag

ISBSG Centre for Software Process Technologies

3

École de technologie supérieure ETS • An Engineering University – Within the network of the Université du Québec • In + 10 cities across Québec

• Engineering Specialties: – – – – – – –

Aeronautical Engineering Automated Manufacturing Construction Engineering Electrical Engineering Mechanical Engineering Software Engineering Telecommunications + Information Technologies Centre for Software Process Technologies

4

ETS Software Engineering Lab. Strengths • Recognized world-wide in software engineering for: – Building consensus in software engineering – Leadership of world-wide initiatives – Strong applied research focus

• www.lrgl.uqam.ca

Centre for Software Process Technologies

5

Software Engineering Management • Sample of R & D topics: – Guide to the Software Engineering Body of Knowledge (SWEBOK) – Second generation of functional size method: COSMIC – Estimation models – Quality Engineering – Risk of measurement programs – Metrology

Centre for Software Process Technologies

6

Agenda • • • • •

Principles of Credible Estimation Overview of Software Functional Sizing Overview of ISBSG Overview of the Repository An example of using ISBSG for Duration Estimation • Conclusion Centre for Software Process Technologies

7

How do you build your estimates? • • • • • • • • •

How do you build estimates in your organization? What are the inputs to your estimates? Is your estimation process documented? Do you collect and use historical data? Do you collect data that is never used? What is the reliability of this data? Do you track your estimates? How do you size the amount of work to be done? Are the quality of your estimates very much based on the competency of a few key people in your organization? Centre for Software Process Technologies

8

What is software engineering ? • IEEE 610.12: (1) The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software. (2) The study of approaches as in (1).

Centre for Software Process Technologies

9

Many Commercial Estimation Tools on the Market • Many well-known commercial tools: – – – – – •

Checkpoint Project Workbench PQM Plus SLIM …

See O. Mendes, A. Abran and P. Bourque. Function Point Tool Market Survey. Université du Québec à Montréal,1996 Available at www.lrgl.uqam.ca/publications/pdf/204.pdf Centre for Software Process Technologies

10

What is the result? ¤ Lots of numbers, graphs, nice slide presentation, ¤ Industry data often not verifiable, ¤ “Black box” approach, ¤ Bottom line: everybody sees what they want to see...

Centre for Software Process Technologies

11

Mixed PUBLISHED Results... • Panoply of software cost models • Several studies have been conducted to assess the accuracy of these models on various databases • However, no study has proven the superiority of any models excepted for limited applications • Often small data samples Centre for Software Process Technologies

12

Underlying Principles of CREDIBLE Estimation - 1 As Defined in Park et al. (94): « Estimates are made by people, not by models. They require reasoned judgments and commitments to organizational goals that cannot be delegated to any automated process ». Centre for Software Process Technologies

13

Underlying Principles of CREDIBLE Estimation-2 « All estimates are based on comparisons. When people estimate, they evaluate how something is like, and how something is unlike, things that they or others have seen before ». Centre for Software Process Technologies

14

Underlying Principles of CREDIBLE Estimation-3 « Before people can estimate, they must acquire knowledge. They must collect and quantify information from other projects, so that they can place their comparative evaluations on demonstrably sound footings ». Centre for Software Process Technologies

15

These Principles Imply: • To be credible, an estimation process must inherently be white-box • Software project estimation which has been plaguing the industry for years can only be solved through a cooperative data collection effort • Much research still has to be done Centre for Software Process Technologies

16

Agenda • • • • •

Principles of Credible Estimation Overview of Software Functional Sizing Overview of ISBSG Overview of the Repository An example of using ISBSG for Duration Estimation • Conclusion Centre for Software Process Technologies

17

Size of what …

Context...

HOW BIG IS IT ?

Project Size The total effort, estimated or actual in work-hours or staffmonths Software size the size of the requirements (functions) or of the deliverables (modules, lines of code) Centre for Software Process Technologies

18

FUNCTIONAL

HOW BIG IS IT ?

TECHNICAL

Software size measurement Mmm… so many programs, so many lines of code... • Meaningful to the technical staff,

• Meaningless to management, • Poor portability, • Only known precisely when too late to use Mmm… so much functionality delivered to the users... • Meaningful to management,

• Meaningful to technical staff, • Portable, • Can be measured early on, • Must be independent from effort, method or technology

Centre for Software Process Technologies

19

The ‘Functional Size’ of software

Ø ISO/IEC/JTC1/SC7

Standard #14143

definition: “ Functional Size : A size of software derived by quantifying the functional user requirements”

An analogy...

2000 sq. ft. 4000 sq. ft.

Software Functionality

500 cfsu

Software Functionality

Centre for Software Process Technologies

1000 cfsu

21

Very Significant Amount of Work on Software Functional Size?” ISO ‘FSM’ Standard MkII FPA 1.3

3-D FP’s MkII FPA Feature Points Allan Albrecht FPA

1980

Full FP’s V.1

IFPUG 4.1

IFPUG 4.0

1985for Software Process 1990 Centre Technologies

COSMIC FFP V. 2

1995

2000 22

Usages of Software Sizing • • • •

Estimation Benchmarking Productivity Trend Analysis Contract Payment Mechanisms – Development – Corrective Maintenance and Support

• Quality Tracking Centre for Software Process Technologies

23

Productivity Trend Analysis FP Unit Costs

Hours/Extended FP

12 10 8 6 4 2 0 1990

40

1991

1992

1993

1994

1993

1994

Year 12

30

Extended FP

Effort

10 20

10

0 1990

1991

1992

1993

8 6 4 2

1994

Year

0

Centre for Software Process Technologies

1990

1991

1992

Year

24

Agenda • • • • •

Principles of Credible Estimation Overview of Software Functional Sizing Overview of ISBSG Overview of the Repository An example of using ISBSG for Duration Estimation • Conclusion Centre for Software Process Technologies

25

ISBSG Mission • “To help improve the management of IT resources by both business and government through the provision and exploitation of a public repository of software engineering knowledge which is standardized, verified, recent and representative of current technologies.” Centre for Software Process Technologies

26

ISBSG The 1st. Phase... Australian Software Metrics Association, (ASMA) • Motivation – practitioners • wanting control • seeking best practices

• Definition of Productivity – Project Delivery Rate = “Hours per Function Point”

Centre for Software Process Technologies

27

ISBSG The 2nd. Phase ...the ASMA Work • First Release of data (24 projects) in December 1992 • Revised standard & collection package several times • 6 Releases of Data up to June 1994

Centre for Software Process Technologies

28

ISBSG The 3rd. Phase Became the… International Software Benchmarking Standards Group ISBSG www.isbsg.org Centre for Software Process Technologies

29

International Membership Current membership: • Australia, Finland, Germany • India, Italy, Japan, Netherlands, • Spain, Switzerland, • United Kingdom, USA

Centre for Software Process Technologies

30

Principles of ISBSG • Principles of Evolution – – – – –

independence data integrity anonymity practitioner driven practitioner accessible

Centre for Software Process Technologies

31

Initial Principles of Evolution • “ Any project team anywhere in the world may, for insignificant cost, benchmark themselves against world’s best”

Centre for Software Process Technologies

32

ISBSG Existing Standard • Collection Philosophy • complete in 30 mins • meaningful, available, objective

• Project Attributes – – – –

Functional Size Work Effort, Team Size, language Cost Quality......Etc........... Centre for Software Process Technologies

33

Collection Process • • • • • • •

Security Procedures Validation Procedures Repository entry Project Benchmark Report Rating comments Re-submission Re-rating and project update Centre for Software Process Technologies

34

Summary - ISBSG Strengths • • • • •

Not profit motivated Provides Project Benchmarking Allows direct access to the data Allows networking Broad representation of IT – technologies, organisation types, geography

• International based standard Centre for Software Process Technologies

35

Agenda • • • • •

Principles of Credible Estimation Overview of Software Functional Sizing Overview of ISBSG Overview of the Repository An example of using ISBSG for Duration Estimation • Conclusion Centre for Software Process Technologies

36

ISBSG Release 8 • Demonstration of ISBSG Release 8 Data Set

Centre for Software Process Technologies

37

Age of Projects (Data from Release 7) REPOSITORY PROJECTS

year not known 1989

PROJECTS

1990

250

1991

200

1993

1992

1994

150

1995

100

1996

50

1998

1997

1999

0

2000

YEAR Centre for Software Process Technologies

2001

38

Projects by Country Australia Japan United States Netherlands Canada United Kingdom India France Brazil Other 0

50

100

150

200

250

300

Number of projects

Centre for Software Process Technologies

39

Projects by Organization Type Communication Government, Public Administration Financial, Property & Business Services Insurance Manufacturing Banking Electricity, Gas, Water Wholesale & Retail Trade Computers, ISP Ordering Billing Energy, oil, mining Voice provisioning Engineering Other 0 50 100 150 200 250 300 350 400 450 500 Centre for Software Process Number of projects Technologies

40

Projects by Business Area Telecommunications Insurance, actuarial Banking Financial (excluding Banking) Manufacturing Accounting Engineering Sales & Marketing Inventory Legal Personnel Research & Development Logistics Other 0

50

100

150

200

Number of projects Centre for Software Process Technologies

250

300

350 41

Types of Projects Enhancement

New development

Re-development

0

200

400

600

800

1000

1200

Number of projects

Centre for Software Process Technologies

42

Application Type Transaction/Production System Management Information System Office Information System Network Management Billing Ordering Electronic Data Interchange Process control, sensor control, real time Decision Support System Executive Information System Voice provisioning Knowledge Based System Other

0

100

200

300

400

Number of projects Centre for Software Process Technologies

500

600

43

Use of Development Techniques Data modelling Process modelling Business area modelling Event modelling Joint App'n Development Rapid App'n Development Object Oriented Analysis Object Oriented Design Prototyping Multifunction teams Regression testing Timeboxing Standards (ISO 9000; CMM) 0

50 100 150 200 250 300 350 400 450 Centre for Software Process Number of projects Technologies

44

The ISBSG Repository Positioning • Probably represents top 20% of industry • Primarily MIS Applications

Centre for Software Process Technologies

45

Agenda • • • • •

Principles of Credible Estimation Overview of Software Functional Sizing Overview of ISBSG Overview of the Repository An example of using ISBSG for Duration Estimation • Conclusion Centre for Software Process Technologies

46

Strategic Importance of Time-to-Market • Project manager’s dream: – – – –

Complete and stable product requirements High quality Low costs Short time-to-market

• Time to market or project duration is often the hardest one to pin down • A variation of the other three have a determining effect on it Centre for Software Process Technologies

47

Adopted Viewpoint in this Research Product Requirements

Product Size

Project Duration

Project Effort Adopted Viewpoint

Centre for Software Process Technologies

48

2. Selecting a Data Sample 2.1

ISBSG release 4 (1997)

2.2

Basic selection criteria

2.3

Distribution analysis



Effort



Duration



Summary Centre for Software Process Technologies

49

2.1 ISBSG Release 4 • Release 4 (1997) contains 396 completed projects – Contribution from 13 countries – 2/3 new development, 1/3 enhancements & redevelopment – 34% Txn proc., 38% MIS, 14% office information – 3/4 developped in-house for internal use – 67% Mainframe – 46% 3GL, 38% 4GL

Centre for Software Process Technologies

50

2.2 Basic Selection Criteria • No reasonable doubts on data validity according to ISBSG screening • Known effort, known duration and known platform

ä

Duration (D) in calendar months

Effort (E) in personhours

Number of observations (n)

312

312

Minimum value

1,0

10

Maximum value

78,0

106 480

Mean value

10,5

5 933

Standard deviation

9,0

12 169

Median

8,0

2 228

D range: 1 to 78 months E range: 0,1 to 761 person-months

312 projects satisfied all criteria Centre for Software Process Technologies

51

2.3 Basic Criteria Scatter plot of effort vs. duration (n=312) months 80 70

DURATION

60 50 40 30 20 10 0 0

20000

40000

60000

80000

100000

person-hours

EFFORT

Centre for Software Process Technologies

52

2.3 Distribution Analysis - Effort No transform

Value Significance (α