Technology Selection: An Educational Approach - Semantic Scholar
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
Technology Selection: WILLIAM E. RIDDLE,
An
Educational Approach
SENIOR MEMBER, IEEE, AND
Abstract-Creating and enhancing a software engineering work force requires several different types of continuing education for software professionals, including: task-oriented education, enhancement-oriented education and selection-oriented education.In this paper, we focus on the important, but often neglected, category of selection-oriented education. We begin with a discussion of technology selection, indicating what it involves, how it contributes to improving the state of practice, and why it is key to technology improvement in general. This is followed by a discussion of some criteria for selection-oriented education programs. We then describe the selection-oriented education activities at the Rocky Mountain Institute of Software Engineering and relate some problems encountered in establishing these activities. Index Terms-Software engineering education, technology selection, technology transfer.
I. INTRODUCTION
MsILLS [1] indicates that continuing professional education is especially important in software engineering because of the difficulty and undesirability of providing directly relevant experience in academic computer science programs, the fact that many people come to software engineering with formal education in other areas (e.g., electrical engineering), and the rapidity of change in software engineering technology. Continuing professional education for software engineers can take one or more of the following forms. * Task-oriented education develops specific skills. This type of education prepares managerial and technical personnel for specific jobs, projects, and organizations. It moves them closer to the state of practice. *Enhancement-oriented education upgrades a professional's capabilities. It exposes experienced personnel to a working knowledge of new software engineering technology and moves them closer to the state of the art by improving their general capabilities. * Selection-oriented education imparts an appreciation of a technology's applicability and cost-to-benefit ratio. This type of education enables managerial and technical personnel to decide whether or not to adopt a particular technology. It helps move them closer to the state of the art by preparing them to decide in which direction to move. In this paper, we focus on the important, but often neglected, category of selection-oriented education. We begin with a brief discussion of the technology selection task Manuscript received June 7, 1986; revised November 9, 1986. W. E. Riddle is with the Software Productivity Consortium, 1880 Commons Drive, North, Reston, VA 22091. L. G. Williams is with the Rocky Mountain Institute of Software Engineering, P.O. Box 13695, Boulder, CO 80308. IEEE Log Number 8717069.
1199
LLOYD G. WILLIAMS
(Section II). This is followed by a discussion of criteria for selection-oriented education programs. (Section III). We then describe the selection-oriented education activities at rMise, the Rocky Mountain Institute of Software Engineering, Boulder, CO (Section IV), and relate some problems encountered in establishing these activities. II. TECHNOLOGY SELECTION
Once new technology has been sufficiently researched and developed, it is the aim of technology transfer to make that technology part of the state of practice. A general software technology transfer paradigm, which can be adapted to meet the needs of specific situations, is pictured in Fig. 1. The paradigm comprises four activities: selection of technology satisfying some criteria, acquisition of suitable versions of the technology, integration of the technology with existing technology, and propagation of the result throughout a target community. These activities will typically occur sequentially, but may take place in parallel. Technology selection is the process of determining which (new or old) methods, techniques, and tools satisfy criteria reflecting a particular target community's requirements. Selection requires several capabilities: the ability to identify a set of candidates to be considered, the ability to evaluate (either comparatively or in isolation) the candidates, and the ability to choose among the candidates based on the evaluations. More often than not, a single pass will not produce a final choice. One pass may, however, focus the process on a smaller set of viable alternatives during a subsequent pass. Selection characteristics that discriminate among alternative methods, techniques, and tools are essential for technology selection. In support of identification, they assist in establishing a taxonomy of alternatives. In support of evaluation, they assist in stating criteria. In support of choosing among alternatives, they enable interpretation of the evaluation results. In addition, they help elucidate the benefit to be derived from adopting the technology versus the cost, effort, and "pain" of acquiring it and putting it into service. Technology selection is the key to technology improvement and transfer. It is the critical first step in improving the state of practice. It is instrumental to improving the general state of affairs through the identification of gaps to be filled by research and development. And it can aid in identifying the -need for new acquisition, integration, and propagation techniques, and perhaps even suggest the general nature or operational details of these techniques.
0098-5589/87/1100-1199$01.00 © 1987 IEEE
1200
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
TECHNOLOGY IMPROVEMENT EA\
ES
,RESEARCH I.
,, I
DEVELOPMENT
N
\
moo TRANSFER
////////////////////
SELECTION
* * * *
SELECTION, CHARACTERISTICS IDENTIFICATION OF ALTERNATIVES EVALUATION OF ALTERNATIVES CHOICE AMONG ALTERNATIVES
ACQUISITION
* BUY vs BUILD DETERMINATION * CONTRACTING * ACQUISITION MANAGEMENT
INTEGRATION
* * * *
ADAPTATION INSTALLATION RELEASE MANAGEMENT ASSISTANCE DEVELOPMENT
.
PROMOTION and MARKETING
PROPAGATION
Fig. 1. The role of technology selection.
SELECTION-ORIENTED EDUCATION PROGRAMS Technology selection requires a broad knowledge of software engineering technology, an understanding of the general state of practice, and a familiarity with technology on the state-of-the-art horizon. It also requires a number of basic skills: experimental design, instrumentation, data interpretation, and survey design. Finally, it requires insight, intuition, and experience so that definitive decisions can be reached when interpreting what will undoubtedly be inconclusive, if not inconsistent, evaluation III.
CRITERIA
FOR
data. Education alone cannot prepare personnel for the task of technology selection. It is unrealistic to assume that any person or team can be broadly enough educated to successfully handle even a small proportion of the situations that will arise. Formal education and enhancementoriented education can, however, provide the requisite skills, while task-oriented education can impart an understanding of the requirements and constraints of specific target communities. It is, then, the specific intent of selection-oriented education to complement these other contributions.
A. General Criteria Selection-oriented education must provide the ability to identify, evaluate, and choose among alternative methods, techniques, and tools. In addition, it must impart an understanding of the other activities in technology transfer (acquisition, integration, and propagation) so that the concerns imposed by these activities can be addressed. Selection-oriented education is intended to augment and be augmented by other forms of education. It need not, therefore, be "complete." It can, instead, provide a partial understanding sufficient to help the student decide whether or not further, more in-depth education is needed and in what areas. The major task is to provide sufficient
insight into a piece of state-of-the-art technology to allow it to be evaluated in comparison to other state-of-the-art and state-of-practice technologies. Other tasks include: introducing general paradigms for technology selection; making the student aware of a variety of identification, evaluation, and choice techniques; and relating the general community's experiences in performing technology
selection. Because it is a form of continuing education, selectionoriented education must meet several contextual constraints. It must be packaged to fit the time constraints and flexibility requirements of the target audience. It must be modular so that customized sequences can be tailored to meet individual needs. Finally, it must capitalize on the students' former education and (generally extensive) experience.
B. Specific Criteria Within the confines of these general criteria, a selection-oriented education program must meet several specific criteria regarding content, form, pedagogical tools, and instructors. 1) Content: Exposure to State-of-the-Art Technology: A primary intent is to impart an understanding of state-of-the-art technology and the technology lying on the technology horizon. Students should understand the concepts fundamental to assessing this technology with respect to their work situation, and they should appreciate the prospects and time frames for developing usable, transferable versions of the technology. * Technology Transfer. As indicated previously, successful technology selection requires consideration of the feasibility and ease of subsequent acquisition, integration, and propagation. The education program should provide an understanding of the techniques available for these subsequent activities and how they interact with the task of technology selection. * Technology for Identification, Evaluation, and Choice: The program must introduce students to a variety of techniques for identifying, evaluating, and choosing among instances of software technology. It must also help them understand the similarities and differences among alternative identification, evaluation, and choice techniques. Students should understand these techniques in the context of basic approaches to technology selection, in particular, and technology improvement, in general. * Experience-Based Content: The content of the program should reflect actual experience as much as possible. This should include experience with various technology selection paradigms and specific identification, evaluation, and choice techniques. It should also include experience with technology the students might actually be considering for adoption. * Decision-Making Orientation: The majority of students will be decision makers within their organizations. They will be responsible for making technology selection decisions (or developing a plan for making these deci-
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION
sions) as well as managing the technology transition process. The program must meet their needs for general information about possibilities as opposed to the details of specific software technologies or technology selection techniques. It must help them prepare a decision-making plan, decide how to upgrade the skills of their support personnel, and guide the process to successful completion. However, not all students will be in the position of making decisions for larger groups. Some will be interested in selecting technology for their own use. Selectionoriented education must serve those decision-making needs which will generally be more specific to particular pieces of technology. In addition, the program should provide students with the information needed to prepare proposals for securing the required resources. 2) General Form of the Program: Selection-oriented education must be strongly experiential. We have already noted the need to include information on actual technology transfer, selection, and usage experiences. The education must, however, go beyond this to provide students opportunities to gain their own experiences. Because of the necessity to base selection, at least for the foreseeable future, upon insight, intuition, and experience, imparting this experiential knowledge must be a cornerstone of a selection-oriented education program. To this end, the program must include demonstrations and hands-on exercises. These can be simple, "canned" exercises intended to highlight specific points. More extensive exercises that run over several days, involve team work, and have rather general goals should also be provided. And these should be the opportunity for students to gain insight, intuition, and experience through "open laboratories" separate and independent from structured courses. Certainly, there is a large disparity between the experience acquired in a typical educational exercise and that provided by "real-world" software engineering projects. Thus, in developing the experiential component of selection-oriented education, we must carefully develop the demonstrations and exercises and prepare convincing, valid arguments of their implications for the real world. Until we have developed the capability to perform valid, scientific experiments, we must caution students to correctly interpret any exercises they may conduct. In part, this involves monitoring the use of any self-help laboratory we make available and assisting students in interpreting the results of experiments they perform. It will also require the inclusion of courses in experimental techniques. 3) Pedagogical Tools: * Interactive Sessions: Students will rarely come to selection-oriented activities with well-formulated questions. And, in our experience, when they do, the questions are frequently the wrong ones, i.e., "Which software method is the best?" The educational program, therefore, must encourage students to voice their concerns and interests during class time. These can then be refined
1201
interactively with the help of the instructor and the other students. Interaction with other students is particularly important, if only to find out that "the situation" is little different in other organizations. * Laboratory Components: As indicated before, all selection-oriented courses should have some laboratory component. Lack of facilities may reduce this to merely providing transcripts of previous demonstrations, exercises, and experiments. The norm should be to offer guided exercise time during which the students can receive individual help from the instructor. In any event, the development and presentation of effective inclass demonstrations will place severe demands on the facilities available at even the best of organizations. 4) Instructors: Instructors must obviously be technically competent. The must also have an intimate knowledge of both the state of the art and the state of practice. The ideal instructor is a researcher, actively involved in advancing the state ofthe art, who is instrumental in moving the technology he or she has developed into practice within some representative community. Instructors must also have demonstrated above-average capability as teachers. They must be familiar and comfortable with teaching for the short, intense times common in continuing education situations, able to synthesize and explain difficult concepts quickly and articulately, and able to provide insight without having to give detailed explanations. The interactive nature of selection-oriented education imposes additional burdens. Instructors must be willing and able to identify and respond to the needs of individual students. They must encourage questions and demonstrate superior interpersonal skills in handling interruptions. Finally, they must be highly flexible and able to quickly adapt the material to the needs of the students. IV. rMise SELECTION-ORIENTED EDUCATION ACTIVITIES
The Rocky Mountain Institute of Software Engineering (rMise) was founded in 1983 as a nonprofit educational and scientific organization. Its goal is to close the gap between the state of the art and the state of practice in modern software technology.1 The focus of rMise efforts is upon improvements to both the state of practice and the state of the art. The intent is to modernize the state of practice through the transfer of advanced technology into widespread use and enrich the state of the art under the influence of information con'The objectives of rMise are to:
provide the practitioner community with the ability to identify, evaluate, and choose among alternative methodologies, tools, and environments, * provide the developers of modern methodologies, tools, and environments with the opportunity to consider feedback reflecting the needs, expectations, and reactions of the professional community, and * complement and support the efforts of other organizations in the transfer of advanced software technology into practice and the transfer of feedback to the developer community. *
1202
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
cerning practitioner-community needs, expectations, and reactions to existing technologies. The rMise strategy for closing the gap between art and practice emphasizes the comparative evaluation of alternative software methods and their supporting software tools and automated software environments. Methods, tools, and environments are emphasized because they provide unifying frameworks for the activities of defining, selecting, and integrating individual technology fragments. These activities are critical to both the transfer of technology and the feedback of information from practitioners to researchers. Comparative evaluation is emphasized because, appropriately done, it simultaneously supports both a practitioner's selection of new technology and a developer's creation and evolution of technology. rMise sponsors many different types of activities, including expositions, workshops, study groups, tutorials, and research projects. The majority of activities have been educational ones oriented towards practitioners, managers, and administrators. These have been supported by technology improvement activities to help the research and development community understand the state-of-theart/state-of-practice gap and develop selection-oriented technology to narrow it. A. Examples of rMise Activities 1) Professional Education Programs: rMise has sponsored an extensive set of professional education activities since 1984. These activities have been attended by hundreds of participants from the United States and overseas. Tutorials have been the principal activity during these Professional Education Programs, with each program consisting of about 20 tutorials. A list of the tutorials in the most recent Professional Education Program appears in Fig. 2. Individual tutorials have treated state-of-the-art and state-of-practice technology in a variety of areas, among them: software methods, project management, software tools, software engineering environments, software analysis, and modem programming languages. The tutorials have been organized into sequences, with earlier tutorials within a sequence treating fundamental concepts and the latter ones treating advanced topics. This organization has allowed participants to choose a subset meeting their needs and experience levels. Many participants have taken advantage of this; the average length of attendance at the Institute's Professional Education Programs has been three
days.
The programs have also included tutorials that directly address technology transfer and selection. A Technology Transition tutorial was part of the program in 1984. The software methods tutorial sequences have also included tutorials specifically designed to compare alternative methods. And several of the tutorials on software engineering environments have covered general paradigms for moving technology into practice. 2) Software Environment Design Tutorial: Many of the
tutorials in the summer programs have provided the opportunity for first-hand experience through exercises and demonstrations. Some have had this as their central focus. An example is the Software Environment Design tutorial which has been held each summer. In this tutorial, participants work in groups to complete an extended, multiday design exercise. Initially, three scenarios are presented. Each group prepares a requirements definition for a software environment meeting the needs of software personnel under one of the scenarios. Each group then presents their requirements definition to "management" with the other groups serving as management consultants. Depending on their needs and experiences, the groups then focus on either project management issues, preparing a project plan, or technical issues, preparing an architectural design. In either case, they must address how existing state-of-the-art and state-of-practice technology can be applied to their plans or designs. These plans and architectures are also presented to "management." Throughout the tutorial, the instructors play the dual roles of "senior management," visiting the groups as they work and inquiring about what is happening, and "consultants," helping the groups develop feasible plans and architectures that make use of moderm technology. 3) Executive Summary of Software Engineering Issues: In 1985, the scope of rMise activities was broadened to include a multiday Executive Summary of Software Engineering Issues designed specifically for toplevel administrators. The intent of the Executive Summary is to provide senior managers with an intense introduction to a broad spectrum of software engineering topics in a way that helps them evaluate the potential for their organizations. The Executive Summary typically covers a wide variety of topics. Topics included in the 1986 Executive Summary were: productivity improvement, expert systems, rights and data issues, human factors, software technology in Japan, large-scale systems, measurement technology, software methods, tools and environments, and test/ evaluation technology. The emphasis was on basic concepts, potential benefit, and experiences (both good and bad). A large portion of the time was used for open discussion and interaction. The instructors were themselves top-level administrators, allowing them to interact with the participants on a peer basis. 4) Software Methods Workshop: In conjunction with the 1985 Professional Education Program, there was a Software Methods Workshop, cosponsored by the Department of Defense's STARS Program. It was attended by over 100 professionals, primarily from industry and Government. In the workshop's initial segment, several technology developers, instrumental in founding specific methods, gave presentations on the philosophies underlying their approaches to the software process. An open discussion among the developers and the workshop participants ended this initial segment. In the middle segment of the workshop, full-day tutorials on six different methods were provided and each participant attended three. In
1203
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION -0
SOFTWARE METHODOLOGY * Methodology Review
* * * * * * * *
Jackson System Development Method An Integrated Data/Control Flow Specification Technique Real-time Structned Analysis and Design Data Strucud System Development Higher Order Software Methodology Software Cost Reduction Methodology PAMELA: A Process Abstraction Medtodology An Intoduction to PAISLEY
G. Booch D. Higgins W. Riddle P. Ward P. Zave J. Caneron M. Alford P. Ward D. Higgins J. Urban P. Clements G. Cherry P. Zave
Rational, Inc. Ken Orr & Associates, Inc. software design & analysis, inc. Yourdon, Inc. AT&T Bel Laboratories Michael Jackson Systems, Ltd. TRW, Inc. Yourdon, Inc. Ken Orr & Associates, Inc. University of Southwestern Louisiana Naval Research LaboratOry Thought**Tools AT&T Bell Laboratories
B. Boehm R. DeMillo L. Druffel
C. Lewis K. Kishida F. McGarry K. Orr T. Probert A. Zelweger
TRW, Inc. Georgia Institute of Technology Rational Inc. Shea & Gardner University of Colorado SRA, Inc. NASA Goddard Space Flight Center Ken Otr & Associates, Inc. Software Synergy Group Federal Aviation Administration
R. Fairley W. Riddle L. Williams A. Pyster D. Weiss
Wang Institute software design & analysis, inc. University of Colorado Digital Sound Corporation Naval Research Laboratory
W. Riddle C. Smith F. DeRemer L. Osterweil L. Clarke C. McClure L. Williams J. Wileden W. Riddle L. Williams
software design & analysis, inc. Duke University MetaWarem kapra University of Colorado University of Massachusetts C. L. McClure & Associates, Inc. University of Colorado University of Massachusetts software design & analysis, inc. University of Colorado
L. Williams W. Cumingham T. Love G. Booch
University of Colorado Tektronix Inc. Productivity Products Intemational Inc. Rational Inc.
SOFTWARE ENGINEERING ISSUES
* Executive Summary of Software Engineering Issues
M. Greberger
MANAGEMENT TECHNOLOGY * Project Management for Software Engineers * Comparative Software Methodology
* Improving Software Productivity * Measurement for Management
TOOLS and ENVIRONMENTS * Developer's Guide to Automated Software Environments * Toois and Techniques for Designing Responsive Software * Source Text Analysis Toois * * * * * *
Experiences with the Toolpack Environment Tools for Software Testing Software Maintenance Tools Distributed Software Envirownents Tools for Developers of Distributed Software Software Environments Design Workshop
OBJECT-ORIENTED TECHNOLOGY * * * *
The Object Model in Software Engineering Programming Applications in Smalltalk-80Th Object-oriented Design with Objective-CTm Object-oriented Design with Ada"m
Fig. 2. rMise 1986 Professional Education Program.
the final segment, each participant picked one of the tZ e methods they had studied and used it, with the instructor's and other participants' help, to solve at least one of three specific problems. This final segment ended with a plenary session in which representative solutions in each method were presented and a group discussion addressed the differences and similarities among the methods. 5) Technology Improvement Program: The principal technology improvement activity has been a Software Environments Workshop cosponsored by the National Science Foundation [2]. The goal was to determine how to improve the state of the art in software environments over the next decade. It provided an indirect contribution to selection-oriented education by developing estimates of
both the current and future state of the art in the software environments arena. Participants formed groups to determine fundamental issues and required activities in several areas: tools, distributed systems, database systems, extensibility, artificial intelligence, integration, prototyping, measurement and evaluation, monolingual versus multilingual environments, human interfaces, and evolutionary versus revolutionary approaches to improvement. Each group assessed the current state of affairs, defined a desirable future state, and considered how to best move from the current to the future state. They also addressed several global topics. First, to provide a context for the issues and activities, they developed
1204
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
a characterization of the software environment situation in the mid-1990's in comparison to the situation today. Second, they considered the general nature of organizations that could successfully undertake the required activities. Third, they reconsidered the results of a similarly intentioned workshop, held five years ago [3], and updated the previous workshop's assessment to reflect the community's emerging concern for general systems issues and the software creation and evolution process.
B. The rMise Approach to Selection-Oriented Education * State-of-the-Art Technology: Attention has been given primarily to topics where the state of the art is well developed and the gap between the state of the art and the state of practice is larger than average. Several studies (see, for example, [4]) have indicated that the state of practice is far behind the state of the art in software methods and software tools/environments. rMise, therefore, has emphasized these topics. It has also treated subjects, such as project management, that directly relate to these topics and affect the extent to which these technologies are used in practice. * Coverage of Technology Transfer: Few rMise activities have focused exclusively on the processes involved in technology transfer. Rather, the coverage has been indirect. The Software Environments Design tutorial and several of the other Tools and Environments tutorials have, for example, treated general paradigms for the transfer of tool/environment technology. It is expected that this mainly indirect approch will continue for the foreseeable future until the understanding of, and experience with, software technology transfer mature significantly beyond their present levels. * Identification, Evaluation, and Choice Techniques: In rMise activities, these subjects are treated within the context of specific technology arenas. This approach emphasizes exposure to the state of the art, with specific technology selection techniques being discussed in the context of specific technologies. Selection-related material that has been included in the tutorials includes: cost estimation techniques, general paradigms for integrating technology, technology classification schemes, measurement technology, comparative evaluations of available technologies, cost/benefit analyses, and techniques for effective participation in team efforts. The treatment of selection techniques has included comparative evaluation exercises. Simple instances have occurred in the Software Environment Design tutorial, in which participants chose technologies to include in their environments. A more extensive comparison was done in the course of the Software Methods Workshop. This workshop's format provided insight into a variety of modern technology options to a level of detail which allowed (informal) comparative assessment. This format would seem to be useful for selection-oriented, experience-based studies of a wide spectrum of software technology. As in the case of technology transfer, rMise has taken
an indirect approach to technology selection. For the foreseeable future, direct treatment will probably be limited to the technology improvement activities, including workshops and study groups on fundamental issues and concepts. The state of the art and the state of practice in technology selection may eventually mature to the point
that direct treatment will be possible. But because selection techniques will always be best understood in the context of their application to specific situations, it would seem that an emphasis on indirect treatment will always be most appropriate. * Experience-Based Content: Reports on experiences in adopting and using software technology have been part of the vast majority of tutorials. The rMise approach has been to seek instructors who, themselves, have extensive experience. This approach was also used in choosing participants for the Software Environment Workshop; people actually involved in the design and construction of environments accounted for about half the participants. * Decision-Making Orientation: The rMise approach has been to orient its activities specifically toward technology selection decision makers. The Executive Summary of Software Engineering Issues tutorial is specifically designed for top-level decision makers. Most of the activities in the Professional Education Programs have been oriented toward project managers and middle-level management or people aspiring to these positions. In addition, decision makers were specifically included in the Software Environments Workshop, and the workshop's results were prepared for use by those responsible for deciding where to invest time and effort in the improvement of the state of the art for software environments. Other continuing education programs (see, for example, the plans discussed in [5]) have also included educational activities specifically oriented towards decision makers. These programs have usually adopted a tiered approach with separate "tracks" designed for executives, managers, and software engineers. rMise has emphasized meeting the needs of the manager and then broadened the scope of its programs to meet the needs of other populations using the resources and materials developed for manager education as a base for its other activities. * Exercises: Exercises have been a key component of many rMise activities. Tutorials offering extended exercises have been included in both the software methods and tools/environments tutorial sequences. A concentrated effort has been made to assure that these exercises do not suffer from the problems noted previously. In addition, several other tutorials have included simple, experienceproviding exercises. Unfortunately, it has not been possible to provide facilities for "live" demonstrations or to provide a laboratory capability. This has been corrected, somewhat, in the Software Methods Exposition, first held as part of the 1986 Professional Education Program. Manual and automated tools- supporting a variety of methods were available for use during the exposition and participants had the oppor-
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION
tunity to gain hands-on experience with these technologies. It is anticipated that rMise will expand this sort of activity, and plans are being made to provide a full-capability laboratory supporting software technology demonstration and experimentation. * Interactive Sessions: The rMise approach to fostering interaction is to keep the number of participants small. Participation in the tutorials is restricted, on a first-come first-served basis to assure highly interactive learning opportunities. Study groups are held to a maximum of 25 participants to foster interaction. Even when the number of participants is small, opportunities are sought to divide the group into working groups of 3-5 people. When the number of participants is large, as in the case of the Software Methods Workshop, the amount of time that the whole group spends together is held to a minimum, and much of this is used for open discussion. * Instructors: rMise has been fortunate to attract instructors for its tutorials (and participants for its study groups) who are well versed in their fields, familiar with both the state of the art and the state of practice, able to handle the rigors of short, intense sessions, and willing and able to participate in free-wheeling discussions. The approach has been to find top-quality people who are heavily involved in defining and expanding the boundaries of their field of interest; no other approach would satisfy the instructor criteria specified previously. V. SUMMARY
Selection-oriented education provides an exposure to new techniques and concepts rather than an in-depth coverage of those techniques and concepts. The coverage is sufficient to impart an appreciation of the technology's applicability and cost-to-benefit ratio. The intent is to enable management and technical personnel to make wellfounded decisions about whether or not to adopt a particular technology. A selection-oriented education program must satisfy many criteria. It must provide an extensive introduction to the state of the art. It must cover all of the activities supporting technology transfer and show how they interact with the specific activity of technology selection. It must cover a wide variety of techniques for performing technology selection and demonstrate their use. It must include information on actual experiences. It must serve the needs of those responsible for selecting technology; often this is an individual who is trying to decide whether or not to upgrade his or her own capabilities. It must provide for a high degree of interaction among instructors and students so that the needs of individual students are articulated and accommodated, the applicability of the material to personal situations is clear, and students are able to learn from each others' experiences. And it must
be conducted by knowledgeable, experienced, articulate, flexible instructors who are accustomed to short, intense, interactive sessions. The Rocky'Mountain Ins'titute of Software Engineering
1205
(rMise) has sponsored a number of activities that contribute to selection-oriented education. The primary activity in this arena has been its Professional Education Program. This program has included activities designed to meet the educational needs of executives, managers, and software engineers with an emphasis on the first two of these populations. rMise also sponsored a workshop that emphasized the comparative assessment of alternative methods through large-group discussion of experiences gained in small-group exercises. And the Institute organized and ran a study group to identify issues fundamental to improving the state of the art for software environments. Finally, a software methods exposition was held to provide more individualized learning through direct experience. The rMise approach to selection-oriented education emphasizes exposure to the state of the art in software methods and software tools/environments, two areas where the gap between art and practice is particularly wide. Techniques supporting technology selection are treated indirectly, in the context of their use rather than as subjects of direct focus. Experiential data are included as much as possible through the use of personnel who are actively involved in both developing the technology within their fields and transferring it into practice. The needs of decision makers at the project management, middle management, and top-executive levels are met by designing specific tutorials to meet their needs. Interactive sessions are assured by limiting participation so that groups are small, dividing participants into working groups of 3-5 people whenever possible. The majority of large-group time is devoted to open discussion. Quality instruction is assured by employing experienced instructors who are instrumental in defining and expanding the boundaries of their respective fields. While these activities do not satisfy all of the criteria for selection-oriented education, they do address the major points. We expect that continued experience in offering selection-oriented education, coupled with expanded capabilities (e.g., a hands-on laboratory) will help move the rMise program closer to the ideal. The rMise enterprise has provided a wealth of experience in setting up and running selection-oriented education programs. The body of this paper has reported our positive experiences. In closing, we would like to offer a few observations about what can go "wrong." First, we have found little interest in activities that directly address technology transfer and technology selection per se. There is interest in these topics within the context of specific situations, e.g., the identification and adoption of specific configuration management tools. There is, however, essentially no interest in these topics as "standalone" subjects. Second, we have found little interest in revolutionary technologies which, while they may be "hot" topics in the research and development community, are sufficiently different from existing technology that their introduction would cause large perturbations. The interest is in much more conservative, advanced technologies for
1206
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
0 which there is little question of applicability and ease of William E. Riddle (M'83-SM'84) is Chief Technical Officer and Executive Vice President at the X introduction. Third, the majority of students will be seekSoftware Productivity Consortium in Reston, VA. ing "the answer" and feel that there must be something He is responsible for establishing and guiding a that will solve all, or most, of their problems if only they software productivity improvement program oriented toward the needs of the Consortium's sponcan find it. One of the major opportunities, and one of the soring, aerospace industry companies. Prior to hardest tasks, is to get these students to realize that they joining the Consortium, he was an independent have the wrong objective. Finally, some students (luckconsultant, founder, and director of the Rocky Mountain Institute of Software Engineering, a diily, a minority) will lack the flexibility and tolerance for of software design and engineering work at ambiguity that are necessary when considering technol- Cray Laboratories, and arector professor of computer science at the Universities ogy selection. They will be sophisticated enough to know of Michigan and Colorado. His technical work emphasizes techniques for description and analysis of software designs and the delivery of this that there is no single answer, but they will want struc- the technology to practitioners through automated software environments. tured courses that deliver logical assessments of technolDr. Riddle is a former ACM National Lecturer and former Chair of the ACM Special Interest Group on Software Engineering. He is frequently ogy for a wide spectrum of situations.
ACKNOWLEDGMENT The success of rMise is due, in iarge measure, to the instructors who have participated in the Professional Education Programs over the past three years. Without their generous and capable assistance, this program would not have been possible. REFERENCES [11 H. D. Mills, "Software enginering education," Proc. IEEE, vol. 26, pp. 1158-1162, Sept. 1980. [2] W. E. Riddle and L. G. Williams, "Software environments workshop report," ACM Sigsoft Software Eng. Notes, vol. 11, pp. 73-102, Jan. 1986.
[3] L. J. Osterweil, "Software environment research directions for the next five years," IEEE Computer, vol. 14, pp. 35-43, Apr. 1981. [4] M. V. Zelkowitz, R. Yeh, R. G. Hamlet, J. D. Gannon, and V. R. Basilli, "Software engineering practices in the US and Japan," IEEE Computer, vol. 17, pp. 57-66, June 1984. 15] M. Ulema and J. Larsen, "Planning for in-house software engineering education," in Proc. 15th SIGCSE Tech. Symp. Comput. Sci. Educ., Philadelphia, PA, Feb. 1984, pp. 130-136.
involved in organizing workshops and conferences and chaired the 9th International Conference on Software Engineering. g3
kg *
;
Lloyd G. Williams received the A.B. degree from Colgate University, Hamilton, NY, in 1969 and the Ph.D. degree in physical chemistry from the
University of Wisconsin in 1976. He is an Associate Professor in the Department of Electrical Engineering and Computer Science at the University of Colorado, Denver. Prior to joining the faculty at the University of Colorado, he was at Hampshire College in Amherst, MA, where he taught computer science and served as founding chairman of the college's multidisciplinary Computer Studies Program. His principal research interests are in software engineering, particularly software development methods, software tools and environments, and distributed systems. He has served as a consultant on software development for a variety of industrial organizations. He is also Director of the Rocky Mountain Institute of Software Engineering, a nonprofit organization founded to assist in the transition of modem software engineering technology into widespread practice. Prior to becoming Director of the Rocky Mountain Institute, he served as Coordinator for Technical Activities. Dr. Williams is a member of the Association for Computing Machinery and the IEEE Computer Society.
Technology Selection: WILLIAM E. RIDDLE,
An
Educational Approach
SENIOR MEMBER, IEEE, AND
Abstract-Creating and enhancing a software engineering work force requires several different types of continuing education for software professionals, including: task-oriented education, enhancement-oriented education and selection-oriented education.In this paper, we focus on the important, but often neglected, category of selection-oriented education. We begin with a discussion of technology selection, indicating what it involves, how it contributes to improving the state of practice, and why it is key to technology improvement in general. This is followed by a discussion of some criteria for selection-oriented education programs. We then describe the selection-oriented education activities at the Rocky Mountain Institute of Software Engineering and relate some problems encountered in establishing these activities. Index Terms-Software engineering education, technology selection, technology transfer.
I. INTRODUCTION
MsILLS [1] indicates that continuing professional education is especially important in software engineering because of the difficulty and undesirability of providing directly relevant experience in academic computer science programs, the fact that many people come to software engineering with formal education in other areas (e.g., electrical engineering), and the rapidity of change in software engineering technology. Continuing professional education for software engineers can take one or more of the following forms. * Task-oriented education develops specific skills. This type of education prepares managerial and technical personnel for specific jobs, projects, and organizations. It moves them closer to the state of practice. *Enhancement-oriented education upgrades a professional's capabilities. It exposes experienced personnel to a working knowledge of new software engineering technology and moves them closer to the state of the art by improving their general capabilities. * Selection-oriented education imparts an appreciation of a technology's applicability and cost-to-benefit ratio. This type of education enables managerial and technical personnel to decide whether or not to adopt a particular technology. It helps move them closer to the state of the art by preparing them to decide in which direction to move. In this paper, we focus on the important, but often neglected, category of selection-oriented education. We begin with a brief discussion of the technology selection task Manuscript received June 7, 1986; revised November 9, 1986. W. E. Riddle is with the Software Productivity Consortium, 1880 Commons Drive, North, Reston, VA 22091. L. G. Williams is with the Rocky Mountain Institute of Software Engineering, P.O. Box 13695, Boulder, CO 80308. IEEE Log Number 8717069.
1199
LLOYD G. WILLIAMS
(Section II). This is followed by a discussion of criteria for selection-oriented education programs. (Section III). We then describe the selection-oriented education activities at rMise, the Rocky Mountain Institute of Software Engineering, Boulder, CO (Section IV), and relate some problems encountered in establishing these activities. II. TECHNOLOGY SELECTION
Once new technology has been sufficiently researched and developed, it is the aim of technology transfer to make that technology part of the state of practice. A general software technology transfer paradigm, which can be adapted to meet the needs of specific situations, is pictured in Fig. 1. The paradigm comprises four activities: selection of technology satisfying some criteria, acquisition of suitable versions of the technology, integration of the technology with existing technology, and propagation of the result throughout a target community. These activities will typically occur sequentially, but may take place in parallel. Technology selection is the process of determining which (new or old) methods, techniques, and tools satisfy criteria reflecting a particular target community's requirements. Selection requires several capabilities: the ability to identify a set of candidates to be considered, the ability to evaluate (either comparatively or in isolation) the candidates, and the ability to choose among the candidates based on the evaluations. More often than not, a single pass will not produce a final choice. One pass may, however, focus the process on a smaller set of viable alternatives during a subsequent pass. Selection characteristics that discriminate among alternative methods, techniques, and tools are essential for technology selection. In support of identification, they assist in establishing a taxonomy of alternatives. In support of evaluation, they assist in stating criteria. In support of choosing among alternatives, they enable interpretation of the evaluation results. In addition, they help elucidate the benefit to be derived from adopting the technology versus the cost, effort, and "pain" of acquiring it and putting it into service. Technology selection is the key to technology improvement and transfer. It is the critical first step in improving the state of practice. It is instrumental to improving the general state of affairs through the identification of gaps to be filled by research and development. And it can aid in identifying the -need for new acquisition, integration, and propagation techniques, and perhaps even suggest the general nature or operational details of these techniques.
0098-5589/87/1100-1199$01.00 © 1987 IEEE
1200
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
TECHNOLOGY IMPROVEMENT EA\
ES
,RESEARCH I.
,, I
DEVELOPMENT
N
\
moo TRANSFER
////////////////////
SELECTION
* * * *
SELECTION, CHARACTERISTICS IDENTIFICATION OF ALTERNATIVES EVALUATION OF ALTERNATIVES CHOICE AMONG ALTERNATIVES
ACQUISITION
* BUY vs BUILD DETERMINATION * CONTRACTING * ACQUISITION MANAGEMENT
INTEGRATION
* * * *
ADAPTATION INSTALLATION RELEASE MANAGEMENT ASSISTANCE DEVELOPMENT
.
PROMOTION and MARKETING
PROPAGATION
Fig. 1. The role of technology selection.
SELECTION-ORIENTED EDUCATION PROGRAMS Technology selection requires a broad knowledge of software engineering technology, an understanding of the general state of practice, and a familiarity with technology on the state-of-the-art horizon. It also requires a number of basic skills: experimental design, instrumentation, data interpretation, and survey design. Finally, it requires insight, intuition, and experience so that definitive decisions can be reached when interpreting what will undoubtedly be inconclusive, if not inconsistent, evaluation III.
CRITERIA
FOR
data. Education alone cannot prepare personnel for the task of technology selection. It is unrealistic to assume that any person or team can be broadly enough educated to successfully handle even a small proportion of the situations that will arise. Formal education and enhancementoriented education can, however, provide the requisite skills, while task-oriented education can impart an understanding of the requirements and constraints of specific target communities. It is, then, the specific intent of selection-oriented education to complement these other contributions.
A. General Criteria Selection-oriented education must provide the ability to identify, evaluate, and choose among alternative methods, techniques, and tools. In addition, it must impart an understanding of the other activities in technology transfer (acquisition, integration, and propagation) so that the concerns imposed by these activities can be addressed. Selection-oriented education is intended to augment and be augmented by other forms of education. It need not, therefore, be "complete." It can, instead, provide a partial understanding sufficient to help the student decide whether or not further, more in-depth education is needed and in what areas. The major task is to provide sufficient
insight into a piece of state-of-the-art technology to allow it to be evaluated in comparison to other state-of-the-art and state-of-practice technologies. Other tasks include: introducing general paradigms for technology selection; making the student aware of a variety of identification, evaluation, and choice techniques; and relating the general community's experiences in performing technology
selection. Because it is a form of continuing education, selectionoriented education must meet several contextual constraints. It must be packaged to fit the time constraints and flexibility requirements of the target audience. It must be modular so that customized sequences can be tailored to meet individual needs. Finally, it must capitalize on the students' former education and (generally extensive) experience.
B. Specific Criteria Within the confines of these general criteria, a selection-oriented education program must meet several specific criteria regarding content, form, pedagogical tools, and instructors. 1) Content: Exposure to State-of-the-Art Technology: A primary intent is to impart an understanding of state-of-the-art technology and the technology lying on the technology horizon. Students should understand the concepts fundamental to assessing this technology with respect to their work situation, and they should appreciate the prospects and time frames for developing usable, transferable versions of the technology. * Technology Transfer. As indicated previously, successful technology selection requires consideration of the feasibility and ease of subsequent acquisition, integration, and propagation. The education program should provide an understanding of the techniques available for these subsequent activities and how they interact with the task of technology selection. * Technology for Identification, Evaluation, and Choice: The program must introduce students to a variety of techniques for identifying, evaluating, and choosing among instances of software technology. It must also help them understand the similarities and differences among alternative identification, evaluation, and choice techniques. Students should understand these techniques in the context of basic approaches to technology selection, in particular, and technology improvement, in general. * Experience-Based Content: The content of the program should reflect actual experience as much as possible. This should include experience with various technology selection paradigms and specific identification, evaluation, and choice techniques. It should also include experience with technology the students might actually be considering for adoption. * Decision-Making Orientation: The majority of students will be decision makers within their organizations. They will be responsible for making technology selection decisions (or developing a plan for making these deci-
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION
sions) as well as managing the technology transition process. The program must meet their needs for general information about possibilities as opposed to the details of specific software technologies or technology selection techniques. It must help them prepare a decision-making plan, decide how to upgrade the skills of their support personnel, and guide the process to successful completion. However, not all students will be in the position of making decisions for larger groups. Some will be interested in selecting technology for their own use. Selectionoriented education must serve those decision-making needs which will generally be more specific to particular pieces of technology. In addition, the program should provide students with the information needed to prepare proposals for securing the required resources. 2) General Form of the Program: Selection-oriented education must be strongly experiential. We have already noted the need to include information on actual technology transfer, selection, and usage experiences. The education must, however, go beyond this to provide students opportunities to gain their own experiences. Because of the necessity to base selection, at least for the foreseeable future, upon insight, intuition, and experience, imparting this experiential knowledge must be a cornerstone of a selection-oriented education program. To this end, the program must include demonstrations and hands-on exercises. These can be simple, "canned" exercises intended to highlight specific points. More extensive exercises that run over several days, involve team work, and have rather general goals should also be provided. And these should be the opportunity for students to gain insight, intuition, and experience through "open laboratories" separate and independent from structured courses. Certainly, there is a large disparity between the experience acquired in a typical educational exercise and that provided by "real-world" software engineering projects. Thus, in developing the experiential component of selection-oriented education, we must carefully develop the demonstrations and exercises and prepare convincing, valid arguments of their implications for the real world. Until we have developed the capability to perform valid, scientific experiments, we must caution students to correctly interpret any exercises they may conduct. In part, this involves monitoring the use of any self-help laboratory we make available and assisting students in interpreting the results of experiments they perform. It will also require the inclusion of courses in experimental techniques. 3) Pedagogical Tools: * Interactive Sessions: Students will rarely come to selection-oriented activities with well-formulated questions. And, in our experience, when they do, the questions are frequently the wrong ones, i.e., "Which software method is the best?" The educational program, therefore, must encourage students to voice their concerns and interests during class time. These can then be refined
1201
interactively with the help of the instructor and the other students. Interaction with other students is particularly important, if only to find out that "the situation" is little different in other organizations. * Laboratory Components: As indicated before, all selection-oriented courses should have some laboratory component. Lack of facilities may reduce this to merely providing transcripts of previous demonstrations, exercises, and experiments. The norm should be to offer guided exercise time during which the students can receive individual help from the instructor. In any event, the development and presentation of effective inclass demonstrations will place severe demands on the facilities available at even the best of organizations. 4) Instructors: Instructors must obviously be technically competent. The must also have an intimate knowledge of both the state of the art and the state of practice. The ideal instructor is a researcher, actively involved in advancing the state ofthe art, who is instrumental in moving the technology he or she has developed into practice within some representative community. Instructors must also have demonstrated above-average capability as teachers. They must be familiar and comfortable with teaching for the short, intense times common in continuing education situations, able to synthesize and explain difficult concepts quickly and articulately, and able to provide insight without having to give detailed explanations. The interactive nature of selection-oriented education imposes additional burdens. Instructors must be willing and able to identify and respond to the needs of individual students. They must encourage questions and demonstrate superior interpersonal skills in handling interruptions. Finally, they must be highly flexible and able to quickly adapt the material to the needs of the students. IV. rMise SELECTION-ORIENTED EDUCATION ACTIVITIES
The Rocky Mountain Institute of Software Engineering (rMise) was founded in 1983 as a nonprofit educational and scientific organization. Its goal is to close the gap between the state of the art and the state of practice in modern software technology.1 The focus of rMise efforts is upon improvements to both the state of practice and the state of the art. The intent is to modernize the state of practice through the transfer of advanced technology into widespread use and enrich the state of the art under the influence of information con'The objectives of rMise are to:
provide the practitioner community with the ability to identify, evaluate, and choose among alternative methodologies, tools, and environments, * provide the developers of modern methodologies, tools, and environments with the opportunity to consider feedback reflecting the needs, expectations, and reactions of the professional community, and * complement and support the efforts of other organizations in the transfer of advanced software technology into practice and the transfer of feedback to the developer community. *
1202
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
cerning practitioner-community needs, expectations, and reactions to existing technologies. The rMise strategy for closing the gap between art and practice emphasizes the comparative evaluation of alternative software methods and their supporting software tools and automated software environments. Methods, tools, and environments are emphasized because they provide unifying frameworks for the activities of defining, selecting, and integrating individual technology fragments. These activities are critical to both the transfer of technology and the feedback of information from practitioners to researchers. Comparative evaluation is emphasized because, appropriately done, it simultaneously supports both a practitioner's selection of new technology and a developer's creation and evolution of technology. rMise sponsors many different types of activities, including expositions, workshops, study groups, tutorials, and research projects. The majority of activities have been educational ones oriented towards practitioners, managers, and administrators. These have been supported by technology improvement activities to help the research and development community understand the state-of-theart/state-of-practice gap and develop selection-oriented technology to narrow it. A. Examples of rMise Activities 1) Professional Education Programs: rMise has sponsored an extensive set of professional education activities since 1984. These activities have been attended by hundreds of participants from the United States and overseas. Tutorials have been the principal activity during these Professional Education Programs, with each program consisting of about 20 tutorials. A list of the tutorials in the most recent Professional Education Program appears in Fig. 2. Individual tutorials have treated state-of-the-art and state-of-practice technology in a variety of areas, among them: software methods, project management, software tools, software engineering environments, software analysis, and modem programming languages. The tutorials have been organized into sequences, with earlier tutorials within a sequence treating fundamental concepts and the latter ones treating advanced topics. This organization has allowed participants to choose a subset meeting their needs and experience levels. Many participants have taken advantage of this; the average length of attendance at the Institute's Professional Education Programs has been three
days.
The programs have also included tutorials that directly address technology transfer and selection. A Technology Transition tutorial was part of the program in 1984. The software methods tutorial sequences have also included tutorials specifically designed to compare alternative methods. And several of the tutorials on software engineering environments have covered general paradigms for moving technology into practice. 2) Software Environment Design Tutorial: Many of the
tutorials in the summer programs have provided the opportunity for first-hand experience through exercises and demonstrations. Some have had this as their central focus. An example is the Software Environment Design tutorial which has been held each summer. In this tutorial, participants work in groups to complete an extended, multiday design exercise. Initially, three scenarios are presented. Each group prepares a requirements definition for a software environment meeting the needs of software personnel under one of the scenarios. Each group then presents their requirements definition to "management" with the other groups serving as management consultants. Depending on their needs and experiences, the groups then focus on either project management issues, preparing a project plan, or technical issues, preparing an architectural design. In either case, they must address how existing state-of-the-art and state-of-practice technology can be applied to their plans or designs. These plans and architectures are also presented to "management." Throughout the tutorial, the instructors play the dual roles of "senior management," visiting the groups as they work and inquiring about what is happening, and "consultants," helping the groups develop feasible plans and architectures that make use of moderm technology. 3) Executive Summary of Software Engineering Issues: In 1985, the scope of rMise activities was broadened to include a multiday Executive Summary of Software Engineering Issues designed specifically for toplevel administrators. The intent of the Executive Summary is to provide senior managers with an intense introduction to a broad spectrum of software engineering topics in a way that helps them evaluate the potential for their organizations. The Executive Summary typically covers a wide variety of topics. Topics included in the 1986 Executive Summary were: productivity improvement, expert systems, rights and data issues, human factors, software technology in Japan, large-scale systems, measurement technology, software methods, tools and environments, and test/ evaluation technology. The emphasis was on basic concepts, potential benefit, and experiences (both good and bad). A large portion of the time was used for open discussion and interaction. The instructors were themselves top-level administrators, allowing them to interact with the participants on a peer basis. 4) Software Methods Workshop: In conjunction with the 1985 Professional Education Program, there was a Software Methods Workshop, cosponsored by the Department of Defense's STARS Program. It was attended by over 100 professionals, primarily from industry and Government. In the workshop's initial segment, several technology developers, instrumental in founding specific methods, gave presentations on the philosophies underlying their approaches to the software process. An open discussion among the developers and the workshop participants ended this initial segment. In the middle segment of the workshop, full-day tutorials on six different methods were provided and each participant attended three. In
1203
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION -0
SOFTWARE METHODOLOGY * Methodology Review
* * * * * * * *
Jackson System Development Method An Integrated Data/Control Flow Specification Technique Real-time Structned Analysis and Design Data Strucud System Development Higher Order Software Methodology Software Cost Reduction Methodology PAMELA: A Process Abstraction Medtodology An Intoduction to PAISLEY
G. Booch D. Higgins W. Riddle P. Ward P. Zave J. Caneron M. Alford P. Ward D. Higgins J. Urban P. Clements G. Cherry P. Zave
Rational, Inc. Ken Orr & Associates, Inc. software design & analysis, inc. Yourdon, Inc. AT&T Bel Laboratories Michael Jackson Systems, Ltd. TRW, Inc. Yourdon, Inc. Ken Orr & Associates, Inc. University of Southwestern Louisiana Naval Research LaboratOry Thought**Tools AT&T Bell Laboratories
B. Boehm R. DeMillo L. Druffel
C. Lewis K. Kishida F. McGarry K. Orr T. Probert A. Zelweger
TRW, Inc. Georgia Institute of Technology Rational Inc. Shea & Gardner University of Colorado SRA, Inc. NASA Goddard Space Flight Center Ken Otr & Associates, Inc. Software Synergy Group Federal Aviation Administration
R. Fairley W. Riddle L. Williams A. Pyster D. Weiss
Wang Institute software design & analysis, inc. University of Colorado Digital Sound Corporation Naval Research Laboratory
W. Riddle C. Smith F. DeRemer L. Osterweil L. Clarke C. McClure L. Williams J. Wileden W. Riddle L. Williams
software design & analysis, inc. Duke University MetaWarem kapra University of Colorado University of Massachusetts C. L. McClure & Associates, Inc. University of Colorado University of Massachusetts software design & analysis, inc. University of Colorado
L. Williams W. Cumingham T. Love G. Booch
University of Colorado Tektronix Inc. Productivity Products Intemational Inc. Rational Inc.
SOFTWARE ENGINEERING ISSUES
* Executive Summary of Software Engineering Issues
M. Greberger
MANAGEMENT TECHNOLOGY * Project Management for Software Engineers * Comparative Software Methodology
* Improving Software Productivity * Measurement for Management
TOOLS and ENVIRONMENTS * Developer's Guide to Automated Software Environments * Toois and Techniques for Designing Responsive Software * Source Text Analysis Toois * * * * * *
Experiences with the Toolpack Environment Tools for Software Testing Software Maintenance Tools Distributed Software Envirownents Tools for Developers of Distributed Software Software Environments Design Workshop
OBJECT-ORIENTED TECHNOLOGY * * * *
The Object Model in Software Engineering Programming Applications in Smalltalk-80Th Object-oriented Design with Objective-CTm Object-oriented Design with Ada"m
Fig. 2. rMise 1986 Professional Education Program.
the final segment, each participant picked one of the tZ e methods they had studied and used it, with the instructor's and other participants' help, to solve at least one of three specific problems. This final segment ended with a plenary session in which representative solutions in each method were presented and a group discussion addressed the differences and similarities among the methods. 5) Technology Improvement Program: The principal technology improvement activity has been a Software Environments Workshop cosponsored by the National Science Foundation [2]. The goal was to determine how to improve the state of the art in software environments over the next decade. It provided an indirect contribution to selection-oriented education by developing estimates of
both the current and future state of the art in the software environments arena. Participants formed groups to determine fundamental issues and required activities in several areas: tools, distributed systems, database systems, extensibility, artificial intelligence, integration, prototyping, measurement and evaluation, monolingual versus multilingual environments, human interfaces, and evolutionary versus revolutionary approaches to improvement. Each group assessed the current state of affairs, defined a desirable future state, and considered how to best move from the current to the future state. They also addressed several global topics. First, to provide a context for the issues and activities, they developed
1204
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
a characterization of the software environment situation in the mid-1990's in comparison to the situation today. Second, they considered the general nature of organizations that could successfully undertake the required activities. Third, they reconsidered the results of a similarly intentioned workshop, held five years ago [3], and updated the previous workshop's assessment to reflect the community's emerging concern for general systems issues and the software creation and evolution process.
B. The rMise Approach to Selection-Oriented Education * State-of-the-Art Technology: Attention has been given primarily to topics where the state of the art is well developed and the gap between the state of the art and the state of practice is larger than average. Several studies (see, for example, [4]) have indicated that the state of practice is far behind the state of the art in software methods and software tools/environments. rMise, therefore, has emphasized these topics. It has also treated subjects, such as project management, that directly relate to these topics and affect the extent to which these technologies are used in practice. * Coverage of Technology Transfer: Few rMise activities have focused exclusively on the processes involved in technology transfer. Rather, the coverage has been indirect. The Software Environments Design tutorial and several of the other Tools and Environments tutorials have, for example, treated general paradigms for the transfer of tool/environment technology. It is expected that this mainly indirect approch will continue for the foreseeable future until the understanding of, and experience with, software technology transfer mature significantly beyond their present levels. * Identification, Evaluation, and Choice Techniques: In rMise activities, these subjects are treated within the context of specific technology arenas. This approach emphasizes exposure to the state of the art, with specific technology selection techniques being discussed in the context of specific technologies. Selection-related material that has been included in the tutorials includes: cost estimation techniques, general paradigms for integrating technology, technology classification schemes, measurement technology, comparative evaluations of available technologies, cost/benefit analyses, and techniques for effective participation in team efforts. The treatment of selection techniques has included comparative evaluation exercises. Simple instances have occurred in the Software Environment Design tutorial, in which participants chose technologies to include in their environments. A more extensive comparison was done in the course of the Software Methods Workshop. This workshop's format provided insight into a variety of modern technology options to a level of detail which allowed (informal) comparative assessment. This format would seem to be useful for selection-oriented, experience-based studies of a wide spectrum of software technology. As in the case of technology transfer, rMise has taken
an indirect approach to technology selection. For the foreseeable future, direct treatment will probably be limited to the technology improvement activities, including workshops and study groups on fundamental issues and concepts. The state of the art and the state of practice in technology selection may eventually mature to the point
that direct treatment will be possible. But because selection techniques will always be best understood in the context of their application to specific situations, it would seem that an emphasis on indirect treatment will always be most appropriate. * Experience-Based Content: Reports on experiences in adopting and using software technology have been part of the vast majority of tutorials. The rMise approach has been to seek instructors who, themselves, have extensive experience. This approach was also used in choosing participants for the Software Environment Workshop; people actually involved in the design and construction of environments accounted for about half the participants. * Decision-Making Orientation: The rMise approach has been to orient its activities specifically toward technology selection decision makers. The Executive Summary of Software Engineering Issues tutorial is specifically designed for top-level decision makers. Most of the activities in the Professional Education Programs have been oriented toward project managers and middle-level management or people aspiring to these positions. In addition, decision makers were specifically included in the Software Environments Workshop, and the workshop's results were prepared for use by those responsible for deciding where to invest time and effort in the improvement of the state of the art for software environments. Other continuing education programs (see, for example, the plans discussed in [5]) have also included educational activities specifically oriented towards decision makers. These programs have usually adopted a tiered approach with separate "tracks" designed for executives, managers, and software engineers. rMise has emphasized meeting the needs of the manager and then broadened the scope of its programs to meet the needs of other populations using the resources and materials developed for manager education as a base for its other activities. * Exercises: Exercises have been a key component of many rMise activities. Tutorials offering extended exercises have been included in both the software methods and tools/environments tutorial sequences. A concentrated effort has been made to assure that these exercises do not suffer from the problems noted previously. In addition, several other tutorials have included simple, experienceproviding exercises. Unfortunately, it has not been possible to provide facilities for "live" demonstrations or to provide a laboratory capability. This has been corrected, somewhat, in the Software Methods Exposition, first held as part of the 1986 Professional Education Program. Manual and automated tools- supporting a variety of methods were available for use during the exposition and participants had the oppor-
RIDDLE AND WILLIAMS: EDUCATIONAL APPROACH TO TECHNOLOGY SELECTION
tunity to gain hands-on experience with these technologies. It is anticipated that rMise will expand this sort of activity, and plans are being made to provide a full-capability laboratory supporting software technology demonstration and experimentation. * Interactive Sessions: The rMise approach to fostering interaction is to keep the number of participants small. Participation in the tutorials is restricted, on a first-come first-served basis to assure highly interactive learning opportunities. Study groups are held to a maximum of 25 participants to foster interaction. Even when the number of participants is small, opportunities are sought to divide the group into working groups of 3-5 people. When the number of participants is large, as in the case of the Software Methods Workshop, the amount of time that the whole group spends together is held to a minimum, and much of this is used for open discussion. * Instructors: rMise has been fortunate to attract instructors for its tutorials (and participants for its study groups) who are well versed in their fields, familiar with both the state of the art and the state of practice, able to handle the rigors of short, intense sessions, and willing and able to participate in free-wheeling discussions. The approach has been to find top-quality people who are heavily involved in defining and expanding the boundaries of their field of interest; no other approach would satisfy the instructor criteria specified previously. V. SUMMARY
Selection-oriented education provides an exposure to new techniques and concepts rather than an in-depth coverage of those techniques and concepts. The coverage is sufficient to impart an appreciation of the technology's applicability and cost-to-benefit ratio. The intent is to enable management and technical personnel to make wellfounded decisions about whether or not to adopt a particular technology. A selection-oriented education program must satisfy many criteria. It must provide an extensive introduction to the state of the art. It must cover all of the activities supporting technology transfer and show how they interact with the specific activity of technology selection. It must cover a wide variety of techniques for performing technology selection and demonstrate their use. It must include information on actual experiences. It must serve the needs of those responsible for selecting technology; often this is an individual who is trying to decide whether or not to upgrade his or her own capabilities. It must provide for a high degree of interaction among instructors and students so that the needs of individual students are articulated and accommodated, the applicability of the material to personal situations is clear, and students are able to learn from each others' experiences. And it must
be conducted by knowledgeable, experienced, articulate, flexible instructors who are accustomed to short, intense, interactive sessions. The Rocky'Mountain Ins'titute of Software Engineering
1205
(rMise) has sponsored a number of activities that contribute to selection-oriented education. The primary activity in this arena has been its Professional Education Program. This program has included activities designed to meet the educational needs of executives, managers, and software engineers with an emphasis on the first two of these populations. rMise also sponsored a workshop that emphasized the comparative assessment of alternative methods through large-group discussion of experiences gained in small-group exercises. And the Institute organized and ran a study group to identify issues fundamental to improving the state of the art for software environments. Finally, a software methods exposition was held to provide more individualized learning through direct experience. The rMise approach to selection-oriented education emphasizes exposure to the state of the art in software methods and software tools/environments, two areas where the gap between art and practice is particularly wide. Techniques supporting technology selection are treated indirectly, in the context of their use rather than as subjects of direct focus. Experiential data are included as much as possible through the use of personnel who are actively involved in both developing the technology within their fields and transferring it into practice. The needs of decision makers at the project management, middle management, and top-executive levels are met by designing specific tutorials to meet their needs. Interactive sessions are assured by limiting participation so that groups are small, dividing participants into working groups of 3-5 people whenever possible. The majority of large-group time is devoted to open discussion. Quality instruction is assured by employing experienced instructors who are instrumental in defining and expanding the boundaries of their respective fields. While these activities do not satisfy all of the criteria for selection-oriented education, they do address the major points. We expect that continued experience in offering selection-oriented education, coupled with expanded capabilities (e.g., a hands-on laboratory) will help move the rMise program closer to the ideal. The rMise enterprise has provided a wealth of experience in setting up and running selection-oriented education programs. The body of this paper has reported our positive experiences. In closing, we would like to offer a few observations about what can go "wrong." First, we have found little interest in activities that directly address technology transfer and technology selection per se. There is interest in these topics within the context of specific situations, e.g., the identification and adoption of specific configuration management tools. There is, however, essentially no interest in these topics as "standalone" subjects. Second, we have found little interest in revolutionary technologies which, while they may be "hot" topics in the research and development community, are sufficiently different from existing technology that their introduction would cause large perturbations. The interest is in much more conservative, advanced technologies for
1206
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. SE-13, NO. 11, NOVEMBER 1987
0 which there is little question of applicability and ease of William E. Riddle (M'83-SM'84) is Chief Technical Officer and Executive Vice President at the X introduction. Third, the majority of students will be seekSoftware Productivity Consortium in Reston, VA. ing "the answer" and feel that there must be something He is responsible for establishing and guiding a that will solve all, or most, of their problems if only they software productivity improvement program oriented toward the needs of the Consortium's sponcan find it. One of the major opportunities, and one of the soring, aerospace industry companies. Prior to hardest tasks, is to get these students to realize that they joining the Consortium, he was an independent have the wrong objective. Finally, some students (luckconsultant, founder, and director of the Rocky Mountain Institute of Software Engineering, a diily, a minority) will lack the flexibility and tolerance for of software design and engineering work at ambiguity that are necessary when considering technol- Cray Laboratories, and arector professor of computer science at the Universities ogy selection. They will be sophisticated enough to know of Michigan and Colorado. His technical work emphasizes techniques for description and analysis of software designs and the delivery of this that there is no single answer, but they will want struc- the technology to practitioners through automated software environments. tured courses that deliver logical assessments of technolDr. Riddle is a former ACM National Lecturer and former Chair of the ACM Special Interest Group on Software Engineering. He is frequently ogy for a wide spectrum of situations.
ACKNOWLEDGMENT The success of rMise is due, in iarge measure, to the instructors who have participated in the Professional Education Programs over the past three years. Without their generous and capable assistance, this program would not have been possible. REFERENCES [11 H. D. Mills, "Software enginering education," Proc. IEEE, vol. 26, pp. 1158-1162, Sept. 1980. [2] W. E. Riddle and L. G. Williams, "Software environments workshop report," ACM Sigsoft Software Eng. Notes, vol. 11, pp. 73-102, Jan. 1986.
[3] L. J. Osterweil, "Software environment research directions for the next five years," IEEE Computer, vol. 14, pp. 35-43, Apr. 1981. [4] M. V. Zelkowitz, R. Yeh, R. G. Hamlet, J. D. Gannon, and V. R. Basilli, "Software engineering practices in the US and Japan," IEEE Computer, vol. 17, pp. 57-66, June 1984. 15] M. Ulema and J. Larsen, "Planning for in-house software engineering education," in Proc. 15th SIGCSE Tech. Symp. Comput. Sci. Educ., Philadelphia, PA, Feb. 1984, pp. 130-136.
involved in organizing workshops and conferences and chaired the 9th International Conference on Software Engineering. g3
kg *
;
Lloyd G. Williams received the A.B. degree from Colgate University, Hamilton, NY, in 1969 and the Ph.D. degree in physical chemistry from the
University of Wisconsin in 1976. He is an Associate Professor in the Department of Electrical Engineering and Computer Science at the University of Colorado, Denver. Prior to joining the faculty at the University of Colorado, he was at Hampshire College in Amherst, MA, where he taught computer science and served as founding chairman of the college's multidisciplinary Computer Studies Program. His principal research interests are in software engineering, particularly software development methods, software tools and environments, and distributed systems. He has served as a consultant on software development for a variety of industrial organizations. He is also Director of the Rocky Mountain Institute of Software Engineering, a nonprofit organization founded to assist in the transition of modem software engineering technology into widespread practice. Prior to becoming Director of the Rocky Mountain Institute, he served as Coordinator for Technical Activities. Dr. Williams is a member of the Association for Computing Machinery and the IEEE Computer Society.
