Support for Collaborative Design: Animated Electronic Meetings
Support for Collaborative Design: Animated Electronic Meetings Gert-Jan de Vreede Delft University of Technology School of Systems Engineering, Policy Analysis and Management P.O. Box 5015, 2600 GA Delft, The Netherlands e-mail: [email protected]
Abstract In an action research study at the Amsterdam Municipal Police Force, we critically evaluate the combined use of Group Support Systems, joint modeling facilities, and animation techniques to support subject matter expert (SME) involvement in organizational change processes. The SMEs involved in the study jointly constructed and evaluated static and dynamic models of the police force's current and future structure and processes. Our findings illustrate the potential of the above facilities used to support SME involvement in terms of efficiency, resulting model quality, and SME satisfaction with process and technologies. Issues for further research include group modeling strategies, techniques, and group size, as well as automatic animation generation.
1. Introduction Given the high rate of change in technology and working environments, many organizations have evolved to the point where their structure, procedures, and technologies are no longer in keeping with the needs of the future [9]. This means that organizations need to study themselves to determine what should be changed. Their components must be reshaped and rearranged to form appropriately coupled new subsystems with greater simplicity and internal cohesiveness than the existing structures. (Information) systems need to be adapted or developed to meet the requirements of the new structures and processes. Recognition must also be given to relationships between the organization and its external environment. An example of an organization that found itself in the situation described above, concerns the Criminal Investigations Department (CID) of the Amsterdam Municipal Police Force. The CID has a difficult task combating organized crime in the capital of the Netherlands. They have limited means (time, money, personnel) to employ against too many suspect criminal organizations. In 1994, the CID management decided to reorganize the functional specialized department into a multi-disciplinary project
based organization in order to utilize their means more effectively and efficiently. However, it was unclear (1) how this transition had to take place, (2) how the project teams, so-called units, and their progress had to be managed, and (3) which technologies should be (developed and) employed to support the future organization. The situation at the CID provided us with an excellent opportunity to apply, evaluate, and further improve a design approach which focuses on the collaborative analysis and design of organizational structures, processes, and information technology. In earlier studies, we investigated various methods, techniques, and tools to support organizational change processes. These studies individually showed the potential of object oriented modeling, simulation and animation techniques, and Group Support Systems [8; 18; 19]. They also suggested that during an organizational change process, one of the main issues is one of interaction: 1. A lot of interaction is required between and among subject matter experts (SMEs) from the organization and analysts to exchange and discuss information and ideas with respect to the organization’s current situation and various design alternatives. 2. Interaction between the people involved in the process is time consuming and difficult because of differences in background, expertise, and personal objectives. In the study described in this paper, we addressed this issue of interaction by marrying two techniques that in separate situations had shown to facilitate interaction between and among members of an organization and analysts: GSS and animated simulation models. Hence, our research objective was to ‘explore the possibilities of the combined employment of GSS and animation techniques to facilitate organizational change in a collaborative fashion.’ By ‘collaborative fashion’ we mean that a group of SMEs from the organization is actively involved in the various modeling and design activities during the change process. For the study described in this paper, we used action research as our research approach. Action research aims to achieve the dual outcomes of action (or change) and research (understanding) [1]. An action researcher participates or
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
intervenes in the situation under investigation in order to apply a theory to practice and evaluate its worth [4]. The reason for employing action research is threefold [2]. First, we needed to study organizational change in its natural setting. It would be very hard to use a constructed setting, that is a valid representation of the complex reality in which our study is carried out. Second, we are interested in ‘how’ and ‘why’ questions. It is more important to explore the process of applying GSS and animation techniques than the resulting products. Third, there are few previous studies and there is a lack of elaborate theoretical understanding with respect to the combined application of GSS and animation techniques in the organizational change arena. In the remainder of the paper, we first give an overview of the design approach we employed and, in particular, the way in which GSS and animation techniques support the various design activities. Then, we describe the results of our study at the CID. The paper concludes with a summary of the most important findings and issues for further research.
2. A Collaborative Design Approach to Organizational Change Experience shows that organizational change is easier said than done. It is a complex activity. First, the goals of the people involved may conflict with those of the organization. Second, there are many possible courses of action to change an organization. Beforehand, it is not clear which course of action will be most efficient and effective. Third, there are usually many options or alternatives for change. As the effects of these change alternatives are often dangerous or impossible to test in reality, the effects of these changes are seldom known in advance. Finally, apart from `rationale', there may be other (conflicting) issues, such as political, cultural, and social issues, that influence the actual change process. It is, therefore, not surprising that a considerable number of approaches and supporting tools have been proposed to facilitate organizational change efforts.
2.1. Problem solving for organizational change As facilitating organizational change can be considered as solving an ill-structured problem, a problem solving strategy is needed that addresses the intricate attributes of such a process. Based on literature research and on experiences from a large number of field studies, we suggest employment of the problem solving process depicted in figure 1 [17]. The problem solving process consists of several design activities that can be carried out in an iterative fashion. First, the problem situation in an organization is conceptualized in order to structure the problem situation in such a way that the efforts for detailed, low-level data gathering for creating the empirical model can be focused and minimized. Next, a
descriptive empirical model is build that can be experimented with in order to analyze and diagnose the problem situation. For this purpose, the empirical model is, for example, implemented in a simulation language. The third activity is aimed at seeking solutions and capturing them in a number of prescriptive models of possible solutions. These models can be compared in order to study the (effects of the) alternatives in more detail. The actual choice is made in the next phase and consists of a combination of possible solutions or can leave the situation as it is. Finally, in order to actually solve the problem situation, the solution has to be implemented. Current organization
A. B. C. D. E.
Conceptual model
Empirical model and Problem Diagnosis
Conceptualize problem Create and validate empirical model, diagnose problem Construct alternative models and conduct experiments Choose most preferred solution Implement solution
Current organization
Model of the Chosen Solution
Models of Possible Solutions
Figure 1 Problem solving for organizational change
2.2. Collaboration in organizational change In practice, many people are usually involved in the various design activities described above [15; 16; 21]. First, there is a team of analysts that is responsible for building and analyzing models of current and future organizational processes. Second, there are people that actually execute the processes, the subject matter experts (SMEs). They have to provide the analysts with relevant information on their working situation. Finally, there are people, the decision makers, that supervise the change process in the organization. Presented with the results of the analysts, they decide which changes the organization is going to make. Frequent interaction between these parties is of vital importance for the quality of the design process and its outcome, see e.g. [22]. SMEs' opinions and perspectives regarding the functioning of the organization may change during the design process, thereby resulting in changes in organizational requirements with respect to processes and systems. Also, it is believed that SME participation enhances the acceptance of eventual changes in their work situation. Furthermore, organizational change processes are usually far too complex for a single person to understand all aspects, issues, and variables. The desire for a design process that is as much collaborative in nature as possible, presents us with a time/completeness paradox. On the one hand, collaborative design efforts need to be time effective. An organization
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
cannot take too long to build models to plan changes if its competitive and financial positions are deteriorating. However, traditional methods for (participative) modeling are often time consuming [15]. They require analysts to move from small group to small group, gathering information, defining terms, resolving differences, trying to make a big picture from all the small fragments. On the other hand, collaborative design efforts pursue completeness. It can be argued to involve as many SMEs as possible. This would generate more ideas with respect to an organization’s current and future situation. It would enable a more complete picture of an organizational situation. Also, broader participation promotes wider buy-in to the outcomes of a study. Finally, larger groups of SMEs are able to evaluate ideas more extensively. However, the broader the SME participation, the more time is needed to carry out the design process. Hence, the participative nature of organizational change efforts sets the requirement to use techniques, methods, and automated support that facilitate the collection, organization, and communication of various types of design information. This design information includes, but is not restricted to, lists of organizational requirements, problem statements, and (dynamic) models of processes and structures. We propose the application of two technologies that together meet the requirements mentioned above: Group Support Systems and animated simulation models. We address these technologies in the next sections.
2.3. Group Support Systems GSS represent information technology that aims to make group meetings and group decision making more productive. Over the years a great deal of research effort has been devoted to the development and application of GSS. As a result of field studies and a large number of experiments, a substantial body of knowledge has emerged about the effects of GSS on group processes and outcomes, especially with respect to GSS used in same time, same place settings. Compared to ‘traditional’ group settings, these types of GSS meetings appear to be more efficient and more effective, while the participants are more satisfied with the results and the process, see e.g. [7; 11]. Several attributes have been identified that appear to bring about the added value of GSS meetings over non-GSS meetings. These attributes include: • Anonymity. By being able to enter ideas, comments, and votes anonymously, silent or shy participants are more encouraged to participate. Group members are hindered from dominating by position or personality. Ideas appear to be judged on their merit, not on the personality of the person that submitted it. • Parallelism. By generating ideas and communicating them in parallel, participants get equal `air time'. This prevents production blocking so that participants can
spend more time on generating new ideas. Furthermore, working in parallel allows groups to generate more ideas. In this respect, it is noteworthy that the number of high quality ideas appears to be directly proportional to the total number of ideas [14]. • Group memory. During an electronic meeting, all ideas, comments, and votes are stored electronically. Hence, previous meeting results are readily available in a follow-up meeting. There is little time needed to produce meeting minutes. Moreover, there is a fundamental difference between the quality and completeness of electronic group memory and that of `traditional' minutes [5]. First, ‘traditional’ minutes are likely to be more subjective as they are written by an individual. The electronic group memory gives an objective history. Second, minutes not often describe participants' initial positions or interim postures. Group memory depicts the evolution of the group's position over time. Finally, group memory makes each comment during a meeting public, to which others can refer at a later stage. • Group size. Findings suggest that GSS have a greater positive impact on large groups (8 members or more) in terms of productivity and participant satisfaction, [6]. In other words, GSS seem to facilitate the effective and efficient participation of larger number of SMEs in a collaborative design process. In table 1, we identify the way in which GSS were used to support the design activities in the CID study. The basic activities of the design approach discussed in Section 2.1 are split into a number of sub-activities, see also [17]. For each subactivity, the parties involved are identified. In order to identify the type of group support we considered to be most appropriate, the nature of each subactivity was determined. Based on the meeting task taxonomy of Bostrom et al. [3], we differentiated between three meetings tasks: generate, organize, and evaluate. Non-meeting tasks are labeled `other'. Finally, we identify which GSS (GroupSystems V modules or TeamGraphics) we used for each subactivity in the CID study.
2.4. Animated simulation models During the process of problem solving described in Section 2.1, a number of models of organizational processes are constructed and analyzed. Modeling organizational processes is a complex activity. First, organizations can be modeled from different perspectives: the way a manager
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 1 GSS for collaborative design support ACTIVITY
Construct conceptual model • Gather conceptual data • Define problem boundaries • Identify relevant object classes • Define relevant object classes • Construct aspect models Construct empirical model • Formulate organizational requirements • Construct simulation model • Gather empirical data • Formulate problem statements • Relate problems to requirements • Validate static empirical model Diagnose problem situation • Formulate cause hypotheses • Investigate cause hypotheses • Formulate solution directions Construct models for change • Identify possible solutions • Select solution to elaborate • Conceptualize solution • Construct simulation model of solution • Evaluate solution Choose change alternative • Compare evaluation results • Determine alternative to implement Implement solution • Write policy document • Introduce changes • Perform post evaluation
ACTORS
NATURE
GSS
Both Both Analyst Analyst Both
Generate Generate & Organize Generate & Organize Generate & Organize Generate & Organize
Topic Commenter Topic Commenter Idea Organizer TeamGraphics TeamGraphics
Both Both Both Both Both Both
Generate Other Generate Generate & Organize Organize Generate & Evaluate
Idea Organizer Topic Commenter Group Outliner Group Outliner -
Both Both Both
Generate & Organize Generate & Organize Generate & Organize
Group Outliner Group Outliner Topic Commenter
Both Both Analyst Analyst Both
Generate & Organize Evaluate Generate & Organize Other Generate & Evaluate
Group Outliner Vote Topic Commenter
Both Both
Organize Evaluate
Topic Commenter Vote
Analyst Analyst -
Other Other Other
-
looks at a process is usually different from the way the one who executes it does. Second, an organizational process usually consists of a great many activities that can be modeled. Third, carrying out activities is often timedependent: many activities are closely adjusted to one another or are being performed in parallel with one another. Consequently, modeling organizational processes requires models which can reflect different abstraction levels, parallel activities and an explicit time dimension. In addition, for analysts, SMEs, and decision makers, models of current and future organizational processes are used as a vehicle of communication. Hence, it is imperative that these models are easy to communicate. This means, that a model has to closely resemble the mental model of the person it is communicated to [4]. Experience suggests that traditional modeling techniques, such as ERDs, DFDs, or IDEF0 models do not meet these requirements [21]: • Diagramming techniques may be able to model the dynamics of a situation, but the resulting diagrams are static in themselves. This makes it difficult to
thoroughly understand and analyze the dynamic characteristics of the modeled processes. • Diagramming techniques are often difficult to communicate to SMEs and decision makers. They usually lack the required training and expertise to build and understand the resulting models. Also, diagrams seldom resemble a problem situation visually. In this respect, simulation techniques appear to be more useful for developing models to analyze process dynamics [21]. However, a simulation model itself can still be considered to be nothing more than a black box hiding the dynamics of the problem situation. It is difficult to investigate the modeled processes by only going through large sets of input and output data. We propose that using animation techniques on top of simulation models has the potential of overcoming this problem. An animation model graphically portrays the dynamic behavior of an process by showing icons that visually resemble objects from the real world, move around a computer screen, changing shape, color etc. Because of their ‘cartoon-like’ behavior,
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
animation models have the potential to resemble closely the mental models of both problem owners and problem solvers. Animation models support the communication between and among analysts, SMEs, and decision makers in various ways [21]: • For an analyst, the use of animation models may decrease the development time of the simulation model because of the enhanced debugging possibilities. With animation it may be quicker to find and localize mistakes than by going through output of traces or using a debugger. • The fact that the visual process information in an animation model is less abstract and, therefore, easier to communicate to a group of SMEs, increases the SMEs' understanding of the organizational processes. As a result, SMEs may participate more actively in the design process. Their contribution may shift from gaining ‘understanding’ to providing ‘insight’. This may lead to model improvements. Also, the SMEs may appreciate and value their involvement more. • During the construction of a descriptive empirical model, using animation offers unique possibilities for face-validity tests. Structural mistakes in the model or deviant behavior can be pointed out by the SMEs. • In the solution seeking activity, animation may help to point out inadequacies of possible solutions which the SMEs or decision makers want to test. With animation these shortcomings may be presented such that the decision makers can understand and accept them. • Animation models help focus on particular aspects of a process under investigation, without having to go through large amounts of printed, numerical data. This is especially useful when presenting the results of a process analysis to a decision maker. In other words, an animation model can illustrate statistical results in an easy accessible manner. At present, animation is often a standard feature of simulation environments. A survey of simulation software showed that 80% of the simulation packages had animation capabilities [13]. In the CID study, we used the simulation language Arena, which is the successor of SIMAN/Cinema [12]. Arena is a general purpose simulation language, which supports both discrete event simulation and continuous simulation. It can be used in both production and service environments. Apart from modeling and portraying the dynamics of organizational processes, it offers facilities to design simulation experiments and to analyze numerical results.
3. The CID study The design approach described in the previous section, was applied to the CID case to support the change process that was taking place. Below we illustrate the application of the
approach with respect to the construction of the conceptual model, the empirical model, and the models for change. We first give some background information on the design sessions in which these models were built and evaluated.
3.1. Session background In the course of the study, five sessions were organized with respect to the models of the CID’s current and future processes, see table 2. The first two sessions lasted about three hours, the other sessions lasted about six hours each. In total, there were 40 participants, 27 of which were unique. The participants used GroupSystems V in each session. TeamGraphics was used to support the construction and evaluation of the conceptual model of the current situation. Arena was used for the same purpose with respect to the models for change. GSS was not a new phenomena for the majority of the participants. About 75 percent had extensive experience using GroupSystems. There were two analysts involved in the study. Each session took place in the Group Support Facility (GSF) at Delft University of Technology (figure 2). The GSF layout consists of a meeting room with a U-shaped table facing a public screen, flanked by a number of breakout rooms. The table consists of 15 slightly wedge-shaped segments with a sunk-in CRT. The public screen has been implemented using a beam projector placed overhead, powerful enough to generate a crisp image even in daylight. The GSF infrastructure serves as a platform for several GSS applications such as GroupSystems V, GroupSystems for Windows, and TeamGraphics.
Figure 2 The Group Support Facility in the study.
3.2. Conceptual model The conceptual model of the CID’s current situation was built in a number of steps following an incremental modeling strategy. First, we held a number of short
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 2 Background information on the model sessions during the study.
Session 1. Conceptual model prototype 2. Conceptual model prototype 3. Conceptual and empirical model 4. Models for change 5. Models for change
Participants 4 4 8 12 12
interviews with some key actors in the CID. In these interviews, information was gathered on the relevant processes in the CID. Second, during the first two sessions, we built first version (prototype) of the conceptual model in TeamGraphics together with a key CID employee. Additional information to build this model was collected using GroupSystems. Part of the prototype model is depicted in figure 3. The modeling technique used concerns ‘process modeling’ where, based on a set of object definitions, the interdependencies between the constituent activities of a process are depicted together with the actors that perform these activities [8]. Finally, in the third session, we presented the prototype of the model in TeamGraphics to a group of CID employees. During this presentation, we used TeamGraphics in ‘facilitation mode’ [23]. Participants could browse through the model. They did, however, not have privileges to make modifications themselves. For this purpose, we used a dedicated chauffeur, that immediately followed up on all participants’ requests.
GroupSystems x x x x x
2.
3.
4.
Tools used TeamGraphics x x x
Arena
x x
current situation. In addition, the CID management requested that it were to be used as a basis in other reorganization efforts in the CID. It was important for the CID actors involved to have a shared view about the situation and the problems that they face. In other words, their mental models of the problem situation needed to be in harmony. This is important to arrive at a diagnosis of the problem situation and a proposed solution that everybody agrees upon. We reasoned that by discussing and improving the prototype of the conceptual model as a group, the mental models of the group members would converge. The fact that all participants expressed that the resulting model was a valid representation of the CID’s processes, confirmed the fact that their mental models converged. We feel that the group modeling process was more efficient that a traditional process where a model is built during a iterative series of individual interviews with SMEs. The prototype of the conceptual model was constructed in six hours. The discussion and modification of this model took another three and a half hours. During this time, a great majority of the SMEs' suggestions to modify and expand the model had been processed. We expected the SMEs to have a tendency to overemphasize details with respect to their processes and hence invest a disproportionate amount of modeling efforts. However, analysis showed that the model size only increased by about 18 percent (from 101 elements to 119 elements) [18]. In other words, the modeling process remained manageable in terms of modeling focus and level of detail.
3.3. Empirical model Figure 3 Part of the conceptual model. Regarding the group modeling process supported by TeamGraphics, we make the following observations: 1. The active involvement of the CID employees in the modeling process effectuated the implicit verification of the resulting model. All participants agreed that the final model was a valid representation of the CID's
Like the conceptual model, the empirical model of the current situation was also constructed in a number of participative steps representing an incremental modeling strategy. First, we collected a number of problems ourselves that we identified during the modeling process of the conceptual model. Second, during the same session in which the conceptual model was evaluated and modified, the CID SMEs used GroupSystems to identify and
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
prioritize a number of organizational requirements (comparable to critical success factors). Finally, the group was invited to brainstorm about known and believed problems that hindered the CID from meeting its organizational requirements. To stimulate this brainstorming activity, we entered our initial list of problems to the outline before the SMEs could enter their ideas. For the brainstorming activity, we used GroupSystems V’s Group Outliner module in combination with TeamGraphics. Every SME had access to two adjoined terminals, one running TeamGraphics, the other one Group Outliner. The SMEs could use TeamGraphics to ‘walk through’ the conceptual model during the brainstorming exercise. In Group Outliner we outlined the hierarchical structure of the conceptual model’s process models. The SMEs were asked to enter the problems they perceived as comments to the most appropriate part of the outline. This implied that very specific problems were entered as comments to the task they related to, whereas the most general problem statements were entered at the top level of the outline. The outline representing the hierarchical structure of the conceptual model’s process models is depicted in figure 4.
reacted on problem statements immediately, fueling the discussion about some problems and enforcing clearer, more detailed problem descriptions. This saved the time needed to validate the analysts’ perception of the problems with each individual SME.
3.4. Models for change Based on the problem statements gathered during the previous activity, a number of potential solutions were identified by the analysts in cooperation with a number of key CID employees. These solutions entailed various structural modifications and extensions with respect to the way in which the main processes in the CID were carried out. The further design of these solution was carried out in a number of steps. First, each new or modified main process was conceptualized by the analysts. Second, these conceptual models for change were transformed into animated simulation models in Arena. These animation models portrayed the dynamic behavior of the actors in the envisioned situation in terms of the order and the interdependencies of the activities they carried out. An example of an animation model is shown in figure 5.
Figure 4 The hierarchical structure of the conceptual model in Group Outliner. Our own initial problem list consisted of 38 problems. The participants extended this list with another 71 problem statements. We found only a small overlap (9 problems) between the two sets of problems. In other words, the SMEs had a significant contribution to the analysis of the CID's current situation. The main difference between the two sets of problems related to their focus: strategy (analysts) vs. operational (SMEs). Clearly, these two focuses complemented each other well. In this sense, the SMEs did not question our perceptions, but accepted it as a basis for a more thorough assessment of the situation. Identifying the problems of CID's current situation in a group setting brought about a rich picture of the problem situation. The SMEs not only outperformed the analysts in terms of the number of problems generated. They also
figure 5 An example of an animation model depicting the proposed new processes. Third, each animated model was presented to two groups of SMEs (20 people in total) during the fourth and the fifth session. During each session, the proposed process was introduced and explained to the SMEs and a demonstration of the animation model was given. As soon as the model and the process it depicted was clear to the SMEs, GroupSystems V’ Topic Commenter module was used to discuss the proposed new or modified process. During this activity, the SMEs were encouraged to be as critical and creative as possible. The SMEs entered comments and held electronic discussions on five predefined discussion topics: ‘expected problems with the
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
proposed process’, ‘questions about the proposed process’, ‘alternative ways to overcome the problems addressed in this process proposal’, ‘ideas regarding a project management system’, and ‘remaining comments and ideas’. Finally, based on the SMEs’ feedback and the subsequent group discussions, the definitive design of the process was finalized immediately after the sessions. The CID management approved of the final design of the processes and initiated a number of activities required to achieve the implementation of the proposed changes. The decision to communicate the models for change to the SMEs as animation models was based on the assumption that the proposed changes would be easier to fully understand which would facilitate a critical evaluation. However, a previous study showed that SMEs may seem overwhelmed by animation models and become interested viewers rather than critical evaluators. This is referred to as the ‘Pacman effect’ [10]. We anticipated this risk by keeping the models as simple as possible and by consciously stimulating the SMEs to have a critical brainstorming attitude. During the introduction of the models, we indeed witnessed a short period in which the SMEs seemed to a bit awed by the models. However, as soon as they had the opportunity to react to the model and engage in an (electronic) discussion of the model, their passiveness made way to a critical and assertive attitude. This was not only illustrated by the tone of their comments and by the drive with which they reacted to each other’s comments, but also by the number of comments, ideas, and suggestions that were submitted. In total, the 20 SMEs participating generated 399 comments, nearly 60 pages of output, covering the five predefined discussion topics with respect to the proposed processes. Hence, we conclude that using animated models in a GSS environment did not seem to harm the SMEs’ active involvement in the design and evaluation process. However, we need to point out that the SMEs’ motivated participation also to some extend seemed to be related to the ‘perceived correctness’ of the animation models. During the sessions, we found that small logical inconsistencies or impossibilities triggered the SMEs to study the proposed process very carefully. In other words, they were stimulated by the ‘imperfectness’ of the models. With respect to the animation models themselves, we surveyed the SMEs whether they felt that the models sufficiently clarified the proposed processes. Their average response was 3.7 (standard deviation 0.9) on a five point scale, 5 between the most positive. In general, the SMEs found the animation models somewhat complex. They thought there were too many elements moving on the screen and too many interdependencies shown to comprehend the proposed situation easily. During the second session, we dealt with this problem by taking more time to explain the models and let the participants get used
to them. Still, compared to the ‘paper’ conceptual process models, the SMEs liked the animation models very much. As one of them remarked: “That animation model tells me more than ten pages of process models!”
3.5. Participants' reflections During and after the sessions, we handed out questionnaires and held personal interviews to investigate the SMEs’ attitude toward the group design process and the GSS technology used. Group design process On the whole, the group design sessions left a positive impression with the participants. The results of the questionnaire are presented in table 3. (In the original (Dutch) questionnaire, not all questions were formulated with a ‘positive’ orientation. For the sake of presentation, we transformed all questions in tables 3 and 4 into positive oriented questions). From the results in table 3 it follows that the SMEs were very satisfied with the sessions, see question 1. They felt comfortable with the way in which the sessions were structured and the issues that were addressed. Furthermore, they subscribed to the results of the session, see question 2, while at the same time they thought that the session helped them to gain new insights, see question 3. The participants felt a lot of consensus was reached, see question 4. This, we feel, is of critical importance to the success of participative design efforts. Notwithstanding the fact that during the ‘models for change’ sessions there were some energetic discussions of various change alternatives between individual actors, the participants obviously enjoyed working together during the sessions, see question 5. The composition of the group was evaluated lower than in the previous study: 3.8 against 4.2. From comments of the participants, this turned out to be due to the fact that during one of the sessions, a lot of people canceled. Consequently, the remaining participants felt that the group was incomplete. Finally, the participants' overall positive attitude toward the sessions is illustrated by their willingness to attend future sessions, see question 7. GSS technology used During the group sessions, we employed both TeamGraphics and GroupSystems V. As TeamGraphics was only used by a few people, we did not hand out
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 3 Questionnaire results with respect to the group design sessions. Question 1 2 3 4 5 6 7
Are you satisfied with the session? Did the session results confirm your ideas on the session topic? Did the session help to gain new insights on the session topic? Has enough agreement been achieved among the participants? Did you enjoy the collaboration with the other group members? Do you approve of the composition of the group? Are you willing to participate in a follow-up session or activity?
Mean*
Std
4.4 3.9 3.9 4.3 4.4 3.8 4.7
0.5 0.9 0.6 0.8 0.6 1.1 0.4
* On a five point scale, 1 being very negative and 5 being very positive.
Table 4 Questionnaire results with respect to the use of GroupSystems V. Question 8 9 10 11 12 13
How do you perceive the quantity of the session results? Is a GroupSystems meeting more productive than a manual meeting? How do you perceive the quality of the session results? How well did you understand GroupSystems V? Did you find GroupSystems V user friendly? Did you find GroupSystems V interesting to use?
Mean*
Std
4.2 4.7 4.2 4.3 4.7 4.5
0.9 0.4 0.6 0.7 0.4 0.4
* On a five point scale, 1 being very negative and 5 being very positive.
quantitative questionnaires. SMEs’ suggest, however, that this technology has a high learning curve and takes some time to get used to it. As a result, we used a chauffeur to translate SMEs comments and suggestions directly into the model during part of the modeling exercise. The SMEs felt very comfortable with this working procedure. The application of GroupSystems V was evaluated using questionnaires. The SMEs involved gave the same positive feedback as in previous studies [19;20]. They liked the ability of GroupSystems to focus and structure a discussion and to work simultaneously and anonymously. The questionnaire results are presented in table 4. Regarding the use GroupSystems, the SMEs gave a lot of positive feedback. The SMEs felt that the quantity of session results was high, see question 8. The SMEs clearly attributed this high productivity to GroupSystems V, see question 9. One of the participants phrased it as follows: “It's more productive because there's less yackety-yack, little distraction and no endless discussions. You are forced to get to the point.” With respect to the quality of the sessions' results (question 10) the participants also displayed a positive impression. Like in previous studies, the SMEs expressed great enthusiasm about GroupSystems itself. They felt they understood the technology very well (question 11), and found it very user friendly and interesting to use (questions 12 and 13).
4. Conclusions and future research In this study, we explored the possibilities of the combined employment of GSS and animation technologies. The situation at the CID represented a realistic research
environment. The results of the study clearly indicate the potential of the technologies concerned during organizational change efforts: 1. Required design information was efficiently gathered and processed. 2. The SMEs involved got highly motivated to actively take part in the design effort. 3. The constructed models were effective in the sense that they triggered detailed collaborative evaluations and resulted in implementation proposals that were accepted by the SMEs involved. 4. The SMEs were very satisfied both with the collaborative design process and the process support. Based on our experiences, we argue that the GSS facilities and animation techniques are complimentary and that their application will be beneficial to the process of facilitating organizational change. However, we also experienced a number of issues that require further attention: 1. Keeping the momentum. It turned to be difficult at certain stages in the project to keep the momentum. As the efficiency of SME involvement increases and the amount of contributions increases as well, the analysts need more time to process all contributions into one meaningful problem analysis and design. However, because of their good experiences during the sessions, the SMEs’ expectations of the process are (too) high with respect to the momentum of the project. 2. Information management. During the various sessions with the SMEs, a lot of design information was collected. It sometimes proved to be difficult to keep this information organized and maintain relationships between pieces of information throughout the project.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
3. Construction of animation models. Transforming a conceptual process model into an animation model can be a time consuming activity. In the CID study, three man weeks were spent building the animation models. 4. Flexible animation models. The animation environment used was not flexible enough to build and modify animation models during sessions. Also, it would have been nice if the SMEs were able to manipulate the animation models themselves. In other words, we felt the need for a ‘group animation model’. 5. Group modeling support. TeamGraphics appears to be a rather complex group modeling tool for SMEs with little drawing tool experience to effectively use it without continuous support from experts. At this moment experiences with design approaches to organizational change that use the technologies described in this paper are still rare. Research is needed to further investigate the issues discussed above. In addition, we found a number of other issues for further research to achieve a more detailed understanding of technology supported collaborative design. With respect to the group modeling technology these issues include the productivity of certain group modeling strategies, the optimal group size and facilitation mode for group modeling efforts, and the modeling techniques that are most suitable to be used by groups of non-modeling experts. With respect to the animation technologies these issues include the automatic transformation of conceptual process models into animation models, and a detailed investigation into the added value of animation model over process models on paper or non-animated simulation models. Although this study confirmed earlier field evidence that suggests that animation models facilitate and enhance the communication processes between people involved in an organizational change project [21], further evidence is needed to determine the exact nature of these benefits. Important questions that remain in this respect concern the levels of detail and complexity of an animation model.
References [1] ARGYRIS, C., AND D.A. SCHÖN, Participatory Action Research and Action Science Compared - A Commentary, American Behavioral Scientist, Vol. 32, No. 5, pp. 612-623, 1989. [2] BENBASAT, I., D.K. GOLDSTEIN, AND M. MEAD, The case research strategy in studies of information systems, MIS Quarterly, Vol. 11, No. 3, pp. 369-386, 1987. [3] BOSTROM, R.P., R. ANSON, AND V.K. CLAWSON, Group Facilitation and Group Support Systems, in: Jessup, L.M., J.S. Valacich (eds), Group Support Systems - New Perspectives, Macmillan Publishing Company, New York, 1993. [4] CHECKLAND, P.B., Systems Thinking, Systems Practice, John Wiley & Sons, Chichester, 1981. [5] CULNAN, M.J., AND M.L. MARKUS, Information Technologies, in: Jablin, F.M., L.L. Putnam, K.H. Roberts, and L.W. Porter (eds).,
Handbook of Organizational Communication - An Interdisciplinary Perspective, pp. 420-443, Sage Publications, London, 1987. [6] DENNIS, A.R., Electronic Support for Large Groups, Journal of Organizational Computing, Vol. 4, No. 2, pp. 177-197, 1994. [7] DENNIS, A.R., AND R.B. GALLUPE, A History of Group Support Systems Empirical Research: Lessons Learned and Future Directions, in: Jessup, L.M., J.S. Valacich (eds), Group Support Systems - New Perspectives, Macmillan Publishing Company, New York, 1993. [8] EIJCK, D.T.T. VAN, AND G.J. DE VREEDE, The Dynamics of Organizational Coordination, Proceedings of HICSS-28, IEEE Computer Society Press, 1995. [9] HUBER, G.P., The Nature and Design of Post-Industrial Organizations, Management Science, 30(8), pp. 928-951, 1984. [10] MEEL, J.W. VAN, Dynamics in Business Engineering, Doctoral dissertation, Delft University of Technology, The Netherlands, 1994. [11] NUNAMAKER, J.F. JR., A.R. DENNIS, J.S. VALACICH, D.R. VOGEL, AND J.F. GEORGE, Electronic Meeting Systems to Support Group Work, CACM, 34(7), pp. 40-61, 1991. [12] PEGDEN, C.D., R.E. SHANNON, AND R.P. SADOWSKI, Introduction to Simulation using SIMAN - 2nd Edition, McGrawHill, 1995. [13] SWAIN, J., Simulation Software Survey, OR/MS Today, pp. 81102, October 1991. [14] VALACICH, J.S., A.R. DENNIS, AND T. CONNOLLY, Idea Generation in Computer-Based Groups: A New Ending to an Old Story, Organizational Behavior and Human Decision Processes, Vol. 57, pp. 448-467, 1994. [15] VOGEL, D.R., N. KATIC, AND B. NEVSTRUJEV, Automated Support For Group Process Modeling, in: G.J. de Vreede, and H.G. Sol (eds), Proceedings of the INFORMS'95 Sessions on GDSS as Support for Designing Organizations and Information Systems, Delft University of Technology, 1995. [16] VOGEL, D.R., R. ORWIG, D. DEAN, J. LEE, AND C. ARTHUR, Reengineering with Enterprise Analyzer, Proceedings of HICSS-26, IEEE Computer Society Press, 1993. [17] VREEDE, G.J. DE, Facilitating Organizational Change, PhDthesis, Delft University of Technology, 1995. [18] VREEDE, G.J. DE, Participative Modeling for Understanding: Facilitating Organizational Change with GSS, Proceedings of HICSS-29, IEEE Computer Society Press, 1996. [19] VREEDE, G.J. DE, S.O. DEN HENGST, AND H.G. SOL, Facilitating User Involvement in Information System Design and Development with GDSS - The Organized Crime Case, in: Olfman, L. (ed.), Proceedings of the ACM SIGCPR Conference, Nashville Tennessee, 6-8 April, 1995. [20] VREEDE, G.J. DE, P.C. MULLER, Why some GSS meetings just don't work: Exploring critical success factors for electronic meetings, working paper, Delft University of Technology, 1996. [21] VREEDE, G.J. DE, AND A. VERBRAECK, Animating Organizational Processes, Simulation Practice and Theory, Vol. 4, pp. 245-263, 1996. [22] WANNINGER, L.A., AND G.W. DICKSON, GRIP - Group Requirement Identification Process, Proceedings of HICSS-25, IEEE Computer Society Press, 1992. [23] WATSON, R.T., M.B. ALEXANDER, C.E. POLLARD, AND R.P. BOSTROM, Perceptions of Facilitators of a Keypad-Based Group Support System, Journal of Organizational Computing, Vol. 4, No. 2, pp. 103-125, 1994.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Abstract In an action research study at the Amsterdam Municipal Police Force, we critically evaluate the combined use of Group Support Systems, joint modeling facilities, and animation techniques to support subject matter expert (SME) involvement in organizational change processes. The SMEs involved in the study jointly constructed and evaluated static and dynamic models of the police force's current and future structure and processes. Our findings illustrate the potential of the above facilities used to support SME involvement in terms of efficiency, resulting model quality, and SME satisfaction with process and technologies. Issues for further research include group modeling strategies, techniques, and group size, as well as automatic animation generation.
1. Introduction Given the high rate of change in technology and working environments, many organizations have evolved to the point where their structure, procedures, and technologies are no longer in keeping with the needs of the future [9]. This means that organizations need to study themselves to determine what should be changed. Their components must be reshaped and rearranged to form appropriately coupled new subsystems with greater simplicity and internal cohesiveness than the existing structures. (Information) systems need to be adapted or developed to meet the requirements of the new structures and processes. Recognition must also be given to relationships between the organization and its external environment. An example of an organization that found itself in the situation described above, concerns the Criminal Investigations Department (CID) of the Amsterdam Municipal Police Force. The CID has a difficult task combating organized crime in the capital of the Netherlands. They have limited means (time, money, personnel) to employ against too many suspect criminal organizations. In 1994, the CID management decided to reorganize the functional specialized department into a multi-disciplinary project
based organization in order to utilize their means more effectively and efficiently. However, it was unclear (1) how this transition had to take place, (2) how the project teams, so-called units, and their progress had to be managed, and (3) which technologies should be (developed and) employed to support the future organization. The situation at the CID provided us with an excellent opportunity to apply, evaluate, and further improve a design approach which focuses on the collaborative analysis and design of organizational structures, processes, and information technology. In earlier studies, we investigated various methods, techniques, and tools to support organizational change processes. These studies individually showed the potential of object oriented modeling, simulation and animation techniques, and Group Support Systems [8; 18; 19]. They also suggested that during an organizational change process, one of the main issues is one of interaction: 1. A lot of interaction is required between and among subject matter experts (SMEs) from the organization and analysts to exchange and discuss information and ideas with respect to the organization’s current situation and various design alternatives. 2. Interaction between the people involved in the process is time consuming and difficult because of differences in background, expertise, and personal objectives. In the study described in this paper, we addressed this issue of interaction by marrying two techniques that in separate situations had shown to facilitate interaction between and among members of an organization and analysts: GSS and animated simulation models. Hence, our research objective was to ‘explore the possibilities of the combined employment of GSS and animation techniques to facilitate organizational change in a collaborative fashion.’ By ‘collaborative fashion’ we mean that a group of SMEs from the organization is actively involved in the various modeling and design activities during the change process. For the study described in this paper, we used action research as our research approach. Action research aims to achieve the dual outcomes of action (or change) and research (understanding) [1]. An action researcher participates or
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
intervenes in the situation under investigation in order to apply a theory to practice and evaluate its worth [4]. The reason for employing action research is threefold [2]. First, we needed to study organizational change in its natural setting. It would be very hard to use a constructed setting, that is a valid representation of the complex reality in which our study is carried out. Second, we are interested in ‘how’ and ‘why’ questions. It is more important to explore the process of applying GSS and animation techniques than the resulting products. Third, there are few previous studies and there is a lack of elaborate theoretical understanding with respect to the combined application of GSS and animation techniques in the organizational change arena. In the remainder of the paper, we first give an overview of the design approach we employed and, in particular, the way in which GSS and animation techniques support the various design activities. Then, we describe the results of our study at the CID. The paper concludes with a summary of the most important findings and issues for further research.
2. A Collaborative Design Approach to Organizational Change Experience shows that organizational change is easier said than done. It is a complex activity. First, the goals of the people involved may conflict with those of the organization. Second, there are many possible courses of action to change an organization. Beforehand, it is not clear which course of action will be most efficient and effective. Third, there are usually many options or alternatives for change. As the effects of these change alternatives are often dangerous or impossible to test in reality, the effects of these changes are seldom known in advance. Finally, apart from `rationale', there may be other (conflicting) issues, such as political, cultural, and social issues, that influence the actual change process. It is, therefore, not surprising that a considerable number of approaches and supporting tools have been proposed to facilitate organizational change efforts.
2.1. Problem solving for organizational change As facilitating organizational change can be considered as solving an ill-structured problem, a problem solving strategy is needed that addresses the intricate attributes of such a process. Based on literature research and on experiences from a large number of field studies, we suggest employment of the problem solving process depicted in figure 1 [17]. The problem solving process consists of several design activities that can be carried out in an iterative fashion. First, the problem situation in an organization is conceptualized in order to structure the problem situation in such a way that the efforts for detailed, low-level data gathering for creating the empirical model can be focused and minimized. Next, a
descriptive empirical model is build that can be experimented with in order to analyze and diagnose the problem situation. For this purpose, the empirical model is, for example, implemented in a simulation language. The third activity is aimed at seeking solutions and capturing them in a number of prescriptive models of possible solutions. These models can be compared in order to study the (effects of the) alternatives in more detail. The actual choice is made in the next phase and consists of a combination of possible solutions or can leave the situation as it is. Finally, in order to actually solve the problem situation, the solution has to be implemented. Current organization
A. B. C. D. E.
Conceptual model
Empirical model and Problem Diagnosis
Conceptualize problem Create and validate empirical model, diagnose problem Construct alternative models and conduct experiments Choose most preferred solution Implement solution
Current organization
Model of the Chosen Solution
Models of Possible Solutions
Figure 1 Problem solving for organizational change
2.2. Collaboration in organizational change In practice, many people are usually involved in the various design activities described above [15; 16; 21]. First, there is a team of analysts that is responsible for building and analyzing models of current and future organizational processes. Second, there are people that actually execute the processes, the subject matter experts (SMEs). They have to provide the analysts with relevant information on their working situation. Finally, there are people, the decision makers, that supervise the change process in the organization. Presented with the results of the analysts, they decide which changes the organization is going to make. Frequent interaction between these parties is of vital importance for the quality of the design process and its outcome, see e.g. [22]. SMEs' opinions and perspectives regarding the functioning of the organization may change during the design process, thereby resulting in changes in organizational requirements with respect to processes and systems. Also, it is believed that SME participation enhances the acceptance of eventual changes in their work situation. Furthermore, organizational change processes are usually far too complex for a single person to understand all aspects, issues, and variables. The desire for a design process that is as much collaborative in nature as possible, presents us with a time/completeness paradox. On the one hand, collaborative design efforts need to be time effective. An organization
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
cannot take too long to build models to plan changes if its competitive and financial positions are deteriorating. However, traditional methods for (participative) modeling are often time consuming [15]. They require analysts to move from small group to small group, gathering information, defining terms, resolving differences, trying to make a big picture from all the small fragments. On the other hand, collaborative design efforts pursue completeness. It can be argued to involve as many SMEs as possible. This would generate more ideas with respect to an organization’s current and future situation. It would enable a more complete picture of an organizational situation. Also, broader participation promotes wider buy-in to the outcomes of a study. Finally, larger groups of SMEs are able to evaluate ideas more extensively. However, the broader the SME participation, the more time is needed to carry out the design process. Hence, the participative nature of organizational change efforts sets the requirement to use techniques, methods, and automated support that facilitate the collection, organization, and communication of various types of design information. This design information includes, but is not restricted to, lists of organizational requirements, problem statements, and (dynamic) models of processes and structures. We propose the application of two technologies that together meet the requirements mentioned above: Group Support Systems and animated simulation models. We address these technologies in the next sections.
2.3. Group Support Systems GSS represent information technology that aims to make group meetings and group decision making more productive. Over the years a great deal of research effort has been devoted to the development and application of GSS. As a result of field studies and a large number of experiments, a substantial body of knowledge has emerged about the effects of GSS on group processes and outcomes, especially with respect to GSS used in same time, same place settings. Compared to ‘traditional’ group settings, these types of GSS meetings appear to be more efficient and more effective, while the participants are more satisfied with the results and the process, see e.g. [7; 11]. Several attributes have been identified that appear to bring about the added value of GSS meetings over non-GSS meetings. These attributes include: • Anonymity. By being able to enter ideas, comments, and votes anonymously, silent or shy participants are more encouraged to participate. Group members are hindered from dominating by position or personality. Ideas appear to be judged on their merit, not on the personality of the person that submitted it. • Parallelism. By generating ideas and communicating them in parallel, participants get equal `air time'. This prevents production blocking so that participants can
spend more time on generating new ideas. Furthermore, working in parallel allows groups to generate more ideas. In this respect, it is noteworthy that the number of high quality ideas appears to be directly proportional to the total number of ideas [14]. • Group memory. During an electronic meeting, all ideas, comments, and votes are stored electronically. Hence, previous meeting results are readily available in a follow-up meeting. There is little time needed to produce meeting minutes. Moreover, there is a fundamental difference between the quality and completeness of electronic group memory and that of `traditional' minutes [5]. First, ‘traditional’ minutes are likely to be more subjective as they are written by an individual. The electronic group memory gives an objective history. Second, minutes not often describe participants' initial positions or interim postures. Group memory depicts the evolution of the group's position over time. Finally, group memory makes each comment during a meeting public, to which others can refer at a later stage. • Group size. Findings suggest that GSS have a greater positive impact on large groups (8 members or more) in terms of productivity and participant satisfaction, [6]. In other words, GSS seem to facilitate the effective and efficient participation of larger number of SMEs in a collaborative design process. In table 1, we identify the way in which GSS were used to support the design activities in the CID study. The basic activities of the design approach discussed in Section 2.1 are split into a number of sub-activities, see also [17]. For each subactivity, the parties involved are identified. In order to identify the type of group support we considered to be most appropriate, the nature of each subactivity was determined. Based on the meeting task taxonomy of Bostrom et al. [3], we differentiated between three meetings tasks: generate, organize, and evaluate. Non-meeting tasks are labeled `other'. Finally, we identify which GSS (GroupSystems V modules or TeamGraphics) we used for each subactivity in the CID study.
2.4. Animated simulation models During the process of problem solving described in Section 2.1, a number of models of organizational processes are constructed and analyzed. Modeling organizational processes is a complex activity. First, organizations can be modeled from different perspectives: the way a manager
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 1 GSS for collaborative design support ACTIVITY
Construct conceptual model • Gather conceptual data • Define problem boundaries • Identify relevant object classes • Define relevant object classes • Construct aspect models Construct empirical model • Formulate organizational requirements • Construct simulation model • Gather empirical data • Formulate problem statements • Relate problems to requirements • Validate static empirical model Diagnose problem situation • Formulate cause hypotheses • Investigate cause hypotheses • Formulate solution directions Construct models for change • Identify possible solutions • Select solution to elaborate • Conceptualize solution • Construct simulation model of solution • Evaluate solution Choose change alternative • Compare evaluation results • Determine alternative to implement Implement solution • Write policy document • Introduce changes • Perform post evaluation
ACTORS
NATURE
GSS
Both Both Analyst Analyst Both
Generate Generate & Organize Generate & Organize Generate & Organize Generate & Organize
Topic Commenter Topic Commenter Idea Organizer TeamGraphics TeamGraphics
Both Both Both Both Both Both
Generate Other Generate Generate & Organize Organize Generate & Evaluate
Idea Organizer Topic Commenter Group Outliner Group Outliner -
Both Both Both
Generate & Organize Generate & Organize Generate & Organize
Group Outliner Group Outliner Topic Commenter
Both Both Analyst Analyst Both
Generate & Organize Evaluate Generate & Organize Other Generate & Evaluate
Group Outliner Vote Topic Commenter
Both Both
Organize Evaluate
Topic Commenter Vote
Analyst Analyst -
Other Other Other
-
looks at a process is usually different from the way the one who executes it does. Second, an organizational process usually consists of a great many activities that can be modeled. Third, carrying out activities is often timedependent: many activities are closely adjusted to one another or are being performed in parallel with one another. Consequently, modeling organizational processes requires models which can reflect different abstraction levels, parallel activities and an explicit time dimension. In addition, for analysts, SMEs, and decision makers, models of current and future organizational processes are used as a vehicle of communication. Hence, it is imperative that these models are easy to communicate. This means, that a model has to closely resemble the mental model of the person it is communicated to [4]. Experience suggests that traditional modeling techniques, such as ERDs, DFDs, or IDEF0 models do not meet these requirements [21]: • Diagramming techniques may be able to model the dynamics of a situation, but the resulting diagrams are static in themselves. This makes it difficult to
thoroughly understand and analyze the dynamic characteristics of the modeled processes. • Diagramming techniques are often difficult to communicate to SMEs and decision makers. They usually lack the required training and expertise to build and understand the resulting models. Also, diagrams seldom resemble a problem situation visually. In this respect, simulation techniques appear to be more useful for developing models to analyze process dynamics [21]. However, a simulation model itself can still be considered to be nothing more than a black box hiding the dynamics of the problem situation. It is difficult to investigate the modeled processes by only going through large sets of input and output data. We propose that using animation techniques on top of simulation models has the potential of overcoming this problem. An animation model graphically portrays the dynamic behavior of an process by showing icons that visually resemble objects from the real world, move around a computer screen, changing shape, color etc. Because of their ‘cartoon-like’ behavior,
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
animation models have the potential to resemble closely the mental models of both problem owners and problem solvers. Animation models support the communication between and among analysts, SMEs, and decision makers in various ways [21]: • For an analyst, the use of animation models may decrease the development time of the simulation model because of the enhanced debugging possibilities. With animation it may be quicker to find and localize mistakes than by going through output of traces or using a debugger. • The fact that the visual process information in an animation model is less abstract and, therefore, easier to communicate to a group of SMEs, increases the SMEs' understanding of the organizational processes. As a result, SMEs may participate more actively in the design process. Their contribution may shift from gaining ‘understanding’ to providing ‘insight’. This may lead to model improvements. Also, the SMEs may appreciate and value their involvement more. • During the construction of a descriptive empirical model, using animation offers unique possibilities for face-validity tests. Structural mistakes in the model or deviant behavior can be pointed out by the SMEs. • In the solution seeking activity, animation may help to point out inadequacies of possible solutions which the SMEs or decision makers want to test. With animation these shortcomings may be presented such that the decision makers can understand and accept them. • Animation models help focus on particular aspects of a process under investigation, without having to go through large amounts of printed, numerical data. This is especially useful when presenting the results of a process analysis to a decision maker. In other words, an animation model can illustrate statistical results in an easy accessible manner. At present, animation is often a standard feature of simulation environments. A survey of simulation software showed that 80% of the simulation packages had animation capabilities [13]. In the CID study, we used the simulation language Arena, which is the successor of SIMAN/Cinema [12]. Arena is a general purpose simulation language, which supports both discrete event simulation and continuous simulation. It can be used in both production and service environments. Apart from modeling and portraying the dynamics of organizational processes, it offers facilities to design simulation experiments and to analyze numerical results.
3. The CID study The design approach described in the previous section, was applied to the CID case to support the change process that was taking place. Below we illustrate the application of the
approach with respect to the construction of the conceptual model, the empirical model, and the models for change. We first give some background information on the design sessions in which these models were built and evaluated.
3.1. Session background In the course of the study, five sessions were organized with respect to the models of the CID’s current and future processes, see table 2. The first two sessions lasted about three hours, the other sessions lasted about six hours each. In total, there were 40 participants, 27 of which were unique. The participants used GroupSystems V in each session. TeamGraphics was used to support the construction and evaluation of the conceptual model of the current situation. Arena was used for the same purpose with respect to the models for change. GSS was not a new phenomena for the majority of the participants. About 75 percent had extensive experience using GroupSystems. There were two analysts involved in the study. Each session took place in the Group Support Facility (GSF) at Delft University of Technology (figure 2). The GSF layout consists of a meeting room with a U-shaped table facing a public screen, flanked by a number of breakout rooms. The table consists of 15 slightly wedge-shaped segments with a sunk-in CRT. The public screen has been implemented using a beam projector placed overhead, powerful enough to generate a crisp image even in daylight. The GSF infrastructure serves as a platform for several GSS applications such as GroupSystems V, GroupSystems for Windows, and TeamGraphics.
Figure 2 The Group Support Facility in the study.
3.2. Conceptual model The conceptual model of the CID’s current situation was built in a number of steps following an incremental modeling strategy. First, we held a number of short
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 2 Background information on the model sessions during the study.
Session 1. Conceptual model prototype 2. Conceptual model prototype 3. Conceptual and empirical model 4. Models for change 5. Models for change
Participants 4 4 8 12 12
interviews with some key actors in the CID. In these interviews, information was gathered on the relevant processes in the CID. Second, during the first two sessions, we built first version (prototype) of the conceptual model in TeamGraphics together with a key CID employee. Additional information to build this model was collected using GroupSystems. Part of the prototype model is depicted in figure 3. The modeling technique used concerns ‘process modeling’ where, based on a set of object definitions, the interdependencies between the constituent activities of a process are depicted together with the actors that perform these activities [8]. Finally, in the third session, we presented the prototype of the model in TeamGraphics to a group of CID employees. During this presentation, we used TeamGraphics in ‘facilitation mode’ [23]. Participants could browse through the model. They did, however, not have privileges to make modifications themselves. For this purpose, we used a dedicated chauffeur, that immediately followed up on all participants’ requests.
GroupSystems x x x x x
2.
3.
4.
Tools used TeamGraphics x x x
Arena
x x
current situation. In addition, the CID management requested that it were to be used as a basis in other reorganization efforts in the CID. It was important for the CID actors involved to have a shared view about the situation and the problems that they face. In other words, their mental models of the problem situation needed to be in harmony. This is important to arrive at a diagnosis of the problem situation and a proposed solution that everybody agrees upon. We reasoned that by discussing and improving the prototype of the conceptual model as a group, the mental models of the group members would converge. The fact that all participants expressed that the resulting model was a valid representation of the CID’s processes, confirmed the fact that their mental models converged. We feel that the group modeling process was more efficient that a traditional process where a model is built during a iterative series of individual interviews with SMEs. The prototype of the conceptual model was constructed in six hours. The discussion and modification of this model took another three and a half hours. During this time, a great majority of the SMEs' suggestions to modify and expand the model had been processed. We expected the SMEs to have a tendency to overemphasize details with respect to their processes and hence invest a disproportionate amount of modeling efforts. However, analysis showed that the model size only increased by about 18 percent (from 101 elements to 119 elements) [18]. In other words, the modeling process remained manageable in terms of modeling focus and level of detail.
3.3. Empirical model Figure 3 Part of the conceptual model. Regarding the group modeling process supported by TeamGraphics, we make the following observations: 1. The active involvement of the CID employees in the modeling process effectuated the implicit verification of the resulting model. All participants agreed that the final model was a valid representation of the CID's
Like the conceptual model, the empirical model of the current situation was also constructed in a number of participative steps representing an incremental modeling strategy. First, we collected a number of problems ourselves that we identified during the modeling process of the conceptual model. Second, during the same session in which the conceptual model was evaluated and modified, the CID SMEs used GroupSystems to identify and
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
prioritize a number of organizational requirements (comparable to critical success factors). Finally, the group was invited to brainstorm about known and believed problems that hindered the CID from meeting its organizational requirements. To stimulate this brainstorming activity, we entered our initial list of problems to the outline before the SMEs could enter their ideas. For the brainstorming activity, we used GroupSystems V’s Group Outliner module in combination with TeamGraphics. Every SME had access to two adjoined terminals, one running TeamGraphics, the other one Group Outliner. The SMEs could use TeamGraphics to ‘walk through’ the conceptual model during the brainstorming exercise. In Group Outliner we outlined the hierarchical structure of the conceptual model’s process models. The SMEs were asked to enter the problems they perceived as comments to the most appropriate part of the outline. This implied that very specific problems were entered as comments to the task they related to, whereas the most general problem statements were entered at the top level of the outline. The outline representing the hierarchical structure of the conceptual model’s process models is depicted in figure 4.
reacted on problem statements immediately, fueling the discussion about some problems and enforcing clearer, more detailed problem descriptions. This saved the time needed to validate the analysts’ perception of the problems with each individual SME.
3.4. Models for change Based on the problem statements gathered during the previous activity, a number of potential solutions were identified by the analysts in cooperation with a number of key CID employees. These solutions entailed various structural modifications and extensions with respect to the way in which the main processes in the CID were carried out. The further design of these solution was carried out in a number of steps. First, each new or modified main process was conceptualized by the analysts. Second, these conceptual models for change were transformed into animated simulation models in Arena. These animation models portrayed the dynamic behavior of the actors in the envisioned situation in terms of the order and the interdependencies of the activities they carried out. An example of an animation model is shown in figure 5.
Figure 4 The hierarchical structure of the conceptual model in Group Outliner. Our own initial problem list consisted of 38 problems. The participants extended this list with another 71 problem statements. We found only a small overlap (9 problems) between the two sets of problems. In other words, the SMEs had a significant contribution to the analysis of the CID's current situation. The main difference between the two sets of problems related to their focus: strategy (analysts) vs. operational (SMEs). Clearly, these two focuses complemented each other well. In this sense, the SMEs did not question our perceptions, but accepted it as a basis for a more thorough assessment of the situation. Identifying the problems of CID's current situation in a group setting brought about a rich picture of the problem situation. The SMEs not only outperformed the analysts in terms of the number of problems generated. They also
figure 5 An example of an animation model depicting the proposed new processes. Third, each animated model was presented to two groups of SMEs (20 people in total) during the fourth and the fifth session. During each session, the proposed process was introduced and explained to the SMEs and a demonstration of the animation model was given. As soon as the model and the process it depicted was clear to the SMEs, GroupSystems V’ Topic Commenter module was used to discuss the proposed new or modified process. During this activity, the SMEs were encouraged to be as critical and creative as possible. The SMEs entered comments and held electronic discussions on five predefined discussion topics: ‘expected problems with the
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
proposed process’, ‘questions about the proposed process’, ‘alternative ways to overcome the problems addressed in this process proposal’, ‘ideas regarding a project management system’, and ‘remaining comments and ideas’. Finally, based on the SMEs’ feedback and the subsequent group discussions, the definitive design of the process was finalized immediately after the sessions. The CID management approved of the final design of the processes and initiated a number of activities required to achieve the implementation of the proposed changes. The decision to communicate the models for change to the SMEs as animation models was based on the assumption that the proposed changes would be easier to fully understand which would facilitate a critical evaluation. However, a previous study showed that SMEs may seem overwhelmed by animation models and become interested viewers rather than critical evaluators. This is referred to as the ‘Pacman effect’ [10]. We anticipated this risk by keeping the models as simple as possible and by consciously stimulating the SMEs to have a critical brainstorming attitude. During the introduction of the models, we indeed witnessed a short period in which the SMEs seemed to a bit awed by the models. However, as soon as they had the opportunity to react to the model and engage in an (electronic) discussion of the model, their passiveness made way to a critical and assertive attitude. This was not only illustrated by the tone of their comments and by the drive with which they reacted to each other’s comments, but also by the number of comments, ideas, and suggestions that were submitted. In total, the 20 SMEs participating generated 399 comments, nearly 60 pages of output, covering the five predefined discussion topics with respect to the proposed processes. Hence, we conclude that using animated models in a GSS environment did not seem to harm the SMEs’ active involvement in the design and evaluation process. However, we need to point out that the SMEs’ motivated participation also to some extend seemed to be related to the ‘perceived correctness’ of the animation models. During the sessions, we found that small logical inconsistencies or impossibilities triggered the SMEs to study the proposed process very carefully. In other words, they were stimulated by the ‘imperfectness’ of the models. With respect to the animation models themselves, we surveyed the SMEs whether they felt that the models sufficiently clarified the proposed processes. Their average response was 3.7 (standard deviation 0.9) on a five point scale, 5 between the most positive. In general, the SMEs found the animation models somewhat complex. They thought there were too many elements moving on the screen and too many interdependencies shown to comprehend the proposed situation easily. During the second session, we dealt with this problem by taking more time to explain the models and let the participants get used
to them. Still, compared to the ‘paper’ conceptual process models, the SMEs liked the animation models very much. As one of them remarked: “That animation model tells me more than ten pages of process models!”
3.5. Participants' reflections During and after the sessions, we handed out questionnaires and held personal interviews to investigate the SMEs’ attitude toward the group design process and the GSS technology used. Group design process On the whole, the group design sessions left a positive impression with the participants. The results of the questionnaire are presented in table 3. (In the original (Dutch) questionnaire, not all questions were formulated with a ‘positive’ orientation. For the sake of presentation, we transformed all questions in tables 3 and 4 into positive oriented questions). From the results in table 3 it follows that the SMEs were very satisfied with the sessions, see question 1. They felt comfortable with the way in which the sessions were structured and the issues that were addressed. Furthermore, they subscribed to the results of the session, see question 2, while at the same time they thought that the session helped them to gain new insights, see question 3. The participants felt a lot of consensus was reached, see question 4. This, we feel, is of critical importance to the success of participative design efforts. Notwithstanding the fact that during the ‘models for change’ sessions there were some energetic discussions of various change alternatives between individual actors, the participants obviously enjoyed working together during the sessions, see question 5. The composition of the group was evaluated lower than in the previous study: 3.8 against 4.2. From comments of the participants, this turned out to be due to the fact that during one of the sessions, a lot of people canceled. Consequently, the remaining participants felt that the group was incomplete. Finally, the participants' overall positive attitude toward the sessions is illustrated by their willingness to attend future sessions, see question 7. GSS technology used During the group sessions, we employed both TeamGraphics and GroupSystems V. As TeamGraphics was only used by a few people, we did not hand out
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
Table 3 Questionnaire results with respect to the group design sessions. Question 1 2 3 4 5 6 7
Are you satisfied with the session? Did the session results confirm your ideas on the session topic? Did the session help to gain new insights on the session topic? Has enough agreement been achieved among the participants? Did you enjoy the collaboration with the other group members? Do you approve of the composition of the group? Are you willing to participate in a follow-up session or activity?
Mean*
Std
4.4 3.9 3.9 4.3 4.4 3.8 4.7
0.5 0.9 0.6 0.8 0.6 1.1 0.4
* On a five point scale, 1 being very negative and 5 being very positive.
Table 4 Questionnaire results with respect to the use of GroupSystems V. Question 8 9 10 11 12 13
How do you perceive the quantity of the session results? Is a GroupSystems meeting more productive than a manual meeting? How do you perceive the quality of the session results? How well did you understand GroupSystems V? Did you find GroupSystems V user friendly? Did you find GroupSystems V interesting to use?
Mean*
Std
4.2 4.7 4.2 4.3 4.7 4.5
0.9 0.4 0.6 0.7 0.4 0.4
* On a five point scale, 1 being very negative and 5 being very positive.
quantitative questionnaires. SMEs’ suggest, however, that this technology has a high learning curve and takes some time to get used to it. As a result, we used a chauffeur to translate SMEs comments and suggestions directly into the model during part of the modeling exercise. The SMEs felt very comfortable with this working procedure. The application of GroupSystems V was evaluated using questionnaires. The SMEs involved gave the same positive feedback as in previous studies [19;20]. They liked the ability of GroupSystems to focus and structure a discussion and to work simultaneously and anonymously. The questionnaire results are presented in table 4. Regarding the use GroupSystems, the SMEs gave a lot of positive feedback. The SMEs felt that the quantity of session results was high, see question 8. The SMEs clearly attributed this high productivity to GroupSystems V, see question 9. One of the participants phrased it as follows: “It's more productive because there's less yackety-yack, little distraction and no endless discussions. You are forced to get to the point.” With respect to the quality of the sessions' results (question 10) the participants also displayed a positive impression. Like in previous studies, the SMEs expressed great enthusiasm about GroupSystems itself. They felt they understood the technology very well (question 11), and found it very user friendly and interesting to use (questions 12 and 13).
4. Conclusions and future research In this study, we explored the possibilities of the combined employment of GSS and animation technologies. The situation at the CID represented a realistic research
environment. The results of the study clearly indicate the potential of the technologies concerned during organizational change efforts: 1. Required design information was efficiently gathered and processed. 2. The SMEs involved got highly motivated to actively take part in the design effort. 3. The constructed models were effective in the sense that they triggered detailed collaborative evaluations and resulted in implementation proposals that were accepted by the SMEs involved. 4. The SMEs were very satisfied both with the collaborative design process and the process support. Based on our experiences, we argue that the GSS facilities and animation techniques are complimentary and that their application will be beneficial to the process of facilitating organizational change. However, we also experienced a number of issues that require further attention: 1. Keeping the momentum. It turned to be difficult at certain stages in the project to keep the momentum. As the efficiency of SME involvement increases and the amount of contributions increases as well, the analysts need more time to process all contributions into one meaningful problem analysis and design. However, because of their good experiences during the sessions, the SMEs’ expectations of the process are (too) high with respect to the momentum of the project. 2. Information management. During the various sessions with the SMEs, a lot of design information was collected. It sometimes proved to be difficult to keep this information organized and maintain relationships between pieces of information throughout the project.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
3. Construction of animation models. Transforming a conceptual process model into an animation model can be a time consuming activity. In the CID study, three man weeks were spent building the animation models. 4. Flexible animation models. The animation environment used was not flexible enough to build and modify animation models during sessions. Also, it would have been nice if the SMEs were able to manipulate the animation models themselves. In other words, we felt the need for a ‘group animation model’. 5. Group modeling support. TeamGraphics appears to be a rather complex group modeling tool for SMEs with little drawing tool experience to effectively use it without continuous support from experts. At this moment experiences with design approaches to organizational change that use the technologies described in this paper are still rare. Research is needed to further investigate the issues discussed above. In addition, we found a number of other issues for further research to achieve a more detailed understanding of technology supported collaborative design. With respect to the group modeling technology these issues include the productivity of certain group modeling strategies, the optimal group size and facilitation mode for group modeling efforts, and the modeling techniques that are most suitable to be used by groups of non-modeling experts. With respect to the animation technologies these issues include the automatic transformation of conceptual process models into animation models, and a detailed investigation into the added value of animation model over process models on paper or non-animated simulation models. Although this study confirmed earlier field evidence that suggests that animation models facilitate and enhance the communication processes between people involved in an organizational change project [21], further evidence is needed to determine the exact nature of these benefits. Important questions that remain in this respect concern the levels of detail and complexity of an animation model.
References [1] ARGYRIS, C., AND D.A. SCHÖN, Participatory Action Research and Action Science Compared - A Commentary, American Behavioral Scientist, Vol. 32, No. 5, pp. 612-623, 1989. [2] BENBASAT, I., D.K. GOLDSTEIN, AND M. MEAD, The case research strategy in studies of information systems, MIS Quarterly, Vol. 11, No. 3, pp. 369-386, 1987. [3] BOSTROM, R.P., R. ANSON, AND V.K. CLAWSON, Group Facilitation and Group Support Systems, in: Jessup, L.M., J.S. Valacich (eds), Group Support Systems - New Perspectives, Macmillan Publishing Company, New York, 1993. [4] CHECKLAND, P.B., Systems Thinking, Systems Practice, John Wiley & Sons, Chichester, 1981. [5] CULNAN, M.J., AND M.L. MARKUS, Information Technologies, in: Jablin, F.M., L.L. Putnam, K.H. Roberts, and L.W. Porter (eds).,
Handbook of Organizational Communication - An Interdisciplinary Perspective, pp. 420-443, Sage Publications, London, 1987. [6] DENNIS, A.R., Electronic Support for Large Groups, Journal of Organizational Computing, Vol. 4, No. 2, pp. 177-197, 1994. [7] DENNIS, A.R., AND R.B. GALLUPE, A History of Group Support Systems Empirical Research: Lessons Learned and Future Directions, in: Jessup, L.M., J.S. Valacich (eds), Group Support Systems - New Perspectives, Macmillan Publishing Company, New York, 1993. [8] EIJCK, D.T.T. VAN, AND G.J. DE VREEDE, The Dynamics of Organizational Coordination, Proceedings of HICSS-28, IEEE Computer Society Press, 1995. [9] HUBER, G.P., The Nature and Design of Post-Industrial Organizations, Management Science, 30(8), pp. 928-951, 1984. [10] MEEL, J.W. VAN, Dynamics in Business Engineering, Doctoral dissertation, Delft University of Technology, The Netherlands, 1994. [11] NUNAMAKER, J.F. JR., A.R. DENNIS, J.S. VALACICH, D.R. VOGEL, AND J.F. GEORGE, Electronic Meeting Systems to Support Group Work, CACM, 34(7), pp. 40-61, 1991. [12] PEGDEN, C.D., R.E. SHANNON, AND R.P. SADOWSKI, Introduction to Simulation using SIMAN - 2nd Edition, McGrawHill, 1995. [13] SWAIN, J., Simulation Software Survey, OR/MS Today, pp. 81102, October 1991. [14] VALACICH, J.S., A.R. DENNIS, AND T. CONNOLLY, Idea Generation in Computer-Based Groups: A New Ending to an Old Story, Organizational Behavior and Human Decision Processes, Vol. 57, pp. 448-467, 1994. [15] VOGEL, D.R., N. KATIC, AND B. NEVSTRUJEV, Automated Support For Group Process Modeling, in: G.J. de Vreede, and H.G. Sol (eds), Proceedings of the INFORMS'95 Sessions on GDSS as Support for Designing Organizations and Information Systems, Delft University of Technology, 1995. [16] VOGEL, D.R., R. ORWIG, D. DEAN, J. LEE, AND C. ARTHUR, Reengineering with Enterprise Analyzer, Proceedings of HICSS-26, IEEE Computer Society Press, 1993. [17] VREEDE, G.J. DE, Facilitating Organizational Change, PhDthesis, Delft University of Technology, 1995. [18] VREEDE, G.J. DE, Participative Modeling for Understanding: Facilitating Organizational Change with GSS, Proceedings of HICSS-29, IEEE Computer Society Press, 1996. [19] VREEDE, G.J. DE, S.O. DEN HENGST, AND H.G. SOL, Facilitating User Involvement in Information System Design and Development with GDSS - The Organized Crime Case, in: Olfman, L. (ed.), Proceedings of the ACM SIGCPR Conference, Nashville Tennessee, 6-8 April, 1995. [20] VREEDE, G.J. DE, P.C. MULLER, Why some GSS meetings just don't work: Exploring critical success factors for electronic meetings, working paper, Delft University of Technology, 1996. [21] VREEDE, G.J. DE, AND A. VERBRAECK, Animating Organizational Processes, Simulation Practice and Theory, Vol. 4, pp. 245-263, 1996. [22] WANNINGER, L.A., AND G.W. DICKSON, GRIP - Group Requirement Identification Process, Proceedings of HICSS-25, IEEE Computer Society Press, 1992. [23] WATSON, R.T., M.B. ALEXANDER, C.E. POLLARD, AND R.P. BOSTROM, Perceptions of Facilitators of a Keypad-Based Group Support System, Journal of Organizational Computing, Vol. 4, No. 2, pp. 103-125, 1994.
Proceedings of The Thirtieth Annual Hawwaii International Conference on System Sciences ISBN 0-8186-7862-3/97 $17.00 © 1997 IEEE
1060-3425/97 $10.00 (c) 1997 IEEE
