Integrating learning effects in management of technology projects.

• Author: Plaza, Malgorzata
• Position: Report

1. INTRODUCTION

   Software implementation projects are challenging to manage. According to studies no more than 25% technology related projects are completed successfully (Barry et al. 2002; Smith and Keil 2003), 18% of projects are prematurely canceled, and 53% of them exceed their cost, schedule, and scope constraints (Marshall 2006). Difficulties with implementation of technology projects are especially prevalent with more complex systems such as Enterprise Resource Planning (ERP) (Gray and Larson 2000; Al-Mashari et al. 2003). The failures are usually blamed on management's underestimation of the time needed to configure and test the system, on the complexity of the project and the inability of the employees to conquer the steep learning curve (Stedman 1999). The considerable problems of learning and implementing new ERP systems are well documented in the literature (Lee and Lee 2000; Motwani et al. 2002; Tchokogue et al. 2005).

   The currently deployed project management systems consider performance of the project team as constant and do not account for learning effects, which causes issues with both planning and control of the project. In this paper we offer a methodology, which improves management of projects strongly affected by learning. This methodology is an extension to the Earned Value Method (EVM), which forecasts project duration from the Schedule Performance Index (SPI). The extended EVM, also utilizes SPI, but in order to account for the impact of learning effects, the forecasts are further refined by the application of Performance Correction Index (PCI).

   This paper expands on the research of (Vandevoorde and Vanhoucke 2006), where the necessity to integrate the learning effects into EVM was extensively discussed, and EVM based model developed from an exponential progress curve (Plaza 2008). The previous model was developed for project control and was based on the assumption that the team begins a project with very low (approximated as zero) performance. This study expands the previous model by allowing a non-zero initial performance. It also extends the model into planning phases of a project and integrates calculations of an initial performance level. The key questions addressed in this paper are: (1) How can managers mitigate the risks of learning and implementation failures by more accurately forecasting project duration, and (2) How does learning impact execution of Information Systems (IS) project.

   The paper is organized as follows: The next section discussed the differences between technology and other projects not impacted by learning effects. In Section 3 the new method, which addresses the inaccuracies caused by learning effects on project management, is introduced. The method is illustrated with an example based on the real case study in Section 4, in which the implication for practice is also discussed.

2. WHY ARE TECHNOLOGY PROJECTS DIFFICULT TO MANAGE

   An Information Systems (IS) project affects the whole company and must be conducted by a cross-disciplinary team composed of company's employees and external consultants. During the project's initial stages team goes through a great deal of knowledge transfer (Boyer 2001; Baccarini et al. 2004). Although IS projects involve not as many activities as other industrial projects (Markus et al. 2000; Parr and Shanks 2000), management usually lacks historical implementation data. As a result, activity duration for the IS project cannot be assessed as accurately (Nelson 1996; Scott and Vessey 2002). Other unpredictable factors such as team instability or unanticipated escalation of requirements can also influence IS project parameters and cause "scope creep" (Gray and Larson 2000; Umble et al. 2003; Wallace and Keil 2004).

   Performance of the IS project team cannot be predicted accurately before the project is launched (Barry et al. 2002; Depledge 2003; Wiegers 2003; Wallace and Keil 2004). In the course of a complex IS implementation internal resources learn the new system and external resources learn the company's business processes (Boyer 2001; Robey et al. 2002; Al-Mashari et al. 2003). The performance is affected by intensive training and knowledge-transfer, which causes issues when forecasting the project duration (Callaway 1999; Barry et al. 2002; Depledge 2003; Wiegers 2003; Wallace and Keil 2004).

   One-dimensional systems control a single parameter such as cost or time (Deng and Hung 1998; De Toni and Tonchia 2001) and do not provide necessary support for management of IS project. The Earned Value Method (EVM), however, is one of the most popular multi-dimensional systems and has been used successfully as a standard project control system since 1960 (Jaafari 1996; Fleming and Koppleman 2000; Raby 2000; Kerzner 2003; Kim et al. 2003; Moselhi et al. 2004; Cioffi 2006). Can it also be used to control IS project?

   Most applications of EVM are based on the assumption that project team performance is a constant function of time (Fleming and Koppleman 2000; Anbari 2003). The knowledge transfer and team integration affect team performance during project implementation (Compeau et al. 1995; Gallivan et al. 2005) in a way that it gradually increases (Karlsen and Gottschalk 2003). This causes difficulties when forecasting the project duration, and makes EVM ineffective when tracking project progress.

   Research reports various issues when learning effects are not included in EVM (Fleming and Koppleman 2000; Anbari 2003; Vandevoorde and Vanhoucke 2006). During IS projects the performance of the project team increases nonlinearly (Haines et al. 2000; Robey et al. 2002). (Vandevoorde and Vanhoucke 2006) demonstrated that the forecasts provided by various EVM applications differ significantly when there are nonlinear learning effects. In order to improve the accuracy of project duration forecasting, nonlinear change in performance must be merged with EVM (Amor 2002; Cioffi 2005).

   There are EVM based commercial products for supporting project management. For example, Decision Edge is a Microsoft certified partner who developed Earned Value Manager (DecisionEdge)--a system, which supports costs management. It can be added as an extension to Microsoft Office and offers advanced reporting options. Unfortunately neither of commercially available systems allows analyzing the impact of learning.

   (Marshall 2006) considered the effects of non-linear elements but failed to thoroughly address the typical nature of learning in cross-disciplinary teams. (Plaza 2008) developed a mathematical model based on the EVM that estimates project duration from a learning curve. The model was further refined by (Plaza and Turetken 2009). Neither of the models includes the initial performance levels and therefore they provide accurate forecasting only in cases where a team begins the project at a very low performance.

   In this paper we expand the model reported in the original studies by: (1) incorporating the calculation of initial performance into EVM (Section 3.1), (2) developing a duration forecasting formula, which includes an initial performance (Section 3.2), and (3) comparing the application areas and potential errors when traditional and extended EVM is used (Section 4).

3. EXTENDED EVM--MODIFIED APPROACH TO MANAGEMENT OF TECHNOLOGY PROJECTS

   Successful project management starts with planning, during which a baseline for project execution is established. It requires that team's performance is evaluated accurately, which in the case of software implementation, is usually done with the input from a software vendor. Extended EVM also requires that a progress curve is established as early as possible. Since a majority of software projects offer some level of initial training before the commencement of a project, a progress curve can be determined during that formal training.

   3.1. Impact of progress curve on Project Planning

   Progress curve is a quantitative direct measure of performance changes. It is frequently used to forecast productivity in various sectors, in which case it is called a learning curve (Yelle 1979; Teplitz 1991; Blancett 2002). Logistic curve (L-curve) is its functional form frequently used for technology implementations (Butler 1988; Teplitz 1991; Teplitz and Amor 1993; Jovanovic et al. 1995; Jackson 1998; Dardan et al. 2006). According to (Plaza et al. 2009) L-curve (1) would be the most accurate approximation of performance changes in situations where project team goes through a formal training before the commencement of the project.

   P(t) = [p.sub.0] (1 - [e.sup.-kt]) if P(0) = 0 (1)

   where:

   * [p.sub.0] is Performance Ceiling, which depicts the peak performance level of the fully trained project team.

   * [T.sub.0] is Planned Duration, which is the time required to complete the project if the team is fully productive, performing at [p.sub.0] all times.

   * k is a Progress Curve Coefficient.

   [FIGURE 1 OMITTED]

   We...