change failure rates
Special Research Intelligence Brief
No.1 The Myth of the 70% Organisational Change Failure Rate The Evidence
By Dr. David Wilkinson
Research Intelligence Brief from The Oxford Review
Do 70% of change programmes really fail? What the evidence is really saying
Probably one of the most frequent statistics quoted in the organisational change literature and by consultants is that about 70% of change initiatives fail or fail to meet their intended objectives1. Coincidently and curiously the project management literature is peppered with exactly the same reported rate of project failure. So, rather than accepting this figure I decided to follow the quoted sources to find out where this figure of failure originally came from and to find out if it is still valid, given all the research and development that has occurred over the years.
Scare tactics Whilst such a dramatic figure suits the aims of many consultants, authors and researchers to focus the attention of readers and add importance to the results of their findings and work, there is a question as to whether the figure is valid, particularly given the improbably rounded number.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review
Where did the 70% figure come from? As the figure is still being quoted in contemporary literature2, I decided to follow the trail through the years of literature. After examining over twenty years of research literature I ended up at a paper written by the management and organisational development guru, John P. Kotter, published in the Harvard Business Review in 1996, called Leading Change: Why Transformation Efforts Fail. This paper is probably one of the most quoted papers in the change literature, not least because it outlines his famous model of organisational change. However it is often quoted as being part of the literature trail that researchers use as evidence of a high failure rate of change programmes. Before looking at the failure rate it is wise to examine the quality of the sources of such information. Academic journals The idea behind academic journals is that papers should only be published once a panel of other eminent and established academics and experts have assessed the research in terms of the validity and reliability of the research. In other words is it describing or measuring what it states it is and is that description or measurement reliable? This is a form of quality control to ensure that what gets published is as correct as humanly possible. Whilst there are criticisms of this system in that errors and false research can and do get through, it is largely considered to be a reasonable checking mechanism. Over and above this system is another system in which every academic journal is rated in terms of the level of impact their articles have. Without going into the technicalities of the system, the impact factor gives an indication of the importance a journal has in the scientific research community. The higher the impact-factor the more important, influential and reliable the journal and hence the articles published therein are considered to be. The Harvard Business Review is not considered to be a peer-reviewed journal and has an impact factor of 1.27. To put this into perspective the highest rated journal, Nature, has a current impact factor of 41.456. The Harvard Business Review is largely considered in academic terms as a collection of both research and non-research opinion and ideas for public consumption, rather than as a serious research journal.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review What Kotter actually said So having gained some context of this article in terms of its research validity and reliability, Kotter says the following: “Over the past decade, I have watched more than 100 companies try to remake themselves into significantly better competitors. They have included large organizations (Ford) and small ones (Landmark Communications), companies based in the United States (General Motors) and elsewhere (British Airways), corporations that were on their knees (Eastern Airlines), and companies that were earning good money (Bristol-Myers Squibb). These efforts have gone under many banners: total quality management, reengineering, rightsizing, restructuring, cultural change, and turn-around. But, in almost every case, the basic goal has been the same: to make fundamental changes in how business is conducted in order to help cope with a new, more challenging market environment. A few of these corporate change efforts have been very successful. A few have been utter failures. Most fall somewhere in between, with a distinct tilt toward the lower end of the scale.” 3 As you will notice there is no mention of a 70% failure rate. Additionally this is an anecdotal and somewhat less than scientific measurement of change failure. It is at best a personal observation based on the experience of the author. Editor’s introduction A clue comes from the editor’s introduction to Kotter’s paper. “Most major change initiatives—whether intended to boost quality, improve culture, or reverse a corporate death spiral—generate only lukewarm results. Many fail miserably.” So we have an interpretation by a non-academic editor of a non-scientific nonresearch paper based on anecdotal experience. The closest this article gets to giving a figure, in Kotter’s estimation, for failure of change programmes is when he says “Well over 50% of the companies I have watched fail in this first phase.” Here he is referring to stage one of his change model, ‘Creating a sense of urgency’. Later he refers to “When is the urgency rate high enough? From what I have seen, the answer is when about 75% of a company’s management is honestly convinced that business as usual is totally unacceptable.” But this is not a reference to failure rates of change programmes.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review Unscientific In 1993 Michael Hammer and Jim Champy published a book titled “Reengineering the Corporation: A Manifesto for Business Revolution” in which they offer “Sadly, we must report that despite the success stories described in previous chapters, many companies that begin reengineering don’t succeed at it...Our unscientific estimate is that as many as 50 percent to 70 percent of the organizations that undertake a reengineering effort do not achieve the dramatic results they intended.” 4 There are two issues here. Firstly as the authors acknowledge this is an unmeasured anecdotal estimate and secondly this is not failure, rather that the results do not achieve the ‘dramatic results intended’. Cracking the code of change The second most quoted source of evidence for 70% failure rates comes from a paper published in 2000 by Nitin Nohria and Michael Beer called Cracking The Code of Change. This was also published in the Harvard Business Review with the caveats mentioned above. Nohria and Beer were academic staff at the Harvard Business School at the time of publication. In the article they state “The brutal fact is that about 70% of all change initiatives fail.”5 However they fail to cite any evidence or source for this strangely rounded number. At least now we have the first explicit allusion to 70% as a failure rate, stated as a fact, even though there is no evidence supporting it. Kotter again We then find that Kotter6 now starts to talk about this figure in a book published in 2008 called A Sense of Urgency, in which he says “From years of study, I estimate today more than 70 per cent of needed change either fails to be launched, even though some people clearly see the need, fails to be completed even though some people exhaust themselves trying, or finishes over budget, late and with initial aspirations unmet”. There is no source or evidence presented for this estimate as it is just that, an estimate, and a personal estimate at that, but not of failure note. It is personal unsubstantiated estimate that includes change programmes not started, not completed or end up over budget or don’t meet all the outcomes expected. Nothing more.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review The management consultants Then in the same year on page one of a self-published white paper by a management consultancy company, Bain & Company, we find the following statement: “People have been writing about change management for decades and still the statistics haven’t improved. With each survey, 70 per cent of change initiatives still fail – and the world is getting more complicated.”7 The paper fails to directly reference any such surveys or sources of such surveys. Since this point on, the figure appears to have entered into the mythology of organisational change literature as a fact, including, sadly, the academic literature where many researchers have singularly failed to check their sources. No rigorous research No rigorous global survey of change success or failure has been undertaken to even provide such figures. Additionally, does failure to meet aspirations for a change programme mean that the change initiative has failed? This highlights a singularly difficult issue. Who decides on the success or otherwise of a change intervention and with what criteria? Simply aligning such a definition would, I suspect pose an almost insurmountable set of problems for researchers. Management consultants again I did however discover a non-academic article8 published in 1999 'The irrational side of change management' in McKinsey Quarterly that explicitly refers to the figure and supposedly to some ‘research’. In the article Atkin & Keller, two consultants at McKinsey say, “In 1996, John Kotter published Leading Change. Considered by many to be the seminal work in the field of change management, Kotter’s research revealed that only 30 percent of change programs succeed.” As we have seen above he didn’t say this and it wasn't research. They then go on to say "…in 2008, a McKinsey survey of 3,199 executives around the world found, as Kotter did, that only one transformation in three succeeds.” Whilst they didn’t reference the actual study the only one I found with that exact number of participants is the 2008 McKinsey Global Survey9 Results: 'Creating Global Transformations'.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review The report looks at results from a survey they conducted looking at the perceptions of how successful transformation (change) was compared to how well defined and ‘stretching’ the success criteria were for the change. Interestingly they removed all the answers which were “don’t know’ or ‘other’. These account for 504 respondents or 15.75% of the total. So out of the original survey of 3,199 respondents, the methods of which are not reported so we have no way of gauging the validity of the study anyway, 2695 respondents thought the following: Ignoring those somewhat limiting and confounding factors they report that 2470 respondents said their transformations were either; ‘somewhat successful’, ‘very successful’ or ‘extremely successful’ and 145 said the transformations were ‘not successful at all’. That gives a 5.87% failure rate. Not anywhere close to what Atkin & Keller claim. In 2016 a McKinsey Slideshare presentation appeared10 with the following graphic:
The slide uses the following reference: Scott Keller & Colin Price Performance and Health: An evidence based approach to transforming your organisation. 2010’ Apart from the fact that the citation fails to mention that this is an out of print McKinsey self-published book, I managed to track the book down. It makes no mention of this study what so ever. The closest it gets on page 26 is to show the above graphic with no commentary and there is no mention of a 2009 study of 2261 participants that is included in this slide.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review
A Lack of criteria for success As many organisations appear to fail completely to set firm and measurable success criteria when they embark on change programmes, assessing failure or success is not a question often asked nor measured in organisations, beyond obviously stark financial criteria. Rather, organisations tend to launch into such initiatives with somewhat vague goals similar to have something ‘better than what’s happening now’ or simply to upscale or downscale operations and/or costs to expand or contract operations in response to market conditions or some regulatory edict.
Proper research A study published in 2007 by researchers from the Department of Business and Economics at the University of Joensuu, in Finland11 found that the perception of success, for example of small business owners, tends to be articulated no more specifically than “making a living” and that “going beyond that is not often seen (to be) of great concern”. Success criteria for larger organizations on the other hand appears largely to be driven by simple financial criteria12 such as share price13, profitability and market value14, dividends15, sales volumes16 etc. In 2016, Professor Jeffrey Pfeffer from Stanford University published a paper in the Journal of Management Studies entitled ‘Why the Assholes are Winning: Money Trumps All’17, in which he notes that “only four companies made both Fortune’s most admired and their best places to work lists in 2015”. What Pfeffer is saying is that at the bottom line, admiration and criteria of organizational success boils down to “financial success in terms of stock price appreciation and wealth creation” and that largely the people in the organisation only matter in as far as they are able to deliver financial returns.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review A 2011 study18 in the Journal of Change Management by Mark Hughes at the University of Brighton UK looked at failure rates of organisational change programmes and found that “In conclusion, whilst the existence of a popular narrative of 70 per cent organizational-change failure is acknowledged, there is no valid and reliable empirical evidence to support such a narrative”. The figure of a 70% failure rate of organisational change programmes therefore is the stuff of myth and legend that has no evidential founding. It is a figure useful for management consultants and those who have a vested interest in such a figure. It appears these consultants are quite happy to engage in pseudo-research to back up the story to their own ends, however the evidence does not support the assertion.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review References Amis, J. M. and R. Aïssaoui (2013). "Readiness for change: An institutional perspective." Journal of Change Management 13(1): 69-95. 1
Kotter, J. P. (1996). Leading change, Harvard Business Press. Beer, M., & Nohria, N. (2000). "Cracking the code of change." Harvard business review 78(3): 133–141. 2
Amis, J. M. and R. Aïssaoui (2013). "Readiness for change: An institutional perspective." Journal of Change Management 13(1): 69-95. 3
Kotter, J. P. (1996). Leading change, Harvard Business Press. P200. 4 Hammer, M. C., J. (1993). Reengineering the Corporation: A Manifesto for Business Revolution. New York, Nicholas Brealey Publishing. P200 5 Beer, M., & Nohria, N. (2000). "Cracking the code of change." Harvard business review 78(3): 133–141. (Pp137) 6 Kotter, J. P. (2008). A sense of urgency, Harvard Business Press. 7
Senturia, T., et al. (2008). "Leading change management requires sticking to the PLOT." London: Bain & Company. 8
Atkin, C. & Keller. S. (2009) The irrational side of change management. Mckinsey Quarterly online: http://www.mckinsey.com/businessfunctions/organization/our-insights/the-irrational-side-of-change-management. Accessed on 2nd March 2016 9 Meany, M. & Pung, C (2008) Creating Global Transformations. 2008 McKinsey Global Survey Results. http://gsme.sharif.edu/~change/McKinsey%20Global%20Survey%20Results.p df. Accessed on 2nd march 2016. 10 http://www.slideshare.net/aipmm/70-26633757 11 Reijonen, H. and R. Komppula (2007). "Perception of success and its effect on small firm performance." Journal of Small Business and Enterprise Development 14(4): 689-701. 12
Pfeffer, J. (2016). "Why the assholes are winning: Money trumps all." Journal of Management Studies. 13
King, B. G. and S. A. Soule (2007). "Social movements as extra-institutional entrepreneurs: The effect of protests on stock price returns." Administrative Science Quarterly 52(3): 413-442. 14
Hayward, M. L. and D. C. Hambrick (1997). "Explaining the premiums paid for large acquisitions: Evidence of CEO hubris." Ibid.: 103-127.
www.Oxford-Review.com
Research Intelligence Brief from The Oxford Review 15 Pfeffer, J. (2016). "Why the assholes are winning: Money trumps all." Journal of Management Studies. 16
Mohr, J. and R. Spekman (1994). "Characteristics of partnership success: partnership attributes, communication behavior, and conflict resolution techniques." Strategic Management Journal 15(2): 135-152. 17
Pfeffer, J. (2016). "Why the assholes are winning: Money trumps all." Journal of Management Studies. 18
Hughes, M. (2011). "Do 70 per cent of all organizational change initiatives really fail?" Journal of Change Management 11(4): 451-464.
www.Oxford-Review.com
