Jim Clifford
Jim Clifford [email protected] 07860 386 081
National Housing Federation: Audit Committees 2011 Maximising Benefit: assessing the social return on investment
www.cass.city.ac.uk
www.cfps.org.uk
• Jim Clifford is Head of Non-Profit Advisory Services at leading Third sector advisors and auditors, Baker Tilly. • Jim has authored a range of high-profile Social Impact studies, and been instrumental in developing methodologies t o make them more accessible • He is a Visiting Fellow at Cass Business School where he is undertaking research into evaluative protocols for transactional decision making (linking Social Impact with conventional valuation and brand valuation). • He is a director of the Centre for Public Scrutiny, an independent charity which supports scrutineers and the development of approaches to scrutiny in public sector and other bodies . • The views expressed in this paper are the author’s own, and do not imply any corporate position by Baker Tilly, Cass Business School or its academic staff, or of CfPS. 2
Maximising Benefit: assessing the social return on investment • What is social impact, and why measure it ? • How does SROI work ? • Applying it effectively: balancing validity and credibility with knowledge transfer
• Examples • Applying it in social housing • Some questions to consider 3
Social Impact, and Outcomes measurement...... • SROI is increasingly being recognised • ..........and better understood
• Government moves towards project-based, or outcomes-linked payment • “Outcomes-based Government” report from CSJ
• Increasing recognition of • the need for clear measurement and • the acceptance of this approach
4
Measuring outcomes Importance of financial measures: • to funders in Government • for prioritisation decisions • for delivery measures in contracts • to philanthropists • in competing for capital • ..and in public scrutiny.......have a look at www.cfps.org 5
Measuring outcomes • A view of your achievement.... • ....but whose view..... • ....and when and why ?
• Do you need: • • • • •
Past or future measures ? Qualitative or quantitative ? .....or financial ? To measure it all....or some of it ? To be measured, or to understand your measurement
• Only one tried and tested financial measurement model......... 6
SROI is being seen used in different ways..... Influence on funders
Alliance of SSCs and BIS Alana House and MoJ North Lanarkshire Leisure and Local Authority
Influence in restructuring the landscape
SSCs and the change in the White Paper in 2009 Foundation of feedback through UKCES on SfH and SfC for DoH consultation paper “Liberating the NHS: Developing the Healthcare Workforce” Somestic Adoption and Fostering
Influence on project managers seeking to increase impact (internal decisions)
Most of the studies done, for example Witney Children’s Centre
Influence on project partners
Various, inc SSCs, Princess Royal Trust for Carers, NLL
Influence in LA funders
Witney, NLL, and various others
Use as agents for change in policy influence networks
Adoption network around PACT’s SROI and BBC2 “A Home for Maisie” – see http://www.baaf.org.uk/node/3356
7
Where do outcomes fit ?
Inputs to
Activities
to
Outputs to
Primary
8
Outcomes
to
Impacts
Secondary
How does it work ?
Σ
Impact = Outcomes – (deadweight + alternative attribution + displacement)
Deadweight The outcome that would have happened anyway
Alternative attribution The outcome that arose as a result of other interventions – importance of recognising the work of others
Displacement The disadvantage or reduction in positive outcome, or social cost arising as a consequence
9
...a developing protocol.... • Action research methodology • Life course analysis • Financial proxy development • Looking more widely (with others) at impacts: • Primary • Secondary
10
Developing your theory of change..... •Identify the right perspective: whose view is it ?
Sound research draws the best from the evidential base ....
•In what terms will you define the change ?
and considers its validity in this context:
•Establish the change....from what to what ?
•External sources:
•Identify how that change comes about ...from which activities ? •Test your findings
• Similar studies • Related work • Relevant source material and statistics
•Internal sources: • Existing research • Opinion of managers and others involved • New research with service users and other stakeholders
...doing the numbers • Compare what will happen against what would have happened
• Typically six stages of logic: • 2 internal validated data • 2 external validated data • 2 reasonable, conservative assumptions
12
Action Research in practice • An iterative process, based on the knowledge of representatives of five carers’ centres
• Four working group meetings interspersed with one-to-one calls with participants • Allowing the answer to emerge from debating realistic case studies • Staying close to the data
13
Some detailed measurement – Example 1 ATSW Efficiency savings
Assumption Benefits (£m)
Saving per site (£) Total acute/independent theatres Proportion adopting ATSWs project in year 1 Proportion of benefit due to development partner Projected annual cost saving achieved post roll-out Duration of savings (years) Discount rate Annuity factor Present value of savings for year 1 roll-out
14
367,000 381 5.5% 7,690,485 35% 5.0 10.0 3.5% 8.3 41.6
Some detailed measurement – Example 2 Summary Table for PACT Adoption
NPV(£)
Additional capacity achieved per annum
17,135,903
Incremental gain on replacements for State approvals from reduced disruptions Increased eduational attainment Reduction in NEET population Displacement: Loss of tax revenue from fostering Total evaluated
1,351,233
Supporting data on numbers "Phone call list" of applicants
Incremental gain on replacements for State approvals from reduced disruptions
262,586
40
Less: initially declined as unlikely to place Less: declined because of lack of capacity or lost 2,832,987 during early stages of process
-2 -18
Adopters accepted and taken to approval Less: additional parents that would not have been -1,022,805 approved by Local Authorities through lack of capacity or otherwise Net additional adopters
Couples taking two children
50%
20
-17 3
£20,559,903
Equivalent multiple to get number of children placed
150%
4.5
5%
-0.225
Less: disruptions in PACT placement (assumed)
4 The evaluation is focussed on the completed placements from those that would otherwise have disrupted assuming State adoptions disrupt in
40% of cases, that is an incremental disruption rate of
35%
....and the quicker placement for the whole population 1.4 Incremental disruptions (care costs) assuming an age at placement of
4 years
assuming an age at disruption of
6 years Giving:
0 years of the
under 3
band of
0
2 years of the
Age 3-8
band of
62,707
4 years of the
Age 8-12
band of
264,144
6 years of the
Age 12-18
band of
623,458
Aggregate per child Less:
950,309
Incremental costs of placement supervision
0
Present Value per child
950,309
Cost for whole cohort of incremental disruptions
1,330,433
4 quicker placements than would be possible in State system
10 weeks
Additional LAC care costs age
15
Total for alternatively sourced placements
4
at giving:
5200 for each child 20,800 £1,351,233
Some detailed measurement – Example 3
16
Measurement in Housing Associations....some thoughts to start yours.... Finding the difference......... primary effect
Finding the difference..... secondary effect
Reducing chaos in lifestyle for parents, leading to a number of effects
Children are cared for and supported; they stay in school, achieve more and have a foundation for adult life that involves access to employment
Access to employment Parents have a better track of health and mental health Reduced need for third party support for the family
Reduction in crime
Managing debts, either generally or in relation to housing; reduced crime
Longer-term escalation of debt and need avoided; economic loss; mental health aspects
Access to employment: productivity
Demonstrating a work ethic to children, wider access to supportive networks; ability to self-support; mental health gains
Ability to find, and engage in a local community to which they can contribute
Children better supported; families more engaged; health and mental health effects
Supported or easy access to health services : better use at an early stage
Ability to care for children and sustain employment Reduced crime, NEET-ism,
Access to home-based care provision: sustaining better health and lowering the risk of escalation
Ability to care for and support the children; violence and marital breakdown risk reduced
17
How we do it........
......then think........ • Effects on policing • Effects on the Fire Brigade • Effects on environment • Effects on local traders and employers (from negative to positive ?) • Healthy eating • And others............ 18
...the non-profit’s choice.......... • Be part of it, and its development • ...or wait until someone else requires it of you......
19
Questions and next steps.......... • How have you evaluated and presented outcomes to date • What do you need, and what do those working with you need ? • Now or later ? What will happen if we delay ? • How, and with whom, can the results be used ? • Should you club with, lead, or reference another organisation ? • Which projects (units of research) ? • How best to make it happen - practically, and for credibility?
20
National Housing Federation: Audit Committees 2011 Maximising Benefit: assessing the social return on investment
www.cass.city.ac.uk
www.cfps.org.uk
• Jim Clifford is Head of Non-Profit Advisory Services at leading Third sector advisors and auditors, Baker Tilly. • Jim has authored a range of high-profile Social Impact studies, and been instrumental in developing methodologies t o make them more accessible • He is a Visiting Fellow at Cass Business School where he is undertaking research into evaluative protocols for transactional decision making (linking Social Impact with conventional valuation and brand valuation). • He is a director of the Centre for Public Scrutiny, an independent charity which supports scrutineers and the development of approaches to scrutiny in public sector and other bodies . • The views expressed in this paper are the author’s own, and do not imply any corporate position by Baker Tilly, Cass Business School or its academic staff, or of CfPS. 2
Maximising Benefit: assessing the social return on investment • What is social impact, and why measure it ? • How does SROI work ? • Applying it effectively: balancing validity and credibility with knowledge transfer
• Examples • Applying it in social housing • Some questions to consider 3
Social Impact, and Outcomes measurement...... • SROI is increasingly being recognised • ..........and better understood
• Government moves towards project-based, or outcomes-linked payment • “Outcomes-based Government” report from CSJ
• Increasing recognition of • the need for clear measurement and • the acceptance of this approach
4
Measuring outcomes Importance of financial measures: • to funders in Government • for prioritisation decisions • for delivery measures in contracts • to philanthropists • in competing for capital • ..and in public scrutiny.......have a look at www.cfps.org 5
Measuring outcomes • A view of your achievement.... • ....but whose view..... • ....and when and why ?
• Do you need: • • • • •
Past or future measures ? Qualitative or quantitative ? .....or financial ? To measure it all....or some of it ? To be measured, or to understand your measurement
• Only one tried and tested financial measurement model......... 6
SROI is being seen used in different ways..... Influence on funders
Alliance of SSCs and BIS Alana House and MoJ North Lanarkshire Leisure and Local Authority
Influence in restructuring the landscape
SSCs and the change in the White Paper in 2009 Foundation of feedback through UKCES on SfH and SfC for DoH consultation paper “Liberating the NHS: Developing the Healthcare Workforce” Somestic Adoption and Fostering
Influence on project managers seeking to increase impact (internal decisions)
Most of the studies done, for example Witney Children’s Centre
Influence on project partners
Various, inc SSCs, Princess Royal Trust for Carers, NLL
Influence in LA funders
Witney, NLL, and various others
Use as agents for change in policy influence networks
Adoption network around PACT’s SROI and BBC2 “A Home for Maisie” – see http://www.baaf.org.uk/node/3356
7
Where do outcomes fit ?
Inputs to
Activities
to
Outputs to
Primary
8
Outcomes
to
Impacts
Secondary
How does it work ?
Σ
Impact = Outcomes – (deadweight + alternative attribution + displacement)
Deadweight The outcome that would have happened anyway
Alternative attribution The outcome that arose as a result of other interventions – importance of recognising the work of others
Displacement The disadvantage or reduction in positive outcome, or social cost arising as a consequence
9
...a developing protocol.... • Action research methodology • Life course analysis • Financial proxy development • Looking more widely (with others) at impacts: • Primary • Secondary
10
Developing your theory of change..... •Identify the right perspective: whose view is it ?
Sound research draws the best from the evidential base ....
•In what terms will you define the change ?
and considers its validity in this context:
•Establish the change....from what to what ?
•External sources:
•Identify how that change comes about ...from which activities ? •Test your findings
• Similar studies • Related work • Relevant source material and statistics
•Internal sources: • Existing research • Opinion of managers and others involved • New research with service users and other stakeholders
...doing the numbers • Compare what will happen against what would have happened
• Typically six stages of logic: • 2 internal validated data • 2 external validated data • 2 reasonable, conservative assumptions
12
Action Research in practice • An iterative process, based on the knowledge of representatives of five carers’ centres
• Four working group meetings interspersed with one-to-one calls with participants • Allowing the answer to emerge from debating realistic case studies • Staying close to the data
13
Some detailed measurement – Example 1 ATSW Efficiency savings
Assumption Benefits (£m)
Saving per site (£) Total acute/independent theatres Proportion adopting ATSWs project in year 1 Proportion of benefit due to development partner Projected annual cost saving achieved post roll-out Duration of savings (years) Discount rate Annuity factor Present value of savings for year 1 roll-out
14
367,000 381 5.5% 7,690,485 35% 5.0 10.0 3.5% 8.3 41.6
Some detailed measurement – Example 2 Summary Table for PACT Adoption
NPV(£)
Additional capacity achieved per annum
17,135,903
Incremental gain on replacements for State approvals from reduced disruptions Increased eduational attainment Reduction in NEET population Displacement: Loss of tax revenue from fostering Total evaluated
1,351,233
Supporting data on numbers "Phone call list" of applicants
Incremental gain on replacements for State approvals from reduced disruptions
262,586
40
Less: initially declined as unlikely to place Less: declined because of lack of capacity or lost 2,832,987 during early stages of process
-2 -18
Adopters accepted and taken to approval Less: additional parents that would not have been -1,022,805 approved by Local Authorities through lack of capacity or otherwise Net additional adopters
Couples taking two children
50%
20
-17 3
£20,559,903
Equivalent multiple to get number of children placed
150%
4.5
5%
-0.225
Less: disruptions in PACT placement (assumed)
4 The evaluation is focussed on the completed placements from those that would otherwise have disrupted assuming State adoptions disrupt in
40% of cases, that is an incremental disruption rate of
35%
....and the quicker placement for the whole population 1.4 Incremental disruptions (care costs) assuming an age at placement of
4 years
assuming an age at disruption of
6 years Giving:
0 years of the
under 3
band of
0
2 years of the
Age 3-8
band of
62,707
4 years of the
Age 8-12
band of
264,144
6 years of the
Age 12-18
band of
623,458
Aggregate per child Less:
950,309
Incremental costs of placement supervision
0
Present Value per child
950,309
Cost for whole cohort of incremental disruptions
1,330,433
4 quicker placements than would be possible in State system
10 weeks
Additional LAC care costs age
15
Total for alternatively sourced placements
4
at giving:
5200 for each child 20,800 £1,351,233
Some detailed measurement – Example 3
16
Measurement in Housing Associations....some thoughts to start yours.... Finding the difference......... primary effect
Finding the difference..... secondary effect
Reducing chaos in lifestyle for parents, leading to a number of effects
Children are cared for and supported; they stay in school, achieve more and have a foundation for adult life that involves access to employment
Access to employment Parents have a better track of health and mental health Reduced need for third party support for the family
Reduction in crime
Managing debts, either generally or in relation to housing; reduced crime
Longer-term escalation of debt and need avoided; economic loss; mental health aspects
Access to employment: productivity
Demonstrating a work ethic to children, wider access to supportive networks; ability to self-support; mental health gains
Ability to find, and engage in a local community to which they can contribute
Children better supported; families more engaged; health and mental health effects
Supported or easy access to health services : better use at an early stage
Ability to care for children and sustain employment Reduced crime, NEET-ism,
Access to home-based care provision: sustaining better health and lowering the risk of escalation
Ability to care for and support the children; violence and marital breakdown risk reduced
17
How we do it........
......then think........ • Effects on policing • Effects on the Fire Brigade • Effects on environment • Effects on local traders and employers (from negative to positive ?) • Healthy eating • And others............ 18
...the non-profit’s choice.......... • Be part of it, and its development • ...or wait until someone else requires it of you......
19
Questions and next steps.......... • How have you evaluated and presented outcomes to date • What do you need, and what do those working with you need ? • Now or later ? What will happen if we delay ? • How, and with whom, can the results be used ? • Should you club with, lead, or reference another organisation ? • Which projects (units of research) ? • How best to make it happen - practically, and for credibility?
20
