The great thing about any evaluation is that it can answer different types of questions about a programme, and with the Local Growth Fund (LGF) programme, there are certainly many to ask.
With £12 billion invested across England and no significant overarching government evaluation planned, there is a large weight of expectation on Local Enterprise Partnerships (LEPs) to answer questions about the economic impacts and the value for money that the programme has generated. The government has suggested the overall programme is expected to deliver a return on investment of £4.81 for every £1 of public money invested, but as the NAO highlights [1], this figure is unsubstantiated, and it clearly needs to be tested.
For LEP partners, other questions are also important. They might want to know whether the strategic focus of their suite of investments remains relevant and appropriate following the significant economic fallouts, firstly from Brexit, and more recently from COVID-19. They might want more detailed evidence of what works in economic growth locally and why to help shape future investment. They might want to know whether the processes for identifying, appraising, contracting, and monitoring projects were effective or whether they could be enhanced.
Across all of these questions, LEPs may also want to demonstrate that they have the capacity and expertise to deliver efficient and high-impact projects in their area, both to make a collective case for greater devolution of funding to the LEP level, and to make an individual case for greater funding allocation.
The LGF has been around now for seven years–a considerable lifespan for an economic development programme in these times–and it has its own unique story and idiosyncrasies. It started as a response to Heseltine's No Stone Unturned review, launching in 2013 as a new single pot for LEP investments. In the first round, investments were largely tied to the source funds of the single pot, and so the funded projects were mostly related to transport, new housing, and skills capital investments. Over rounds two and three the requirements gradually softened though, allowing for more workspace- and innovation-type investments, but with a continued focus on capital investment.
For evaluation, these characteristics of LGF bring an interesting set of challenges:
- When to evaluate: the longer timescales for delivery of large-scale capital projects and the subsequent time lapse before we can witness measurable outcomes suggests that impact evaluation and value-for-money assessments might best take place some years after allocations are made; however testing the continued relevance and appropriateness of investments is relevant now, and reviewing governance and delivery processes may also be better to do sooner, while those processes are fresher in stakeholders’ minds.
- Budget for evaluation: guidance from the Cities & Local Growth Unit indicates that counterfactual impact evaluation approaches should be the aspiration for an ideal LGF evaluation, however, this can be costly, requiring very particular expertise, methodologically challenging for many types of intervention and often still requires other complementary approaches (e.g., theory of change, process review, and beneficiary surveys) to provide a rounded evaluation that answers the range of questions LEPs will be interested in. Added to this, evaluation budgets may not have been built into original LGF budgets for many LEPs, as the main guidance on LGF evaluation followed some years after early allocations, meaning evaluation budgets may be tight.
- Diversity of methodologies: the approaches needed to test the impacts and value for money of a major new road, a new facility at a local college, and an extension to an innovation center are highly diverse. Different methodologies are required for each to consider aspects such as attribution, dead weight, and displacement of impacts, and the cost and effort of gathering that evidence need to be considered in proportion to the scale of investment in, and importance of, each type of investment locally.
- LGF as a programme or a set of projects: there is a question of whether the suite of LGF investments at the LEP level constitutes a cohesive programme of investments and should be evaluated as such, or whether they are a set of disparate projects that can only be evaluated individually (or indeed if they are somewhere between the two). This distinction may affect the way that impacts and lessons about investments can best be analyzed and presented.
A final and quite fundamental factor is considering exactly what the government guidance requires of LEPs. The National Local Growth Assurance Framework (MHCLG, 2019) sets out a requirement for all LEPs to evaluate their LGF funding. There is little nailed down beyond that, but the Cities & Local Growth Unit provided some useful pointers in its October 2019 guidance note, including a recognition that theory-based evaluation approaches can be very valuable where counterfactual approaches are not feasible; that quality evaluation of a smaller part of the programme is better than a cursory review that tries to cover everything and that innovative approaches such as only focusing on one investment theme, or commissioning evaluation alongside a neighboring LEP would be welcomed.
There is a lot to consider in all of this and the answer may be that an interim evaluation is useful or necessary at this stage, while a more substantial impact evaluation may be important a little further down the line when impacts can be better understood and assessed more robustly.
Ultimately, the scope and focus for an LGF evaluation needs to be based on a pragmatic but well-considered approach to:
- the questions that LEP partners are most keen to answer about their LGF investments
- the importance placed on using LGF evaluation evidence to ensure the LEP is well placed to secure and deliver future devolved funding
- the budget available to achieve this.
Hatch has been supporting LEPs and local partners with LGF bid writing, appraisal, and evaluation since the outset of the scheme, and we have worked with more than a third of all LEPs on this since 2013. We are experts in process and impact evaluation of economic development schemes (including currently leading the evaluation of the £3.3 billion English ERDF programme), and our team is at the forefront of economic research and strategy on COVID-19 and Brexit response plans for many local areas across England.
If you would like to discuss your plans for LGF evaluation work and further explore how you can get the greatest value from this work, please do not hesitate to contact Stuart Merali-Younger or Neil Evans.
Footnote 1: Local Enterprise Partnerships: An Update on progress, May 2019.