Annex 1 - Synthesis of national evaluations and bibliography of literature reviewed

• Author: Cambridge Econometrics, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Eurocentre, ICF
• Pages: 142-205

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

142

ANNEX 1 – SYNTHESIS OF NATIONAL EVALUATIONS

AND BIBLIOGRAPHY OF LITERATURE REVIEWED

This section presents a synthesis of key findings from national evaluations of ESF TO9

operations. It presents the methodology to identify key findings from relevant national

ESF evaluations. It then presents the key findings by evaluation criterion.

Methodology

The sample of national evaluations was mainly drawn from an evaluation library of the

European Commission.285 The study team applied several search criteria to identify

published evaluations from the library that were relevant for the evaluation. The search

criteria included:

 European Social Fund;

 2014-2020 programming period; and

 Thematic Objective 9.

In total, 125 national evaluations were identified by the research team that met the

three criteria above. Each was reviewed more closely (by reviewing the country and OP

name) to determine if the evaluation was of an OP that fell either solely or partly under

Thematic Objective 9. This review was conducted in an Excel spreadsheet which included

several criteria to determine the relevance of each national evaluation. These criteria

included reference to an OP that planned for TO9 operations (the list of the 145 OPs is

presented in Annex 4) and/or social inclusion. Several evaluations that were not relevant

for TO9 were discarded, which reduced the number of evaluations to 114. These

evaluations included impact/result orientated evaluations, monitoring/ progress-

oriented evaluations and process/ implementation-oriented evaluations.

The reports for these 114 evaluations were distributed to the country experts who

reviewed them as part of their desk research that concluded by November 2019. The

country experts identified six additional evaluations through their research that had not

been published in the European Commission portal in July 2019. Country experts

reviewed the relevance of each evaluation from their country and, for those deemed

relevant, extracted and summarised key findings for their country-based analyses.

Specifically, country experts searched within each evaluation from their respective

country for information that could be used to answer the evaluation questions for

assessing the effectiveness, efficiency, relevance, coherence and added value of ESF

TO9. Findings from each relevant evaluation were shared with the study team, who then

developed a synthesis by evaluation criterion, drawing on the information provided by

the national experts.

In total, 40 national evaluations from 16 Member States were identified as being

relevant by the national experts for the overall study and are reflected in the synthesis.

The countries covered by these evaluations include: CY, CZ, DK, DE, EE, ES, FI, FR, IE,

IT, LT, LU, NL, PL, SI, UK.

285Commission website where national evaluations are uploaded:

https://ec.europa.eu/regional_policy/en/policy/evaluations/member-states/

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

143

Key findings from national evaluations by evaluation criterion

The findings of the identified national evaluations were summarised by five evaluation

criteria of the Better Regulation guidelines (effectiveness, efficiency, relevance,

coherence and EU added value).

Effectiveness

A number of evaluations from Member States pointed at a high level of effectiveness of

ESF TO9 related operations.

Several evaluations drew attention to result targets being met or being

exceeded, also noti ng positive soft outcomes. For example, the evaluation of the

Communities for Work (CfW) programme in OP West Wales & the Valley ESF 2014-2020

and OP East Wales ESF 2014-2020 (2014UK05SFOP001 and 2014UK05SFOP002

respectively) has shown positive results, including soft outcomes.286 The CfW was

designed to respond to the Welsh Government’s Tackling Poverty Action Plan. This

specifically targets long-term unemployed and economically inactive adults, young

people between 16 to 24 years old, who are not in employment, education or training

(NEETs). The Communities for Work (CfW) programme seeks to increase employability

of the target groups or guide them closer to employment.287 According to the evaluation,

the pr oportion of participants progressing into jobs exce eded the programme’s

benchmark level, across participants from nearly every target group by a considerable

margin, e.g. those aged 55 or over, those with no or low qualifications and individuals

with an ethnic minority background. The evaluation also point to positive effects upon

wider aspects of participants’ lives in terms of their sense of well-being and resilience in

addressing their barriers to employment.288 In additionз the evaluation point to ‘the time

available and flexibility of the support provided by CfW as a distinct advantage over

mainstream employability provision.289 Specifically, this refers to the time input of the

286 Welsh Government (2018). Evaluation of Communities for Work - Stage 3:

Emerging Outcomes and Impacts Report. [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-06/evaluation-

of-communities-for-work-stage-3-emerging-outcomes-and-impacts-report.pdf

[Accessed July 2019]

287 Welsh Government (2018). Evaluation of Communities for Work - Stage 3:

Emerging Outcomes and Impacts Report. [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-06/evaluation-

of-communities-for-work-stage-3-emerging-outcomes-and-impacts-report.pdf

[Accessed July 2019]

288 Welsh Government (2018). Evaluation of Communities for Work - Stage 3:

Emerging Outcomes and Impacts Report Summary. [online] Welsh Government.

Available at: https://gov.wales/sites/default/files/statistics-and-research/2019-

06/evaluation-of-communities-for-work-stage-3-emerging-outcomes-and-impacts-

report-summary.pdf [Accessed July 2019]

289 Welsh Government (2018). Evaluation of Communities for Work - Stage 3:

Emerging Outcomes and Impacts Report Summary. [online] Welsh Government.

Available at: https://gov.wales/sites/default/files/statistics-and-research/2019-

06/evaluation-of-communities-for-work-stage-3-emerging-outcomes-and-impacts-

report-summary.pdf [Accessed July 2019]

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

144

programme’s specialist youth and adult mentorsз employment advisers and social

workers, who all work in community settings with individual participants.

The ‘Parentsз Childcare and Employment’ гPa CEд is another project implemented in

Wales under OP West Wales & the Valleys ESF 2014-2020 and OP East Wales ESF 2014-

2020 (2014UK05SFOP001 and 2014UK05SFOP002 respectively), designed to tackle

poverty through sustainable employment.290 Specifically, the aim of PaCE is to provide

individual support to unemployed parents for whom lack of childcare has been identified

as a main barrier to employment and/or training.291 The evaluation suggests, the project

has substantially exceeded its target in terms of helping participants progress into work

helping 34% of all participants compared to the target of 20%.292 It identified a

correlation ‘between the engagements achieved per adviser in a month and the

outcomes per adviser delivered within a month’з reflecting the critical role of advisers in

engaging, motivating and supporting participants.293 PaCE also had wider effects on

participants, which included: (i) increased confidence; (ii) feeling a greater sense of

purpose and fulfilment and associated self-reported improved mental health; (iii)

reduction in their sense of isolation and improved social life; (iv) feeling that they are a

good role model for their children and able to instil in them a strong work ethic.294

The independent evaluation of Le Chéile, a project funded under ESF Programme for

Employability, Inclusion and Learning (PEIL) 2014-2020 (2014IE05M9OP001) has

shown a range of positive impacts for young people and parents who engage with the

service. Specifically, for young people engagement with Le Chéile mentoring programme

resulted in ‘reduction in alcohol and substance misuseз improved self-confidence,

290 Welsh Government, (2018). Evaluation of the Parents, Childcare and Employment

(PaCE) Project: process and outputs evaluation. [online] Welsh Government.

Available at: https://gov.wales/evaluation-parents-childcare-and-employment-

pace-project-process-and-outputs-evaluation [Accessed July 2019]

291 Welsh Government (2018). Evaluation of Parents, Childcare and Employment

(PaCE) Report Summary. [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-01/evaluation-of-

the-parents-childcare-and-employment-project-process-and-outputs-evaluation-

summary.pdf [Accessed July 2019]

292 Welsh Government (2018). Evaluation of Parents, Childcare and Employment

(PaCE). [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-01/evaluation-of-

the-parents-childcare-and-employment-project-process-and-outputs-evaluation.pdf

[Accessed July 2019]

293 Welsh Government (2018). Evaluation of Parents, Childcare and Employment

(PaCE). [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-01/evaluation-

of-the-parents-childcare-and-employment-project-process-and-outputs-

evaluation.pdf [Accessed July 2019]

294 Welsh Government (2018). Evaluation of Parents, Childcare and Employment

(PaCE). [online] Welsh Government. Available at:

https://gov.wales/sites/default/files/statistics-and-research/2019-01/evaluation-

of-the-parents-childcare-and-employment-project-process-and-outputs-

evaluation.pdf [Accessed July 2019]