Evaluation findings

• Author: Cambridge Econometrics, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Eurocentre, ICF
• Pages: 54-130

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

54

5. EVALUATION FINDINGS

This section presents findings by evaluation criterion in the form of specific answers for

each evaluation question and sub-question, with due reference to the evidence sources

and annexes where more detail is provided.

5.1. Question 1 – Effectiveness: How effective was the ESF in

achieving the objectives of Thematic Objective 9?

The assessment of effectiveness reviews the extent to which ESF support to social

inclusion contributed to the promotion of social inclusion, combatting poverty and

discrimination. The assessment considers the translation of national strategies and

policy contexts i nto TO9 operations, the changes that they generated , and the factors

that facilitated and hindered their implementation.

The assessment draws on all qualitative and quantitative information coll ected an d

analysed as part of the study including an extraction from the ESF monitoring data. The

assessment was supported by answers to five sub-evaluation questions, which are

presented below.

EQ 1.1 Effectiveness: To what extent did the financial implementation and

the achievement of the expected outputs progress according to the targets

set in the progr ammes? What were the main factors involved (delays in

implementation, ESF absorption…)?

Sub-question 1.1 reviews the progress made in the financial implementation of ESF

support for TO9.

The project selection rate was 71%, which is rather low considering the

advanced stage of the programming period, but in pace wit h other TOs.

Overall, there were delays in the financial implementation of ESF support to social

inclusion. These delays are reflected in the low project selection rate – only about 71%

of planned funds were committed to social inclusion operations by the end of December

201838. The absorption r ate was similarly low for other TOs suggesting that similar

challenges were encountered across the ESF39.

The project selection rate varied across the Member States and was especially low in

Italy (48%), Greece (50%) and Bulgaria (54%):

 In Italy (2014IT05SFOP001) the low project selection rate appears to have been

driven by changes to the national anti-poverty strategy40 and related legislation

that changed the means-testing rules and the thresholds, leading to an increase

38 The methodology and estimates by Member State can be found in Annex 4.

39 European Commission (2019). Synthesis Report of ESF 2018 A nnual

Implementation Reports (AIRs), Final report.

40 The national anti-poverty strategy changed names from 'Support for a ctive

inclusion’ - SIA in нлмсз to ‘Inclusion income’ -REI in нлму an d finally to ‘Citizen

income’ - RdC in 2019.

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

55

in the number of persons targeted41. Implementation was further delayed due to

the long negotiation periods and coordination efforts between the Ministry of

Labour and Social Policies and other administrations/actors at multiple

governance levels. Beneficiary organisations also required time to understand

and adapt to the new approaches.

 In Greece, significant delays in 13 regional OPs were attributed to difficulties with

internal management42.

 In Bulgaria, the delays in implementation were attributed to an insufficient level

of coordination and lack of clarity in the definition of responsibilities between the

institutions managing the ESF and ERDF funds that supported the operations.

The country-based anal yses identified several other reasons for delays in the

implementation of TO9 operations. These reasons included complexity in the

requirements of the ESF framework, which requires involvement of multiple actors and

coordination. Several examples include:

 Spain: The perceived complexity of requirements for the new ESF framework

(mainly related to audit procedures and data collection systems), delays in the

designation of intermediate bodies and overlaps with the previous programming

period all contributed to delays in implementation;

 Slovenia: Coordinating between the six ministries involved in the implementation

of TO9 was a challenge in particular with regards to the budget allocation (some

budgets had to be cut/scaled down) in the initial phase of implementation; and

 Portugal: Difficulties in coordinating the national government bodies and the MAs

in the operationalisation of types of operations that were already not part of the

implementation routineй The ESF’s administrative demands in terms of map ping

and procedures for contracting also contributed to delays.

A lack of expertise, capacity or relevant p revious experience among beneficiaries was

also noted (e.g. IT, LV, ES, PL). The country-based analyses (e.g. IT, PL, PT, LV, ES)

suggest that the introduction of administrative procedures to the programming period

contributed to a lower project selection rate. Lastly, it is important to highlight that

some Managing Authorities exhibit different patterns in how fund are all ocated – some

do it in phases and some allocate the full amounts early in the programmi ng period.

Over time these differences should be less critical.

Figure 4 below plots the project selection rate against another measure of the absorption

rate of TO9 funds, namely the share of planned funds that was declared as expenditures

by beneficiaries to the Man aging Authorities. A linear trend between the two m easures

evidently suggests a correlation: a Member State that has allocated a higher share of

planned funds is al so likely to have more funds declared as spent to th e Managing

41 Annual Implementation Report, 2018 from the National OP in Italy

(2014IT05SFOP001), pg 78.

42 There were planned funds but no allocated funds for one OP in Greece

(2014GR05M9OP001). The planned amount was 57.3 million, which was small

relative to the overall planned amount of 1.2 billion and is therefore unlikely to be

a significant driver of the overall project selection rate in Greece. Th e project

selection rate across OPs in Greece varied between 31% and 63%

Study supporting the evaluation of promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

56

Authority or Authorities43. Yet, there are some Member States where the two measures

are not aligned (e.g. IE, HU, HR, MT). The country-based analyses suggest that the

declaration of expenditures by beneficiaries to the Managing Authorities in these

Member State were delayed.

Figure 4. Project selection rate and the sha re of planned funds that were declared

expenditures for ESF support for social inclusion, by Member State

Source: SFC2014, based on AIR2018, data extracted on December 10, 2019; Note: A linear trend

line based on the data is presented in the figure. *The project selection rate is the share of planned

funds for TO9 operations that were allocated by Managing Authorities.

Overall, the gen eration of outputs for social inclusion operations has been

high, often exceeding the targets set for the end of the program ming period.

There is a regulatory obligation to set targets where relevant. Most Managing Authorities

set targets for programme-specific indicators rather common indicators, to reflect

progress made towards the specific objectives of their actions. In effect, targets were

set for almost all44 recorded values of programme-specific outputs. Output-level

achievement rates were esti mated as the share of recorded values for programme-

specific outputs against the targets set for the end of the programming period 45.

The overall output-level achievement rate by the end of December 2018 was estimated

to be 99%46. In some Member States, the number of outputs generated from TO9

operations surpassed the targets leading to achievement rates greater than 100%. This

43 The underlying figures can be obtained in Annex 4.

44 Targets were set for 1,023 of 1,025 recorded values of programme -specific outputs.

45 More information on the methodology and the findings can be found in Annex 4.

46 More information can be found in Annex 4.