Annex 4 - Descriptive analysis of monitoring data

• Author: Cambridge Econometrics, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Eurocentre, ICF
• Pages: 316-384

Study supporting the 2020 evaluation of promoting social inclusion, com bating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

316

ANNEX 4 - DESCRIPTIVE ANALYSIS OF MONITORING DATA

Overview

This annex presents findings from a descriptive analysis of data extracted from the

SFC2014. The data received included "financial" and "physical" indicators with a cut-off

date for the end of the 2018 calendar year.

The analysis of the SFC extraction covers financial and physical indicators reported by

the Managing Authorities, including common and programme-specific indicators. The

findings generated from the analysis of the SFC2014 extend from 1 January 2014 to 31

December 2018. The quantitative data provides an indication of the financial resources

committed to T09, the outputs and results generated.

The research team encountered some inconsistencies in an initial data extraction

provided by the European Commission, which were reviewed against other sources

including the AIRS and the ESF Synthesis Report of the 2018 Annual Implementation

Reports, which includes an analysis of recorded entries into the SFC that were made by

the end of 2017536. For example, no data for outputs and results was available for

Estonia in the 11 July 2019 extraction from the SFC2014. Most of these inconsistencies

were resolved with the subsequent extraction, which was received on 10 December

2019. The analysis of the data from this more recent extraction from SFC2014 is

reflected in the Final Report.

TO9 operations

The research team in consultation with the European Commission identified a total of

145 OPs that foresaw TO9 actions. Table 31 presents the breakdown of the number of

OPs by Member State and also the number of operations planned for by the end of

December 2018. As stated in Regulation (EU) No 1303/20моŚ ‘operation’ means a

project, contract, action or group of projects selected by the managing authorities of

the programmes concerned, or under their responsibility, that contributes to the

objectives of a priority or priorities. The breakdown includes within its scope operations

carried out under Tτф in ‘Multiple Thematic τbjectives Priority Axes’й

Table 76 at the end of this Annex shows OPs under TO9 operations and number of

operations by OP.

TO9 operations are complex, highly diverse and delivered to a range of vulnerable

groups as well as entities. TO9 promotes social inclusion, combating poverty and any

discrimination through its six investment priorities (IPs):

i. Active inclusion, including with a view to promoting equal opportunities and

active participation, and improving employability;

ii. Socio-economic integration of marginalized communities such as the Roma;

iii. Combating all forms of discrimination and promoting equal opportunities;

536 European Commission (2018). Final ESF Synthesis Report of Annual

Implementation Reports 2017 submitted in 2018. This report finds that the

allocated costs overall were 15,697 million euros. Of this, 7,764 million euros were

in more developed regions, 5,536 million euros in less developed regions and

2,396 million euros in transition regions.

Study supporting the 2020 evaluation of promoting social inclusion, com bating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

317

iv. Enhancing access to affordable, sustainable and high-quality services,

including health care and social services of general interest;

v. Promoting social entrepreneurship and vocational integration in social

enterprises and the social and solidarity economy in order to facilitate access

to employment; and

vi. Community-led local development (CLLD) strategies.

Table 31 shows that the most OPs covered IP9i. The definition of IP9i is broad and

actions may more easily fit within it than under other more narrowly defined investment

priorities such as IP9v or IP9vi.537 The table also shows that operations under IP9i

addressed all types of operation 1-сз which implies that this IP was used as a ‘catch-all’

for a wide variety of actions. The definition of IP9i is broad in comparison with priorities

such as IP9v or IP9vi. The same applies, albeit to a lesser extent, to IP9ii and IP9iii.

IP9iv and IP9v included actions that fell under two types of operations only. A more

granular mapping of TO9 operations (by type of operation and target group) can be

found in Annex 2.

Table 31 shows that Type 1 operations were the most common (being reflected in 136

OPs) while Type 3 operations were the least common (being reflected in 25 OPs). Most

of the Type 1 operations fell under IP9i while most of Type 4 operations fell under IP9iv

and most of Type 5 operations fell under IP9v. The overall correspondence is in line with

the definitions of the IPs and the types of operations. Actions aimed at influencing

attitudes and systems (Type 6) were found in each IP, whilst most of the IPs included

operations focussing on access to services (Type 4).

Table 31. Number of TO9 operations by type of operation and IP

Type of

operation

Number of

OPs:

IP9i

IP9ii

IP9iii

IP9iv

IP9v

IP9vi

Type 1

Employment-

focused actions

136

129

16

8

2

12

Type 2 Enhance

basic skills

29

20

5

4

Type 3 Basic

school education

25

8

12

5

Type 4 Access to

services

97

52

21

20

66

1

2

Type 5 Social

entrepreneurship

75

19

50

9

Type 6 Actions

influencing

attitudes and

systems

79

37

15

13

32

20

7

537 ESF Transnational Platform, 2018. Social inclusion indicators for ESF investments -

areas for development in addressing the 20% social inclusion target

Study supporting the 2020 evaluation of promoting social inclusion, com bating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

318

Note: Project team assessment of the 145 OPs that planned for TO9 operations. One OP can include multiple types of

operations.

Analyses carried out

A descriptive analysis was carried out with respect to: ESF financial allocation to TO9,

common output indicators from TO9 operations, and common result indicators

generated from TO9 operations. The descriptive analysis was carried out for TO9

operations overall as well as by Member State, IP, and category of region. For more

information on the types of TO9 operations which were defined for the evaluation, please

see Annex 2.

The analysis constructed several measures relevant for the evaluation. These included

the following:

 Absorption rate (financial indicators) – This rate was defined in three ways (1)

the percentage of planned fund that were allocated, (2) the percentage of

planned funds that were declared by beneficiaries to Managing Authorities and

(3) the percentage of allocated funds that were declared by beneficiaries to

Managing Authorities. The methodology for the calculation is presented in

Section 2 of this Annex. Definition (1) is also known as the project selection rate

and was the primary definition of absorption rate used in the Final Report.

 Success rate (output and result indicators) – The share of participations that

generate a result. Success rates were estimated for each of the nine common

result indicators. The methodology for the calculation is presented in Section 4

of this Annex.

 Achievement rate (programme-specific indicators) – The level of recorded values

for specific output and specific results in relation to target values set for the end

of the programming period. The methodology for the calculation is presented in

Section 0 of this Annex.

The financial, output and result indicators were also broken down by two variables that

were not available in the SFC2014: (1) NUTS-2 level regions and (2) type of operation.

The methodology taken for each is presented below.

Breakdown of ESF monitoring data – type of operation

Section 3 of the Final Report presents a typology for TO9 operations and a mapping of

operations recorded in SFC2014 until 31 December 2018. More information about the

definition of the typology can be found in Annex 2.

The ESF monitoring data was reviewed alongside this mapping and the indicators were

broken down accordingly. TO9 operations mapped to one or more types of operations.

In cases where it mapped to one type, all of the recorded values (financial, output,

result) were allocated to that type of operation. In cases where the TO9 operation

mapped to more than one type, the recorded values (financial, output, result) were

broken down evenly. For example, if an operation mapped to three types, the planned

and allocated amounts were divided by three, where each share was allocated to one

type of operation. This methodology assumed an even breakdown of the recorded values

by type of operation. In practice this may not be the case – some types of operations

may have generated higher recorded values. Nevertheless, in the absence of more

detailed information, this provided the only practical way to go about this work.

NUTS-2 regions