Annex 5 - Cost benefit analysis

• Author: Cambridge Econometrics, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Eurocentre, ICF
• Pages: 386-422

Study supporting the evaluation of promoting social inclusion, combating poverty and a ny discrimination by

the European Social Fund (Thematic Objective 09)

386

ANNEX 5 – COST BENEFIT ANALYSIS

This annex provides information about the cost -benefit analysis that was carried out to

support th e assessm ent of Efficiency (see Section 2.2). A preliminary analysis at the

EU-level was carried out for the Interim Report.

The EU-level analysis commenced with a descriptive statistical analysis of the SF C2014

database for the purpose of performing a cost-benefit anal ysis. A cost-effectiveness

analysis is carried out for a selection of indicators. Then, the methodology for th e cost-

benefit analysis is presented. We apply this methodology on both the EU-level co st-

benefit analysis and on five case studies.

Processing of the extraction from SFC2014

The extraction from the SFC2014 was provided in the form of an excel file showin g, for

each Operational Programme (OP) and Investment Priority (IP) combination, the total

eligible cost amount, the total declared amount, Common Output Indicators (COIs) and

Common R esult Indicators (CRIs), broken down by cat egory of region. The following

steps were performed:

 The data was filtered by choosing the priority axes relative to TO9 and summing

by OP and IP (343 OP-IPs).

 The data was then split by type of operation includ ed in each OP-IP combination

where the information was available (for 10 OP-IPs this information was not

available). The same proportion of costs and output/results by OP -IP was

allocated to each type of operation. Finally, the proportion related to operations

of type 5 and 6 (64 OP-IPs) was taken out, which left 279 OP-IPs. For example,

if an OP-IP c ombination involves operations of type 1 and type 5, onl y 50% of

the costs and output/results would be taken into account.

 Then, data cleaning was performed. First, the data was cleaned by eliminating all

OP-IP combinations showing either: a) zero expenditure declared; b) a positive

expenditure declared but no results reported. Cost per participation by OP and IP

were calculated to identify and eliminate outliers. For IP9i, some small and high

numbers were observed. Therefore, the minimum value and the values above

the 95th percentile of the cost p er participation were taken out. For IP9ii, IP9iii

and IP9vi, the outliers were easy to detect using graphical analysis and they were

taken out. No out lier was present in IP9iv. Only two OPs remained for IP9v and

four for IP9vi, since all the others referred to types 5-6 operations or were filtered

out. The final dataset contains 168 OP-IPs combinati ons.

Descriptive statistical analysis

The reason for undertaking a descriptive statistical analysis of the ava ilable data is to

ascertain whether the data is robust (r eliable) enough to use f or the CBA, as well as

whether it provides suitable information for the CBA.

For the purpose of the analysis in this section, we construct aggregate measures of cost,

output and results at country and IP level. It is not possible to distinguish in the data

extracted from the SFC whether an operation was fully or partially implemented.

Study supporting the evaluation of promoting social inclusion, combating poverty and a ny discrimination by

the European Social Fund (Thematic Objective 09)

387

According to gui dance from the ESF555, Member States may opt to report data on

partially or fully implemented operations meaning that the time-lag between

participants entering an operation and the reporting of financial or participation and

results data can be significant. Moreover, an operation can be fully implemented even

if not all rel ated payments have yet been made b y beneficiaries or the corresponding

public contribution has not yet been paid556. Furthermore, participations are recorded in

output indicators in the year participants and entities enter operations 557.

Costs

The cost of ESF operations is understood to be the funds spent by programme

beneficiaries. The p ossible variables available from the SFC2014 to measure costs are

(1) Total eligible costs of operations sel ected for support until 31/12/2018 (Eligible

costs) and (2) Total eligible expenditure declared by beneficiaries to the managing

authorities by 31/12/2018 (Expenditure declared). Cost variable (2) is used in the CBA

at EU level (see Table 9).

In addition to the costs ag reed (Eligible cost) or spent (Expendi ture declared), ESF

operations have additional costs, such as administration costs (i.e. technical assistance)

which are usually 4 % of the total amount of the Funds allocated to operational

programmes in a Member State558, and indirect costs such as co sts of participation in

terms of i ncome forgone or lost production and earnings which are also not available.

These costs could therefore not be reflected in the cost-ben efit analysis.

For the CBA we are interested in what has actually been spent on operati ons, rather

than what could be spent ( total eligible cost), as we want to be ab le to calculate the

cost of each result (i.e. benefit ) achieved by the operation. Total eligible expenditure

declared therefore appears to be the most appropriate measure t o use, since it should

show how much each operation costs to be delivered and thus how much it costs to gain

the benefits to society. However, there is a question about whether the expenditure

declared data is timely enough, i n comparison to the COI and CRI data, to use in the

CBA. According to European Commission Guidance document559, there might be a time-

lag between the start of an operation and the reporting of financial or participation data.

555 European Comission. (2018). Monitoring and Evaluation of European Cohesion

Policy - ESF Guidance document. Employment, Social Affai rs & Inclusion. Brussels:

European Comission.

556 European Comission. (2018). Monitoring and Evaluation of Europ ean Cohesion

Policy - ESF Guidance document. Employment, Social Affai rs & Inclusion. Brussels:

European Comission.

557 European Comission. (2018). Monitoring and Evaluation of European Cohesion

Policy - ESF Guidance document. Employment, Social Affai rs & Inclusion. Brussels:

European Comission.

558 Article 119 (1) of Regulation (EU) No 1303/2013.

559 European Comission. (2018). Monitoring and Evaluation of European Cohesion

Policy - ESF Guidance document. Employment, Social Affai rs & Inclusion. Brussels:

European Comission.

Study supporting the evaluation of promoting social inclusion, combating poverty and a ny discrimination by

the European Social Fund (Thematic Objective 09)

388

In order to investigate this, we first take a look at how much of the eligible cost of

selected TO9 projects has been declared so far, starting with the dataset before the

procedure detail ed in Section 0 was applied, to g et a broad pi cture. At EU l evel, only

39% of the allocated amount to operations was spent (declared) by 31 December 2018.

Table 77 shows a significant variation among Member States (MS) in terms of Total

eligible costs of operations selected for support and Total eligible expenditure declared.

Luxembourg has the highest share of expenditure declared in the eligible costs ( 70%),

while Ireland has the lowest (only 5%). France has both the highest declared amount

and the highest number of participations under TO09 operations.

Table 77. Summary of financial and output data by country for ESF - TO9 purposes,

all data

Country

Total Eligible

Cost of selected

proj. (EUR)

Total

Expenditure

Declared

(EUR)

Share of

declared in

eligible

amount (%)

No of

participation

s

AT

185,362,212

69,724,171

38

67,062

BE

600,145,027

202,427,690

34

271,372

BG

289,792,352

173,785,315

60

146,691

CY

61,914,757

23,722,275

38

1,333

CZ

647,778,382

217,319,426

34

41,430

DE

3,401,481,120

1,685,240,379

50

492,739

DK

57,845,841

18,186,045

31

2,753

EE

126,073,052

48,501,461

38

19,900

ES

2,019,517,327

597,537,821

30

831,059

FI

133,968,822

77,945,862

58

31,144

FR

2,642,118,876

1,678,270,916

64

1,656,754

GR

596,162,584

336,024,458

56

169,676

HR

278,240,272

44,553,971

16

19,460

HU

1,246,055,642

203,669,812

16

104,257

IE

297,615,568

14,505,089

5

47,704

IT

1,803,516,285

528,930,055

29

703,623

LT

177,576,156

94,815,062

53

101,491

LU

7,751,600

5,444,134

70

1,405

LV

242,009,158

54,198,447

22

61,296

MT

40,493,708

10,900,822

27

3,138

NL

831,052,396

356,189,636

43

517,297

PL

1,761,962,918

492,650,756

28

323,428