Conclusions and suggestions

• Author: Naldini, Andrea; Pompili, Marco; Peruccacci, Eleonora
• Pages: 71-87

71

Sections 5.Conclusionsand suggestions

This section is divided into two parts: inpart one,the m ain findings of the study are

sum marised and the consequent issues for t he evaluat ion process of aid schemes are

highligh ted;in par t t wo, possible changes and improvement s in t he evaluat ion pr ocess

are proposed.

5.1.Sum maryof the main r esult s

5.1.1.Main fin dings of task 1

Task 1 aimed at investigating whether and how MSs evaluate aid schemes

independently of EU-specific rulesandto what extentMSs are prepared to follow EU

guidelines on aid scheme evaluations and what kind of gaps in terms of organisation,

evaluation cap acity and culture have to be filled.

In any hypothesis of extendingorren ewin g the evaluation rules,the ad ministra tive

burdenm ust be taken into accou nt. The study offers some im por tant find ings:

−In coun tries covering ar ound t he 70% of EU State aidexpenditure obligat ions to

ev alua te alr eady ex ist and , conse quen tly , a g reat er nu mb erof Stat e aidsch emes

could beevaluated with out an excessiv e addit ional bur den;

−On th e ot he r h and , i f t he m ain pr iori ty is to in vol ve all th e co unt rie s in ev alu ati ons

ofState aid, the potential burden could becomesignificant for som e of them.

How eve r, the ev alu ati on u nit s pr ov ided by ESIF pro gramm es are wid espr ead and

relat ively dev eloped in many countries; involving t hese unit s or expanding them

inState aidm anag ementcou ld reduce the administrative impact of new

evaluations.

−Monetary costs of the evaluations are limited and represent a fractionof the

budget for St ate aid.

Consequently, administrative costs do not represent a significant obstacle to aid

schemes evaluation;on the contrary, theym ustto be considered as a factor that

reinforces evaluation act ivities.

A second impor tant aspect concerns thetypes of e valuat ion . I n practice, all t he MSs

implement some kind of evaluation of State aidschemes. In countries where impact

ev aluat ion s ar e n ot carr ied out , ex-an te asses smen ts to ver ify com pli ance w ith EU r ules

and national budget constraint s are g enerally implement ed. All countr ies h ave set up a

m ore or le ss so phi stic ate d m on it ori ng syst em o f a id schem es to v eri fy th eir ex pen dit ur e,

theirbeneficiaries and provide the EC with the required information. Some countries

use m anagement contr ol m eth ods toqu antify the financial and phy sical impact s of the

aid sch emes.

One possible way of expanding and consolidatingevaluat ions would be to include the

experiences gainedand the ex-post assessment s in the regular activ ities. This change

would be a logical integration w ith the existing national systems for controlling State

aid, and would not require excessiveeffort, because the new assignmentscould be

cov ered by already ex isting m anagement st ruct ures andlim it inv estment s in resou rces

and skills.

A t hir d a spect , not to b e un der esti mat ed, con sist s in t he w i llin gn ess to e va lua te . The

previous analysis shows how in countries with obligations to evaluate,ev aluation

72

capacit y h as been bu ilt up o ver m any years of p olit ical andadm inistr ative commitm ent

t o ass ess p ubli c po lici es. The lack of th is co mm it me nt is evid ent in som e oth er coun tr ies

wi tho ut ob liga tio ns, in wh ich the m anag eme nt of Sta te aid is or gan ised mo re to add ress

EC requests and comply with EU rules than to examinethe efficiency and the

effectiveness of the aid schemes. Even the ESIF evaluations are often closer to an

exercise of formal compliance with EU requirements than to an actual assessment of

the effectiveness of the measures evaluated. Therefore, they are often of poor qualit y

and do not really influence the policy decision mechanisms. The EC DGs in charge of

ESIF are aware of this problem and are committed to improving the quality of

evaluations39.

It is no easy task to improve and reinforce the com mitment to evaluate. In the 2014

State aidreform, the introduction of the EU obligation to evaluate some large aid

schemes w as counterbalanced by a g reat er flexib ility in t he EU rules on grant ingSt ate

aid. There are other reasons for expanding t he coverage of State aidevaluations. For

instance, all nat ional m anagement syst ems of State aidare based on the principles of

financial st ability, generally through the con trol of the Ministr y of Finance; and mark et

competition, generally through the control of the Authority for Competition, but the

cur ren t inst ru men ts used to moni tor an d as sess a id sc hem es ca nnot always ful ly satisfy

these pr inciples. In add ition, th e cur rent huge increases in State aidin respon se to t he

Covid-1 9 crisi s ne cessitate ac cura te eva luat ion in all cou nt rie s to un der sta nd the effect s

onthe economy and m arket com pet ition, and t o better pr epare f or fut urechallenges.

A fourthconsider ation concer ns t he ESI F eva luation . This evaluation is significant in

all MSs, as described above, and its scope extend s to many State aidm easur es.

However, the analysis of theseevaluations also pinpoint ed a difficulty in identifying

types and characteristics of these aid schemesclearly. This raises problems in the

identification of possible overlaps and also the necessity for better focussingofESI F

ev alua tio ns i f a n in te gra tion w ith ai d sch em e ev alu ati on h as t o be p ur sued . Th e an aly sis

also showed that the quality of the ESIF counterfactual impact evaluations was not

particularly good. This may be due toseveral factors: a less careful selection of the

evaluators, feweravailable resources for each evaluation report, a less rigorous

app licat ion of evalu ation meth odologies, the broad scope an d thenumberofquest ions

to be answ eredin each ev aluation.

This in tegrat ion with ESI F evaluation seems potentially fr uitful for St ate aidevalu ation

and would allowcoverage in those MSs which havehithertonot been involved in this

evaluation no rbeendr iven b y nation al obligat ion to evaluat e. How ever , t he i ntegration

requires a coordination of rules and operational arrangements to startthe new

programming period with clear and harmonised guidelines. In particular, a potential

relaxationof th e cu rre nt obli gation t o car ry out t he i mpa ct e valu ati ons cont ain ed in th e

proposal of the 2021-2027 ESIF Regulation40should be counterbalanced by an

increasing effortin coordinating and steering evaluation activities at EU and national

level.

39 The 1303/ 2013 EU Regulation for ESIF indicates that MSs have to ensure the necessary data (ar t.54 )

and th e ade quat e eva luat ion c apacit y ( art .56 ). Th e SWD (20 19) 44 5 fi nal “ Synt hesi s of the find ings of t he

evaluat ions of ESIF Prog rammes” reco gnizes a discr ete qual ity of the ESIF eval uatio ns but i dentifies var y

shor tcom ing s to recov er.

40Se e Regulation 375/2018 (articles 39 and 40) and A.Naldini, 2018,Improvements and risks of the

proposed evaluation of Cohesion Policy in the 2021–27 period: A personal reflection to open a debate,

Evaluation;20 18, Vol. 24( 4) 496 –504.