Conclusions and suggestions

• Author: Naldini, Andrea; Pompili, Marco; Peruccacci, Eleonora
• Pages: 71-87

71

Sections 5.Conclusions and suggestions

This section is divided into two part s: in part one, the m ain fin dings of t he st ud y are

sum mar ised and the consequent issu es for t he evaluat ion process of aid schemes are

highligh ted; in par t t wo, possible changes and improvement s in t he evaluat ion pr ocess

are proposed.

5.1 .Sum mary of the m ain r esult s

5.1.1. Main fin dings of task 1

Task 1 aimed at investigat in g whether and how MSs evaluate aid schemes

independent ly of EU-specific r ules and t o wh at extent MSs are prepared t o follow EU

guidelines on aid schem e evaluations and what kind of gaps in terms of or ganisation,

evaluation cap acity and culture have to be filled.

In any hypothesis of extending or ren ewin g t he ev aluat ion r ules, the ad ministra tive

bu rden m ust be taken into accou nt. The study offers some im por tant find ings:

− In coun tries covering ar ound t he 70% of EU State aid expenditure obligat ions to

ev alua te alr eady ex ist and , conse quen tly , a g reat er nu mb er of Stat e aid sch emes

could be evaluated with out an excessiv e addit ional bur den;

− On th e ot he r h and , i f t he m ain pr iori ty is to in vol ve all th e co unt rie s in ev alu ati ons

of State aid, the potential burden could becom e significant for som e of them .

How eve r, the ev alu ati on u nit s pr ov ided by ESIF pro gramm es are wid espr ead and

relat ively dev eloped in many countries; involving t hese unit s or expanding them

in State aid m anag ement cou ld reduce th e adm in ist rative im pact of new

evaluat ions.

− Monetary costs of t he ev alu at ions are limited and repr esent a fract ion of the

budget for St ate aid.

Consequent ly , adm inistrative costs do not repr esent a significant obstacle to aid

schemes ev aluation; on t he contrary, they m ust to be considered as a factor that

reinforces evaluation act ivities.

A second impor tant aspect concerns the types of e valuat ion . I n practice, all t he MSs

implement some kind of evalu at ion of Stat e aid schemes. In countries w here im pact

ev aluat ion s ar e n ot carr ied out , ex-an te asses smen ts to ver ify com pli ance w ith EU r ules

and national budget constraint s are g enerally implement ed. All countr ies h ave set up a

m ore or le ss so phi stic ate d m on it ori ng syst em o f a id schem es to v eri fy th eir ex pen dit ur e,

their beneficiar ies and provide the EC with t he requ ired infor mation. Som e countries

use m anagement contr ol m eth ods to qu antify the financial and phy sical impact s of the

aid sch emes.

One possible way of expanding and consolidating evaluat ions wou ld be to include t he

experiences gained and the ex-post assessment s in the regular activ ities. This change

would be a logical int egrat ion w ith the ex ist ing nat ional systems for controlling State

aid , and would not requ ire excessive eff ort, becau se th e new assignments could be

cov ered by already ex isting m anagement st ruct ures and lim it inv estment s in resou rces

and skills.

A t hir d a spect , not to b e un der esti mat ed, con sist s in t he w i llin gn ess to e va lua te . The

previous an aly sis shows how in count ries with obligat ions to evaluate, ev aluation

72

capacit y h as been bu ilt up o ver m any years of p olit ical and adm inistr ative commitm ent

t o ass ess p ubli c po lici es. The lack of th is co mm it me nt is evid ent in som e oth er coun tr ies

wi tho ut ob liga tio ns, in wh ich the m anag eme nt of Sta te aid is or gan ised mo re to add ress

EC requests and com ply with EU rules than t o examine the efficiency and the

effectiveness of t he aid schem es. Even the ESI F ev aluations are often closer to an

exercise of form al com plian ce with EU r equir em ents than t o an act ual assessment of

th e effectiveness of the measures evaluated. Ther efore, they are often of poor qualit y

and do not really influence t he policy decision m echanism s. The EC DGs in charge of

ESIF are awar e of this problem and are committ ed t o improving t he qu ality of

evaluat ions39.

It is no easy task to improve and r einforce the com mit m ent to evaluat e. I n the 2014

State aid reform , the intr oduct ion of the EU obligation to evaluate som e lar ge aid

schemes w as counterbalanced by a g reat er flexib ility in t he EU rules on grant ing St ate

aid . There are oth er reasons for expanding t he coverage of State aid evaluations. For

instance, all nat ional m anagement syst ems of State aid are based on the principles of

financial st ability, generally through the con trol of the Ministr y of Finance; and mark et

com petition, generally t hr ough the control of t he Authority for Compet it ion, but the

cur ren t inst ru men ts used to moni tor an d as sess a id sc hem es ca nnot always ful ly satisfy

these pr inciples. In add ition, th e cur rent huge increases in State aid in respon se to t he

Covid-1 9 crisi s ne cessitate ac cura te eva luat ion in all cou nt rie s to un der sta nd the effect s

on the economy and m arket com pet ition, and t o better pr epare f or fut ure challen ges.

A fourth consider ation concer ns t he ESI F eva luation . This evaluation is sig nificant in

all MSs, as described above, and it s scope extend s to many State aid m easur es.

However, the analysis of these evaluations also pinpoint ed a d ifficulty in ident ifying

types and characteristics of t hese aid schemes clear ly. This raises problem s in t he

ident ificat ion of possible overlaps and also the necessity for bet ter focussing of ESI F

ev alua tio ns i f a n in te gra tion w ith ai d sch em e ev alu ati on h as t o be p ur sued . Th e an aly sis

also showed that t he quality of the ESIF count erfactual impact evaluations was not

par ticularly good. This may be due to several f act ors: a less careful selection of the

evaluators, fewer available resources for each evaluation report , a less rigor ous

app licat ion of evalu ation meth odologies, the broad scope an d the number of quest ions

to be answ ered in each ev aluation.

This in tegrat ion with ESI F evaluation seems potentially fr uitful for St ate aid evalu ation

and w ould allow cover age in t hose MSs which have hitherto not been involved in t his

evaluation no r been dr iven b y nation al obligat ion to evaluat e. How ever , t he i ntegration

requires a coordination of rules an d operational arrangem ents to star t the new

program m ing period with clear and harm onised guidelines. In particular, a potential

relaxation of th e cu rre nt obli gation t o car ry out t he i mpa ct e valu ati ons cont ain ed in th e

proposal of the 2021-2027 ESI F Regulation 40 should be counterbalan ced by an

increasing effort in coordinating and steering evaluation activit ies at EU and nat ional

lev el.

39 The 130 3/ 2013 EU Regulation for ESIF indicates that MSs have to en sure t he necessary data (ar t.54 )

and th e ade quat e eva luat ion c apacit y ( art .56 ). Th e SWD (20 19) 44 5 fi nal “ Synt hesi s of the find ings of t he

evaluat ions of ESIF Prog rammes” reco gnizes a discr ete qual ity of the ESIF eval uatio ns but i dentifies var y

shor tcom ing s to recov er.

40 Se e Regulation 375/ 2018 (ar ticles 39 and 40 ) and A.Naldin i, 2018, I mp rov ements and risks of the

proposed ev aluatio n of Cohesion Policy in t he 202 1–27 period: A p ersonal reflect ion to open a deb ate,

Evalu ati on; 20 18, Vol. 24( 4) 496 –504.