Results of task 2

• Author: Naldini, Andrea; Pompili, Marco; Peruccacci, Eleonora
• Pages: 49-70

49

Section 4.Results of Task2

This section presents the findings and the evidence that have emerged from the

analyses carried out in Task 2. This t ask focu sed o n ap proa ches and processes use d by

MSs to im plement the evaluations requested by the current EU rules. 48 aid schem es

def inedin the 2014-2019 period and subjected to evaluation in 15 MSs were

investigated. An analysis of the evaluation plans of these aid schemes and a survey

distr ibut ed to the authorities responsible for their evaluations provided the direct

information sources;in addition, the nat ional expert s interviewed som e of the officials

involved in the evaluat ions and provided a general pict ure of t he evaluation pr ocesses.

4.1.The analy sis of t he eva lua tion pla ns

The an alysi s of the evaluat ion pl ans (EPs) no tif ied and ap proved bet wee n 2015 and t he

end of 2019 had the objective of examining how the national authorities outlined the

process of implementingthe aid scheme evaluation. In total, 48 EPs have been

reviewed, of w hich 45 hav e been implement ed, while the r emaini ng t hree wer e not16.

The minimum contents and structure of the EPs are set out in the Commission Staff

Working Document “Common methodology for State aidEva lua ti on” 17and a

supplementary information sheet for the notification of an evaluation plan18has to be

sub mit tedby Member States within the first 20 working days following the entry into

force of t he schem e.

Thus, the main elements of the EPs have been analysed against the prescriptions set

out in the Commission’s Staff Working Document, with the aim to identify the main

choices of the Member States in methodological terms (objectives, evaluation

questions, methods, data)and in relation to the timeline and the most relevant

milestones of the evaluation, the identification and selection of the evaluator, the

en visa ged ch oices in m ak ing t he re sult s ava ilab le and in e xpl oit in g t he m to de sign f ut ur e

policies.

I t is i mp ort ant t o p oin t out t hat r espe ctin g wha t is en vi saged in t he EPs is a f un dam ent al

element: in all the EPs ap proved, the Commission underlined that t he Member St ate’s

commitment to carrying out the evaluation according to the plansubmittedwas an

essent ial elem ent for app roval.

Table 15b elowsumm arises t he ma in cont ents of t he analysis of EPs.

16SA. 4 034 8 ( ES), SA. 4440 0 ( FR), SA. 462 85 ( LT).

17SW D(20 14) 179 fin al.

18https://ec.europa.eu/competition/state_aid/overview/transparency_and_evaluation.html.

50

Ta ble 15M ain elem en ts of th e a nalysis of th e Eva lua tion Plan s

Con ten t of EP s

Ma in the me s of an aly sis

Ob ject ive s of

th e a id sche me

−

Expe cted imp acts of t he a id sch eme and the pote nti al neg ativ e ef fects

Eva lua ti on

questions

−Whet her all thr ee ev aluat ion quest ions (con cernin g t he d irect eff ects, the

ind irect eff ects and the prop ort ional ity and appr opri atene ss of the aid schem e)

are env isaged and clear ly d escrib ed

Re sult

indicators

−

Selection of more common results indicators, both for the direct and indirect

eff ects, and th eir rela tion to th e sug gest ed i ndicat ors in Annex I I o f t he Com m on

me thod ology for Sta te a idevaluation

Methods

−

Whi ch qu anti tat ive met hods a re g oing tob e used mo re19

−

How and fo r w hic h p urp osesaret he q uali tat iv e m eth ods used, if t hey ar e ap plie d

Da ta coll ect ion

−

Sour ces of inf orm atio n and th eir conte nts

−

Whether specific techniques to collect data have been envisaged since the

app rov al of aid schem e

−

Accessi bilit y of dat a

−

Role of p rim ary data colle cted for the evalu atio n

Ti me line of th e

evaluation

−

Id ent ificat ion of t he t ime line and m ilest ones of t he evalu ation

−

Poten tial risk s ide ntif ied b y MSs in t he EPs a ffect ing the evalu atio n pr ocess

Th e b ody

con duc tin g t he

evaluation

−

If the evaluator s had alr eady been select ed from the beginning (when noti fying

th e EP) o r i n a late r ph ase

−

Whe ther th e eva luat or p arti cipat ed in th e pr epara tion of t he EP

−

Mecha nism to guar ant ee t he in depen dence of t he evalu ator

−

Whe ther th e ex perie nce an d sk ills of t he ev aluat or are indic ated

−

Fina ncial reso urces dedi cated to th e ev aluat ion

Pu blic ity of th e

evaluation

−

Descr ipt ion o f t he st rat egy t o m ake the find ings of t he e valua tion pub lic

−

Type and intensity of the involvement of stakeholders in the discussion of the

res ults of t he e valua tion

−

How the results will be used to review the State aidscheme or set up new

schemes

Sou rce: I smer i Eur opa

The evaluation plans have been analysed in qualitative terms and for each com ponent

ofthe EPs a classification system and coded information has been extracted, to m ake

the compar ison bet ween EPs st raig htfor war d. When possible, in for mat ion der ivi ng from

the plans has been complemented by evidence emerging from the questionnaires

sub mit ted under task 2.

In the following sections we present the most relevant evidence emerging from the

analysis of the EPs.

4.1.1.Distr ibution of t he EPs by count ry and type of scheme

48 ev aluation plans fr om 15 Member States20w ere reviewed in the analysis. Figure11

shows the countries which notified the highestnumber of EPs: Germany (8 plans),

France (9 plans), Poland (7 plans), United Kingdom (6 plans) and Italy (4 plans). In

m ost cases 21, th ese c ount ri es a lso h ave a n at iona l le gal obl igat ion t o ev alu ate Sta te aid,

19Previous analyses on a sample of EPs found that among the methods suggested by the European

Commission, the Differences-in-differences is the most frequently mentioned in the EPs. SeeHan s

Friederiszick, Ela Głowicka, Linda Gratz, Simone Lünenbürger, Andr eas Rose nfeld , 2 018, Ex Post Sta te a id

Evalu atio n in Envir onm enta l Aid , in Euro pean Stat e ai dLaw Quart erly , v olum e 17 (4 ), Page s 509 – 524.

20Austria, Czech Republic, Germany, Greece, Spa in, Finland, France, Hungary, Ireland, Italy, Lithuanian,

Poland, Portugal, Sweden, Unite d Kin gdom .

21Ge rma ny, Franceand Unit ed Kingdom.