Annex 1: key activities undertaken during the evaluation

• Author: CSES, Directorate-General for Health and Food Safety (European Commission), Maastricht University, PPMI
• Pages: 125-134

Study suppo rti ng t he eva luat ion of t he op erat ion of t he Consu m er s, Heal th,

Agr icult ure an d Food Execu tiv e Agen cy ( 201 4-2016)

125

ANN EXES

AN NEX 1 : KEY ACTI VI TI ES UN DERTAKEN DURI N G TH E EVALU ATI ON

This annex presents our det ailed app roach to t he key data coll ect ion and analysis

met hods that were appli ed in th e evalu ation. The met hods incl ude:

 extensiv e documentary r ev iew, desk r esearch and an alysis of business

pr ocesses;

 stat istical anal ysis o f adm inist rat ive and moni torin g data ;

 intervi ew p rog ram me;

 case st udi es;

 sur ve ys;

 cost -benef it analysis;

 focus gro up;

 benchmarking.

1 . Do cum ent ar y re vie w and de sk r ese arch

Accordin g t o t he Tender Sp eci ficati on s the contract or was to recei ve monit or in g data

and sta tist ics col lected by CHAFEA an d the Comm issio n, n otab ly:

 dat a on CHAFEA’s finan cial and non-fin an cial perform ance av ailable fr om t he

Ann ual Act ivit y Rep orts and oth er d ocum entat ion;

 quant itative dat a collect ed f rom CHAFEA’s int ern al syst ems, ot her sources and

staff surv eys concern ing th eir sati sfact ion levels;

 cont act dat a requ ired for t he st akehol der consul tat ion strategy .

Dur ing t he k ick-off m eeting the St eering Group transfer red a set of dat a to PPMI,

following which t he ev aluat ion t eam assessed the qualit y of the dat a received and

identi fied fur ther dat a needs. The r esear ch t eam u ploaded th e dat a whi ch t hey init ially

received from t he Ag en cy t o PPMI’s in t er nal serv er. Over 120 files w ere r eceived and

placed int o 11 differ ent fold ers: survey data/ findings (8) , St eerin g Com mit tee meet ing

minut es and agendas (16 ) , reports ( 1), management an d st rategic p lans ( 13) , lists of

contract s (4) , l ists of pr og ram m es/ contracts/ f in ancial decisions/ot her r elev an t

docu m en ts (11), evaluations ( 7 ) , Com m i ssion d eci sions and regulation s (18 ) , CBA

studies (2) , AWP (2 7) and annu al acti vit y re port s (22 ).

Followin g initial assessm en t of t he data received t he research team r equest ed som e

addit ional data w hich w er e relevant for task 4 (cost -benefit an alysi s) . To facilitate

fu rth er eff ecti ve d ata transfer, th e evaluation team pr epar ed a d raft data managem ent

plan outlinin g key/ m ost pressing data needs of t he stu dy. Th e plan w as sen t t o

CHAFEA on 7 February . The plan out lined a lon g li st of data needs; alt hough som e

dat a w er e m ore ‘nice-t o-h av e’ to be able t o sh or t en the su rvey quest ionnair es ( e. g.

dat a on pay m en t s received , reports su bm itted or am endm ent s m ade; having these

dat a w ou ld reduce t h e need f or con t rol quest ions in the su rvey), oth er data were

identified t o be ‘m u st -have’ ( e.g. pr oj ect/ cont ract numbers, total budget & co-

finan cing rate, lead part ners an d their cont act details). While differ ent sets of data are

availab le f or diff er en t pr og ram m es and in st ruments; the goal is t o receiv e data that

are as simi lar/ com par able acro ss t he p rogram mes and inst rum ents a s possib le.

In th e files initially shared by CHAFEA officials, PPMI received a list o f con tracts

concluded in the four pr ogram mes during 20 14-20 16. Based on t he init ial assessm ent

it appear ed t hat som e proj ect s m ight be m issing (e.g . ExO cont r act s signed in 20 15 -

2016, bu t also som e other progr am mes for wh ich the n umbers did not add up to t he

totals provided in CHAFEA’s annual r ep or t s) . This confi rmed t he need f or mor e

structur ed dat a wit h list s of project s by contract/ pr oj ect number , progr am m e,

instrum ent and WP year. Furth er m ore, it was n ot ed t hat sh ou ld there be fu rther da ta

th is could benef it the st udy e.g . dat a on cont r act extensions in t h e BTSF progr am me