Executive summary

• Pages: 4-6

4

EXE CU TI VE S UM MA RY

The purpose of this meta-evaluation of interventions for Roma inclusion is to learn

lessons a bout what works and what does not work in this field and why. These

questions are particularly relevant because of the need to come to terms with

seemingly scarce evidence of success in this policy area, although considerable

efforts have been made.

The meta- evaluation assessed sixty-four reports on Roma-inclusion measures in

the fields of education, employment, healthcare, housing and non -

discrimination/fighting antigypsyism. Since very few evaluation reports are

available for some of these fields and we included a few study reports wi th

equivalent information in the deficient fields. This means that about 20 % of the

‘primary reports’ are for mally not an evaluation. The reports are commissioned by

a range of national and int ernational actors from 2005 onwards. They i nclude

information from thirty countries covering a set of more than 140 interventions,

roughly equally distributed over policies, programmes and p rojects. Most of the

evaluations selected are in English.

Two critical observations in this meta-evaluation regarding the primary evaluation

reports:

 The number of easily accessible evaluation reports in the various fields is

relatively low;

 Evaluations using robust evaluation methods are scarce.

In fact, in much more than half of the reports the baseline data against which the

achievements should be measured, are inadequate or absent. The same is true for

an identification of t he targets for the intervent ion. Gender-disaggregated data or

well-founded needs anal yses are rare to find. Only a few reports have an eye for

the impact and just a handful of evaluations compared what happened to what

would have happened without t he intervention (counterfactual anal ysis).

Nevertheless, the sample size allows a number of aggregated observations

regarding the interventions:

 Most of the evaluations, especially in the area of education, can demonstrate

results related to the intervention. In education there are several eval uations

that present medium-term effects and consequences linked to the project

goal. There are also examples in health care, but to a lesser extent. Still fewer

examples can be identified in employment and housing, but there are positive

results. Few interventions are ai med at combati ng racism and discrimination

against Roma in Europe and we found very few evaluations and little

evidence of demonstrable results.

 Both the instituti onal and the financial mechanisms of the evaluated

interventions are very diverse in terms of scale, duration and adequacy.

Through the Instrument for Pre -Accession Assistance (IPA) and t he European

Structural and Investment Fund (ESIF), the EU is a major financier of specific

Roma interventions, in some cases with large-scale projects.