The findings of the evaluations

Pages12-31
12
2 THE FINDING S OF THE EVALUATI ON S
2. 1 TH E RES UL TS OF TH E INT ER VEN TI ON S
The following presentation of t he results of t he interventions frequently refers to
content of the primary evaluation report by giving the number o f the report as it is
included in the full set of reports for the meta-evaluation in Table 2 (Annex II).
Every mention of such a primary report is provided with a link to the entry in the
table, throughout the electronic version of this report.
2. 2 BA SE LIN E DAT A
A critical i ssue with the primary evaluations is that as many as 35 did not report
any kind of baseline information, i.e. a set of data collected before the
intervention has occurred. From the remaining part, many reports had some kind
of baseline information, for a variety of reasons often with limited validity. The
most frequent here is that the baseline information had not been collected at the
beginning of the intervention. In other cases the available data were too general,
not fully related to the objectives of the intervention, or not fully valid to measure
progress.
For instance, in the areas of education and health some interventions reported on
the Roma needs based on desk research and using secondary sources but not
necessarily corresponding to the target population. Some interventio ns, developed
in several regions or municipali ties, counted only on baseline data from some of
them or presented an overview of the situation in the country (e.g. r eport 24, 38).
Sometimes it was impossible to estimate the level of achievement, since no
targets were defined for some of the indicators and the level of contribution of
the intervention to the results could not be quantified (e.g. report 43). Some
reports observed a great difference in quality of the baseline of differe nt projects
under one programme (e.g. report 3). In a Roma-inclusi on study developed in BG,
RO, CZ and SK, baseline data were not planned along with the intervention or
collected at its start, and it was not possible to set up the base line retrospectively
due to the limited availability of data (report 42).
A mino rity of the evaluations used adequate baseline data to measure changes
between the beginning of the project and the time of the evaluation.
For instance, several interventions in education took students’ academic
achievements in the previous year or semester as the baseline. Some of the
interventions which developed training or supported participation or awareness
raising activities, collected data through questionnaires to the ta rget audience
( report 1, 22 and 60). For some interventions, ad hoc evaluations were developed:
for instance, UNDP developed a research in partnership with Babes Bolyai University
on a social intervention aiming at re-housing of families belonging to a marginalized
Roma community (report 60). Some interventions counted on databases (e.g. report
40) found interventions with a Database 1, existing since 2004-2005 and
managed by Ruhama foun dation and Database 2, created during the e xternal
evaluation phase of the project. The 2004 Slovakia Roma Atlas was used as a
baseline in report 57. The pr oject, Kindergarten access for the disadvantaged in
Bulgaria implemented a baseline survey at the start in 2 017 and covering 5712
13
households in 236 segregated communities (report 64). Some inte rventions took
information from previous reports as a baseline (report 4, and 62).
Whereas there are specific fields in which gender responsiveness has great
potential to increase the effectiveness of interventions, only a few reports
presented gender-disaggregated data.
IDE NT IF IC AT IO N OF T AR GE TS 2.2.1
More than half of the re ports did not identify any target for the i nterventions
evaluated. In most of the remaining reports the identi fied targets were very broad,
typically not quantified and sometimes not related to or beyond the capacity of
the intervention.
Examples of too broadly formulated targets are:
‘bridging the gap between Roma and nonRoma communities’ (report 22);
to reduce the gap between Roma and non-Roma children (report 6);
to stop cycle of poverty’ (report 20);
‘to reduce the disparity between Roma and non-Roma in ECEC’ (report 43).
‘Reducing the level of segregation’ or ‘improving Roma school attendance’ is the
target of many interventions in the field of education. Health interventions aim
‘for mediators to bridge the gap between Roma and institutions’ (report 7), or ‘to
increase individuals´ health knowledge, motivation and initiative and facilitate
healthcare access to Roma in disadvantaged segregated l ocalities’ (report 10). ‘To
improve Roma employability’ is an exam ple of a very broad target in the field of
employment (report 2).
Nevertheless there are several examples of well-defined targets.
The Community prosperity project in Slovakia quoted as t arget: 90 trained staff;
300 traine d participating organisations members; 10 projects submitted on
improving social and economic life of marginalized Roma; 10 equipped community
centres; 6000 targeted children by community centres activities; 70 of involved
children and youth will have finished at least one course or activity (report 16).
The target of Romaversitas in Hungary was for all students to pass at least the
intermediate exam in time for graduation and for 50 students to maintain a
minimum of 3.0 combined average GPA (report 37) and in Romaversitas in North
Macedonia graduation was the target (report 63). The target of UNICEF’s Invisible
Children programme in Romania was to extend access to essential services to 30
000 poor, excluded, vulnerable children (report 59) and the REF/RCRC Fo undation
High school support in Romania aimed to support 275 Roma high-school students
(report 26).
ACH IE VE ME NT O F TH E TA RG ETS 2.2.2
Although the interventions´ result may be good, many evaluati ons cannot assess
whether targets were achieved, as they are very broad and not expressed in
numbers. In t he field of education some evaluations reported full achievement of
their targets. Report 40 concluded that the participation target of 2158 children
and young people and 7298 parents was ac hieved. Report 47 quotes that the
involvement of 275 young Roma adults was achieved.

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT