Evaluation of strengths and weaknesses

• Author: Applica, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Luxembourg Institute of Socio-Economic Research (LISER)
• Pages: 6-11

Case studies on the effectiveness of funding programmes – Key findings and study reports

6

evaluation2. For i nstance, in th e case of FEAD in Germany, the evaluation used a mixed-

methods approach with the use of monitoring data, qualitative analysis on the basis of two

waves of case studies and two standardised surveys of project promoters, as well as

interviews with experts. In the ca se of t he Flemish AMIF-funded prog ramme, monitoring

and evaluation were carried out in a participatory way, i.e. exp erimental measures were

continuously monitored and discussed with both implementers and beneficiaries. For

programmes funded by EEA grants, a sample of projects was monitored on an annual basis

and quarte rly progress reports we re prepared ; and an independent evaluation of

programmes in each country was carried out at some point during the Financial Mechanism.

Monitoring and evaluation were also part of the design of the Prevention and Early

Intervention Initiative: each beneficiary organisation was given a budget to commission

evaluations on the impact of their services (quasi-experimental studies, randomised

controlled trials, qualitative methods) and received practical support on how to plan and

execute evaluations from Atlantic Philanthropies staff. Grant recipients were also expected

to submit regular progress reports, and monitoring was supervised by the ‘advisory group’

and the ‘implementation group’й

2 Evaluation of strengths and weaknesses

This section presents a summary of the main strengths and weaknesses of the programmes

analysed in the case studies, i n terms of impact on the target group, on leveragin g extra

resources for the target group and on national and sub-national policies and programmes.

It also presents some lessons for EU funding programmes.

2.1 Impact on target group

All programmes under review were successful in achi eving their objectives and improving

the lives of the disadvantaged children targeted.

• Despite the fact that there were no quantitative targets for some programmes, it

was observed that the measu res concerned increased the number children

attending the services or facilities in question. For instance, in the case of the

programmes funded by EEA grants in Lithuania, the numbe r of children at risk

attending day-care centres increased significantly and the likelihood of children

entering foster care was reduced. These programmes had also a positive impact on

the development of public services for children at risk. In the case of the

programmes funded by the World Bank, th ere was an increase of the enrolment of

Roma children in early childhood education and care services.

• The programmes under review had also a positive effect on the lives of the children

concerned. In th e case of the FEAD funded programmes for instance, there is

evidence of children having improved acc ess to medical care, learning German,

going to kindergarten and participating in sports and leisure activities. The case

study analysing EU funds to address homelessnes s and housing exclusion also

reported an improvement in children’s health and well-being, as well as better

education achievements.

• In some cases, the positive impact also went beyond the target group to reach other

vulnerable children. For example, the case study on the integration of refugee and

migrant children into the Greek education system reported an increase in the

2 However, monitoring was not systematic in the case of the World Bank project for Roma children in Romania

and Bulgaria (no quantitative estimate) and it was underdeveloped in the case of the integration of refugees and

migrant children into the Greek educational system (no monitoring tools and processes had been provided for).

In addition, only an ad hoc evaluation of the first year of programme was carried out in Greece. In the case of

the Sure Start Children’s Homes in Hungary, monitoring efforts were done, although not on a very rigorous basis.