Annex 6.1 - Delphi survey findings

• Author: Cambridge Econometrics, Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Eurocentre, ICF
• Pages: 449-465

Study supporting the evaluation promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

449

ANNEX 6.1 – DELPHI SURVEY FINDINGS

Overview

An online EU-level Delphi survey was carried out to support a European Commission

evaluation of ESF Thematic Objective 9. The survey was administered in two rounds

between 9 April 2020 and 13 May 2020. A total of 10 representatives from a range of

organisations involved in policy debates at the European level on social i nclusion, poverty

and discrimi nation as well as a Managing Authority not consult ed in other stages of the

study and an auditor participated in the Delphi survey.594

In Round 1 of the survey, respondents reviewed a background document that presented

the overall evaluati on findings and more specific findings i n relati on to two focus areas.

Respondents were then asked to provide their reflections and additional inputs. The

background document was revised based on the inputs received. In Round 2, the

respondents reviewed the revised background document and provided further clarification

and validation.

Section 2 pr ovides an overview of Round 1 survey results while Section 3 presents the

Round 2 survey results.

Sections 4 and 5 represent the substance of the background docum ent that was refined

based on the inputs gathered through the Delphi survey.

Summary of Round 1 survey results

This section summarizes the results and contributions from the open-ended questions from

Round 1 of the Delphi Panel. Round 1 was completed by a total of 9 participants. The

below section reflects on the inputs they provided on the different evaluati on criteria.

Effectiveness

6 out of 9 participants agreed with the findings in relation to effectiveness, notably

regarding the identified key chal lenges such as low ad ministrative capacity coupled with

perceived complex administrative requirements for receiving ESF funds.

Respondents hig hlighted several success factors for the effectivene ss of ESF support for

TO9:

 Coordinated and coherent approach to investment

 Alignment of priorities with broader EU and national policy goals

 Linkages between policy goals and local/regi onal needs to help build political support

for reforms

On the other hand, respondents con firmed a number of challenges that were also

identified in the evaluation:

 Requirements for receiving ESF funds as wel l as requirements for proje ct

implementation are co mplex and challengin g for potential benefi ciaries e.g. eli gibility

rules, payment applications etc.

 Low administrative capacity of potential beneficiaries to comply with project planning,

management and reporting requirements

594 10 representatives participated in Round 1. Of these 10, 4 participated in Rou nd 2.

Study supporting the evaluation promoting social inclusion, combating poverty and any

discrimination by the European Social Fund (Thematic Objective 09)

450

Stakeholders noted that in order to fully measure the effectiveness of ESF support for TO9

in the ar eas of employment, participation in education and training, etc. there must be

linkages to policies promoting social inclusion, poverty reduction and combat ing

discrimination.

Efficiency

The majority of participants agreed with the key findings of the evaluation in relation to

efficiency. The participation of local communities was identified a key success factor of

many ESF-fun ded pr ojects. H owever, m ore cou ld be done to faci litate a systematic and

transparent system for generating the invol vement of local stakeholders. Coll aboration

through cross-sectoral networks we re found to be useful to capitalise on ESF funding and

develop integrated solutions to promote ESF objectives.

The implementation of Simplified Cost Options (SCOs) was often perceived to be difficult

and time consuming for both Managing Authorities and beneficiaries.

Training/workshops/guidelines etc. f or benefi ciaries and also for the Managing Authority

could decrease this burden for stakeholders.

The development of innovative approach es was considered to have a negative impact on

efficiency. These approaches are overall less- cost effective as their design, set-up and

implementation requires additional efforts. However, in the long run testing these

innovative approaches can lead to new wa ys of worki ng and simplified procedures. The

development of these approach es is considered crucial to increase efficiency i n long-term

especially for small- and medium-sized beneficiaries.

It was also suggested that efficiency considerations should not be limited to costs, but

should reflect processes e.g. time-length of action from inception to resu lt.

Relevance

The majority of participants agreed with the key findings of the evaluati on in relation t o

relevance. According to respondents, the relevance of ESF projects is better ensured when

they:

 Target social problems with the rig ht solutions. Respondents emphasized that

relevance can be ensured when the objective of the interventions are clearly linked to

strategic policy documents (e.g. health strategy). In this case they are more likely to

deliver the right outcomes. Evidence-, needs-, socio-economic conte xt- and

community-based consultation are recommended.

 Involve a wide spectrum of stakeholders in the design and/or implementation of a

project is important to build buy-in for the project and its results and to ensure that

the objectives and activities are realistic. It was emphasized that identification of target

groups of beneficiaries in early stages of the ESF-funded projects is essential to

increase the relevance of the projects

A successful project example from the health sector showed that it is beneficial for

successful ESF-funded h ealth projects to directly engage with a wide rang e of health and

other important stakeholders such as patients, trade unions and professional associations,

hospitals and public health institutions, as well as targeted users of the outputs.

Stakeholder feedback can identify not only existing needs, but also the most suitable

solutions and approaches to addressin g these gaps. Ensuring that ESF-funded projects

support workable solutions that have the acceptance of the various sta keholders involved

can facilitate the subsequent roll-out and replication of successful practi ces.

One of the key issues that emerged from the responses for further improvement is the

relevance of TO9 operations in regions (or rath er areas/local communiti es) with a poorer

socio-economic situation. Some areas in need of ESF support for TO9 do not have an NGO