Survey design and methods

• Author: Ford Hickson - Axel J. Schmidt - David S. Reid - Peter Weatherburn - Ulrich Marcus - Susanne B. Schink
• Pages: 9-26

TECHNICAL REPORT EMIS-2017 – The European Men-W ho-Have-Sex-With-Men Internet Surv ey

9

2. Survey design and methods

2.1 Overview

Questionnaire design – the qu estionnaire was based on a previous version of the survey (EMIS 2010).

Development was informed by a scoping exercise of available MSM questionnaires published since EMIS 2010,

followed by three consultations among our partners, an observed pre-test including assessment of completion time

(‘time trial’) and a formal pilot of the online version in English only.

Final questionnaire content – the final version included:

Demographics: 34 descriptive items.

Morbidities: 16 items concerning health outcomes.

Behaviour: 83 items concerning people’s actions.

Needs: 22 items or sets of needs-related items relating to opportunities, capabilities and motivations for risk

and precaution behaviour.

Interventions: 35 items or sets of items on the actions of others that meet or undermine needs.

From these items, we were able to construct 17 ECDC Dublin Declaration Monitoring (DDM) indicators 15 and

another two Global AIDS Monitoring (GAM)16 indicators.

Translation and online preparation – after translation from English and checking by partner s, online versions were

checked for structural homogeneity. National partners confirmed the terminology fitted with their perceptions of

the norm for the target group in their country, reviewed the final survey, and signed-off their main language

version. The survey was available in 33 languages, including 23 of the 24 official languages of the EU.

Online recruitment – online promotion began on 18 October 2017 and ran until 31 January 2018. Country leads co-

ordinated national advertising including the number, size and specification of promotional materials. Sigma staff

commissioned advertising from ten multi-country dating platforms.

Final sample – the narrative element of this report is ba sed on 127 792 MSM living in 48 countries (all 28 EU

countries, three EFTA countries, four non-EU European microstates, seven countries of the EU enlargement area,

three countries from the Eastern Partnership, two countries from the Southern Neighbourhood and Russia). The

two other countries that w ere included – Canada and the Philippines – appear only in the country tabl es at the end

of each chapter, and are n ot included in the report’s narrative descriptio n of the findings.

Country-specific findings – in Chapters 3–7 and 9 the key findings are presented in tables by country of residence

at the end of the chapter, with countries presented alphabetically in the four primary country groupings listed

above. Key variation betw een the residents of specific countries is also illustrated in maps throughout the report.

Ethics approval was granted by the Observational Research Ethics Committee at the London Sch ool of Hygiene and

Tropical Medicine (reference 14421 /RR/8805) on 31 July 2017.

2.2 Countries participating in EMIS-2017

As a minimum, we sought to recruit 100 000 MSM resident i n 33 countries where funding existed for translation of

the survey and its promotion. These 33 countries included all 28 EU Memb er States and two EEA EFTA countries

(Norway and Iceland), and thr ee other countries – Bosnia & Herzegovina, Moldova and Ser bia - that are members

of the third European Health Programme (EHP)17.

We did not exclude residents of the four non-EU European mi crostates that adjoin or are encompassed by larger

countries already included in the sample. Although we did not expect to rec ruit a viable sample in these

microstates we included men living in them in the dataset for adjoining or encompassing countries – Andorra with

Spain; Liechtenstein with Switzerland; Monaco with France and San Marino with Italy.

Outside the EU Health Programme, we included only countries where we had an established relationship with

national partners who were actively seeking to be involved. Although our main funding disallowed dedicated

investment in recruitment for these countries, these p artners provided any required translation themselves and

invested time and energy, and sometimes money in promoting the survey in their country.

Outside the main budget for translation and promotion we accepted participation from Switzerland (as the only

EFTA country not in the EU Health Programme); and from residents in eight other countries which are either EU

15 https://ecdc.europa.eu/en/all-topics/hiv-infection-and-aids/prevention/ monitoring-implementation-dublin-2018

16 http://www.aidsinfoonline.org/gam/libraries/aspx/ Home.aspx

17 https://ec.europa.eu/health/programme/who_can_participate_en

EMIS-2017 – The European Men-Who -Have-Sex-With-Men Internet Surv ey TECHNICAL REPORT

10

Enlargement Area countries 18 or are part of the European Neighbourhood Policy (ENP)19. This group of countries

includes all remaining EU Enlargement Area countries (Albania, Kosovo§, North Macedonia, Montenegro, and

Turkey). The other two enlargement area countries – Bosnia & Herzegovina and Serbia – were already included as

members of the Health Pro gramme.

We also accepted participation from two of the six countries in the Eastern Partnership (EaP )20 of the European

Neighbourhood Policy (Belarus and Ukraine but not Armeni a, Azerbaijan and Georgi a). The EaP is a joint policy

initiative which aims to deepen and strengthen relations between the European Union and its six eastern

neighbours. Moldova is the third country in the Eastern Partnership but it was already included as a contributing

member of the Health Programme. In all three of the countries included (B elarus, Ukraine and Moldova) we had

partners keen to be involved. We excluded residents in the o ther three Eastern Partnership countries – Armenia,

Azerbaijan, Georgia – as we w ere not able to find partners in these countries.

We also included two of the ten countries participating in the Southern Neighbourhood21 within the framework of

the European Neighbourhood Policy. Partners in Israel and Lebanon translated and promoted the survey at their

own cost. We excluded MSM living in all the other ENP Southern Neighbourhood countries – Algeria, Egypt, Jordan,

Libya, Morocco, Palestine, Syria and Tunisia – as we could not identify gay or MSM organisations with which we

could work.

The final group of countries where participation was allowed included Russia, Canada and the Philippines. Russia

was included as a majority of its population resides in Europe and it had also participated in EMIS 2010, so we had

a pre-established partnership with local stakeholder s willing to undertake translation into Russian. Russian

participation was funded by the Arctic University of Norway and University Hospital of North Norway. The Public

Health Agency of Canada approached us and asked us to extend the availability of our questionnaires to Canadian

provinces (since EMIS-2017 included both of Canada’s official languages). Promotion in Canada was funded by the

Public Health Agency of Canada. The World Health Organization’s Regional Office for the Western Pacific

approached the Robert Koch Institute and asked if the questionnaires could be made available to the Philippines.

Online translations into the main language s of the Philippines were provided free of charge i n collaboration with

the Philippines Ministry of Health. Promotion in the Philippines was funded by the Office of the WHO

Representative in the Philippin es.

In each of these 50 countries, except for the microstates described above, we sought to recruit at least 100

residents to ensure a viabl e reporting sample, although we recognised this may not be possible in countries with a

small overall population, such as Iceland (350 000), Malta (475 000), Luxembourg (600 000) or Montenegro

(680 000). We also committed to create and pass back a national dataset to all lead agencies for countries with

100+ respondents, which included all valid participants resident in the country.

Our partnership development process ensured we had a national lead for the promotion of EMIS-2017 and/or a

national lead for data analysis in all countries except Luxembourg and Kosovo.

The narrative element of this report includes men resident in all EU countries and adjoining microstates22; all

countries in the European Free Trade Association; men resident in the above candidate or potential candidate

countries in the current EU Enlargement area; men r esident in countries in the Eastern and Southern Partnership

countries of the European N eighbourhood Policy area an d men resident in Russia. In this chapter and Chapters 3–7

and 9 the key summary figures are presented in tables by country of residence, with countries presented

alphabetically for the four primary groupings above. We also show residents of Canada and the Philippines in these

country tables, but they are not included in the narrative element of the report.

§ This designation is without prejudice to positions on status and is in line with UN SCR 1244/1999 and the International Court of

Justice Opinion on the Kosovo declaration of independence.

18 https://ec.europa.eu/neighbourhood-enlargement/countries/check -current-status_en

19 https://eeas.europa.eu/headquarters/headquarters-homepage/330/european-neighbourhood-policy-enp_en

20 The Eastern Partnership (EaP) is a joint initiative involving the EU, its M ember States and six Eastern European Partners:

Armenia, Azerbaijan, Belarus, Georgia, the Republic of Moldova and Ukraine. The E astern Partnership, which is a dimension of

the European Neighbourhood Policy, aims to build a common area of shared democrac y, prosperity, stability and increased

cooperation. https://ec.europa.eu/neighbourhood-enlargement/neigh bourhood/eastern-partnership_en

21 EU cooperation with the ten partner countries of the Southern Neighbourhood takes plac e within the framework of the

European Neighbourhood Policy (ENP https://ec.europa.eu/neighbourhood-enlargemen t/neighbourhood/southern-

neighbourhood_en)

22 The United Kingdom includes England, Scotland, Wales, Northern Ireland and British Overseas T erritories and Crown

Dependencies. France includes French Overseas Departments, Territories and Collec tivities. Denmark includes the Faroe Islands

and Greenland. Finland includes Åland. Italy includes the Vatican City. Nor way includes Svalbard. Portugal includes the Azores

and Madeira. Spain includes the Canary Islands. The Netherlands does not include the Caribbea n island territories of Aruba,

Curaçao, or Sint Martin.