Annex 5 - methodology for main task 5 (behavioural experiment)

• Author: Consumers, Health, Agriculture and Food Executive Agency (European Commission), Deloitte consortium, IPSOS, London Economics
• Pages: 79-101

79

Annex 5 Methodology for Main Task 5 (Behavioural

experiment)

The behavioural experiment addressed two key objectives in the context of consumer

search for information on real estate advertisements:

• How easy it is for consumers to find and assess information; and

• Assess how much information can be ‘digested’ by consumers.

A5.1 Key design features of the behavioural experiment

The experiment featured at the end of the consumer survey from Main Task 2. and had an

average duration of 15 minutes. The experiment was conducted with 1,003 respondents

in France, and 1,001 respondents in the UK.

Components of the experiment

The components of the online experience for respondents were as follows:

• Allocation to t reatments and scenarios: Thi s stage, unseen by respondents,

divided respondents into different groups.

First, respondents were randomly allocated to one of four scenarios. These were:

o Domestic rental

o Domestic purchase

o Cross-border scenario 1: Purchase of a property abroad from a website based

in the home country.

o Cross-border scenario 2: Purchase of a property abroad from a website based

in the host country.

Within each scenario, there were five treatments. The exception is domestic rental in which

there was one additional treatment.

• Explanation of tasks and incentives: This part briefly explained th e experiment

tasks and scenarios to respondents and mentioned the functi oning of additi onal

monetary incentives.

• Stage 1 – comparison of multiple property advertisements

• Stage 2 – closer examination of a single property

• Post-experiment survey questions: These questions were asked to assess

respondents’ understanding and qualitative appreciation of the previous tasks.

Moreover, additional respondent characteristics, such as behavioural drivers, were

elicited.

Treatments

The following treatments systematically varied the look and feel of th e experiment for

respondents. Respondents were allocated randoml y and independently to a variant within

each treatment category. Each treatment category had two to three variants. Unless

indicated otherwise, the treatments were applied across stages 1 and 2 of the experiment.

• Size treatment: This treatment varied whether th e property’s surface in sqm (or

sqft in the UK) was displayed, or not.

• Key features treatme nt (stage 1): Thi s treatment varied whether key features

of the real estate properties were shown via icons, and whether these icons were

displayed consistently across properties, or on an ad-hoc basis.

• Energy performance certificate treatment: This treatment varied whether the

EPC of the property was displayed, or not.

80

• Price framing treatment: This treatment varied whether the displayed rental and

purchasing prices were inclusive or exclusive of, respectively, residential taxes, or

stamp duty.

• Rental price treatment: This treatment was applied to the domestic rental

scenario only. It varied whether the displayed rental prices were shown as prices

per month, or per week.

• Key features treatme nt (stage 2): Thi s treatment varied whether key features

of the property were displayed in icons, or not.

• Tabular format treatment (stage 2): This treatment varied whether the features

of the property were displayed in textual format only, or also in a tabul ar format.

Performance in the experiment was incentivised using Ipsos panel points which convert to

monetary p ayoffs via consumption vouchers a t popular retailers in the respective panel

countries. More specifically, respondents earned additional points via correct answers t o

objective questions with correct/incorrect answers.

A5.2 Weighting the data

As decided during the inception phase of the study, 50% of respondents in the behavioural

experiment were required to ha ve relevant experience in the real e state market (called

Group 2); this was a screening requirement for taking part in the general consumer survey

(Main Task 2)7. The remaining 50% of respondents were recruited from the general public

without any further restrictions (called Group 3).

The Tender Specifications require the experiment sample to be representative of the

general population of consumers for the issue at stake. Foll owing the fieldwork, it

was concluded that the natural incidence of consumers with experience in the real estate

market was approx. 50% in France and Germany. Due to the construction of the sample,

consumers with experience in the real estate market are overrepresented in the final data

set:

• There are approx. 1,000 (i.e. 500 per country) respondents with experi ence in the

real estate market from Group 2.

• There are a further approx. 500 (i.e. 250 per country) respondents with experience

in the real estate ma rket f rom Group 3. This is due to the natural incidence of

experience in the market among the general population.

• The final dataset thus contains approx. 1,500 respondents with experience in the

real estate ma rket. This corresponds to an incidence of 75% of consumers with

experience in the real estate market, which is hi gher than the natural incidence of

such consumers (50%).

As a result, a weighting variable was constructed that reweights the data to achieve

representativeness in terms of demographics and experience in the real estate market.

A5.3 Detailed results from the behavioural experiment

A5.3.1 Consumer understanding of real estate advertisement

This Annex reports the detailed findings from the b ehavioural experiment conducted for

this study. Similar to tasks undertaken during the myste ry shopping, the experiment lead

respondents through two typical stages of real estate websites (see Full Report text for a

short overview). In th e first stage, respondents compared multiple properties on screen

with limited informati on regarding each property. In the second stage, respondents were

shown one property with in-depth information. At both stages, respondents were asked to

answer questions regarding the real estate advertisements at hand. Key performance

7 Other respondents in the consumer surveys, not taking part of the experiment, were labelled Group1.