Conclusions

• Author: Giulio Caperna - Eleni Papadimitriou
• Pages: 20-20

20

6 Conclusions

The JRC statistical audit delves into the extensive work carried out by the developers of the IDM with the aim

of suggesting improvements in t erms of data characterist ics, structure and methods used. The analy sis aims

to ensure the transparency of the IDM methodology and the reliability of the results. This report focuses on the

assessment of the statistical c oherence of the Individual Deprivat ion Measure, for t he Fiji 2017 dataset, by

carrying out a multilevel analysis of the correlations within and across the indicators and dimensions as well as

by an assessment of the impact of k ey modelling assumptions on the I DM scores and ranking.

The analysis suggests that the IDM is statistically well balanced with respect to the 13 dimensions used in the

specific datase t, with the exception of the Time –Use dimension (dim 14). For most dimensions, the correlations

are significant and positive an d the indicators are more correlated wit h their own dimension than with any other,

thus suggesting t hat they provide meaningful information on the variation of the score s. Similarly, all themes

are correlated with their dimensions and the overall IDM, with positive, though often not very strong correlations.

However, two issues were identified: First, the Time use dimension does not correlate with the overall index and

most of the other dimensions and even shows a negat ive relationship with some of them. JRC re commen ds to

consider modifying or even excluding the Time Use dimension from the framework. The same dimension or its

components can still be used for further analysis, in relation with the IDM. Second, the themes within the Health

dimension are negatively correlated with each other, which often implies that they are describing a different

concept than the one assumed in the conceptual framework. JRC would suggest special attention to this aspect,

given the importance of Health. The revision or exclusion of indicators 4.1.1 and 4.2.1 could be c onsidered, in

order to improve the coherence of the Health dimension and its influence in the overall score. The developers

consider Time Use to be an important conceptual element of a gender sensitive measure of deprivation and so

have indicated their intention to modify the scoring method of the indicat ors within both the Time Use and

Health dimensions to improve statistical coherence.

The conceptual framework of the IDM c ategor ises t he 15 dimensions into three groups of importance and

weights those accordingly. However, as in the Fiji dataset the two dimensions of Violen ce and Family Planning

are missing, the w eights of the dimensions in the seco nd and third gr oup are highe r than they would be if al l

dimensions were present. The JRC analysis showe d that in the spec ific dataset t his choice of weighting is not

influencing the final results too much. However, it is important to consider if the three groups need to maintain

this difference in w eights irrespective of the number of dimension each group contains ; or the individual

dimensions should maintain the assigned difference between them. I n the first case, having less dimensions in

the second or third group increases the risk for these dimensions to account for more (or similar) than those in

the first group. The suggestion is to seriously consider the influence of missing dimensions on the weighting

scheme when implementing the IDM in other countries, particularly where multiple dimensions are missing from

the same group.

The sensitivity analysis present ed in the report, shows a comparison be tween the arithmetic mean of the

dimensions, which is the aggregation method chosen by the developers, and the geometric mean. The first

implies a stro ng compensabilit y that allo ws outstandin g performance in some aspect s to balance for

weaknesses in others while the second penalizes the existence of a low value, even when the other values are

not so low. The analysis shows no m ajor differences in the final ranking for the two met hods, suggesting a

satisfying robustness of the index in respect to this methodological choice. As the nature of the data allows for

that, JRC suggests considering also an alternative aggregation method, the Copeland scoring, which is one of

the least compensatory aggregation methods. The comparison of this method with the arithmetic average

showed that indeed the aggregation of the 13 dimensions of the IDM is influenced by the degree of

compensability of the aggregation function. A downside of this method in the specific dataset is that although

it offers great advantages for the between individuals comparisons, it would not allow for comparisons between

different countries or over time.

In general, the present audit confirms that the IDM Fiji dataset meets the quality standards for statistical

soundness and acknowledges the important efforts of the developers’ team in the definition of a composite

indicator for individual s. The IDM can ser ve as a tool to provide insights for individual de privation and poverty.