A New Typology of Monitoring and Evaluation Indicators

• Pages: 113-139
• Date: 01 December 2021
• Published date: 01 December 2021
• Author: Dominique E. Uwizeyimana
• Subject Matter: Derecho Público y Administrativo

Special Issue

December, 2021

European Journal of Economics, Law and Social Sciences

IIPCCL Publishing, Graz-Austria

ISSN 2519-1284

Acces online at www.iipccl.org

128

A New Typology of Monitoring and Evaluation Indicators

Dominique E. Uwizeyimana

The University of Johannesburg

Abstract

The objective of this article is to describe and explain what evaluation indicators are, what

their role is in systematic evaluations, and to dierentiate them according to the dierent

classif‌ication frameworks gleaned from the current literature. Many classif‌ication systems

in literature have been used to classify the dierent types of evaluation indicators. The fact

that any classif‌ication depends on one’s perspective makes it almost impossible to provide a

systematic distinction between the dierent evaluations. The methodology used in this article

is mainly based on critically analysing various secondary sources to achieve a more systematic

discussion and consolidated conclusions that will help to harmonise the competing types of

evaluation indicators and to suggest how they can be systematically classif‌ied. After critical

analysis of literature on the dierent classif‌ication evaluation indicators’ general types, the

study concluded that the competing types of evaluation and evaluation indicators in the

current literature can be systematically organised and classif‌ied into two main groups: namely

(a) classif‌ication of evaluation indicators by their nature; and (b) classif‌ication of evaluation

indicators by what these indicators are used to measure.

Keywords: M&E, Monitoring and Evaluation, Public policy, Indicators, evaluation indicators,

performance indicators.

Introduction of Monitoring and Evaluation (M&E)

Cloete, Rabie and De Coning (2014, 11) argue that because “change and development

are complex societal phenomena and are always subject to normative interpretation

and assessment… the complexity of societal change and development makes

accurate measurement of these phenomena dicult”. To deal with the complex

nature of societal changes, evaluators “must consider very closely which measuring

approaches and instruments they apply to reect, as accurately as possible, on the

evaluand. Evaluation indicators make it possible to make evidence-inuenced (or

evidence-based) judgements about the merit or worth of the intervention concerned,

instead of subjectively informed opinion-inuenced judgements that might reect

one or more deliberate or subconscious biases of some sort” (Cloete et al., 2014;

Uwizeyimana, 2020b, 114).

Therefore, monitoring of government interventions involves “continuous and

systematic collecting, analysing, and reporting of policies’, programmes’, and

projects’ performance data to support eective management (Cloete, et. al, 2014, 4).

It is important from the start to clarify that the concept “government interventions”

will be used as a blanket term to refer to the implementation of government policies,

programmes, projects, strategies, and plans. It refers to all government eorts to solve

or deal with any social, economic, political, environmental, and technological issues

European Journal of Economics, Law and Social Sciences

IIPCCL Publishing, Graz-Austria

Special Issue

December, 2021

ISSN 2519-1284

Acces online at www.iipccl.org

129

or problems that are aecting citizens. The Organisation for Economic Co-operation

and Development (OECD) (2002, 21) def‌ines evaluation as the “systematic and

objective assessment of ongoing or completed government interventions, its design,

implementation and results”. Cloete et al. (2014) and the OECD’s (2002) def‌initions of

the concept of evaluation identify important lessons about what evaluation is, and the

criteria it must meet to be useful to policy-makers, the government, and the citizens.

The f‌irst lesson is that evaluation can be done on an “ongoing or completed”

intervention. It is impossible and needless to evaluate public policies (or generally

government interventions) that have not been implemented. No policy can succeed

or fail if it has not been implemented. Here, the ongoing evaluation or formative

evaluation or “ex-ante evaluation” includes the evaluation of the planning phase, or

the design phase (OECD, 2002: 22).

The second lesson is that evaluation can focus on the implementation process for

government interventions. This type of evaluation is called ongoing or process

performance evaluation (Rabie & Cloete, 2009, 11). The third lesson is that evaluation

can also focus on the results of the fully or partially implemented government

interventions. The concept “results” is used in this article to refer to outputs,

outcomes, and impacts of government interventions. The evaluation that focuses on

the results is called “summative evaluation” (Cloete, 2009: 206). The dierent types

of summative evaluation are output evaluation, outcome evaluation, and impact

evaluation (McDavid & Hawthorn, 2006: 33).

The fourth lesson is that an intervention must have specif‌ic targets that are to be

achieved, and specif‌ic key indicators that must be used to evaluate the extent to

which they have been achieved or not achieved. These specif‌ic targets must abide

by the regular interdependent constraints of policy (and programme and project)

implementation. These constraints relate to the fact that government interventions

must be time-bound, cost-bound (f‌inance, budgets), quality-bound, and scope-

bound. They must also be eective and ecient in dealing with the specif‌ic issues

that are aecting the intended communities or society.

The f‌ifth lesson is that for evaluation reports (the f‌indings and recommendations)

to be useful, the evaluation process must be systematic, the targets must be clearly

def‌ined, and the indicators that are used to evaluate must be “objective” (Auriacombe,

2011, 43). In addition, the evaluation process must be meticulously planned and

systematically executed by evaluators who are properly qualif‌ied to conduct credible

evaluations (Uwizeyimana, 2020). The sixth lesson is that to determine the focus of

evaluation, the type of evaluation to be conducted, the type of evaluation indicators to

be applied by the evaluators, and the design of an evaluation must be guided by some

specif‌ic questions. According to Weiss (1997), the many questions that evaluation

must strive to answer include (but are not limited to) the following:

a) If the focus of the evaluation is on the plan: Was the plan clear about what is

to be done, why it should be done, where it must be done, who must do it, and what

resources (budget, people, tools, machinery) are needed to get the job done?

b) If the focus is on the process of implementing the interventions: Was the

process of converting inputs into outputs well planned? If the process was well

planned, was it properly implemented? If it was well implemented, did it lead to