Statistical and Methodological Myths and Urban Legends in Strategic Management Research: The Case of Moderation Analysis

• Date: 01 March 2019
• Author: Ming Li,Barton M. Sharp,Donald D. Bergh,Robert Vandenberg
• DOI: http://doi.org/10.1111/emre.12319
• Published date: 01 March 2019

Statistical and Methodological Myths and

Urban Legends in Strategic Management

Research: The Case of Moderation Analysis

MING LI,

1

BARTON M. SHARP,

2

DONALD D. BERGH

3

and ROBERT VANDENBERG

4

1

University of Liverpool Management School, Liverpool, UK

2

Department of Management, Northern Illinois University, DeKalb, IL, USA

3

Daniels College of Business, The University of Denver, Denver, CO, USA

4

Terry College of Business, University of Georgia, Athens, GA, USA

This paper examines whether methodological precedence in applying moderation analysis to strategic

managementresearch relieson myths and urban legends, andif doing so affected empiricalconclusions, implications

for theory development, and practicalrecommendations. An in-depthanalysis of 69 studies publishedin the Strategic

Management Journal between 2000 and 2014 using moderation analysis finds that strategic management scholars

typically rely on statistical myths and urban legends when applying moderation analysis including: (1) interpreting

main effects separately from theirsignificant interaction withother variables; (2) failing to reportreliability values of

interaction terms; and (3) relying on hierarchical approaches that can lead to interpretation errors. Further

examples illustrate how these practices could lead researchers to draw incomplete and possibly inaccurate

conclusions. Overall, problematic precedents have become the gold standards for testing and interpreting

moderation models. Best practice recommendations for redirecting future research to more solid methodological

grounding are provided.

Keywords: methods; moderation; moderators; myths; regression

Introduction

When justifying their methodological decisions,

researchers tend to use the common practices in a field

as precedent and support(Mizruchi and Fein, 1999; Boyd

et al., 2005b). Some of t hese practices bec ome widely

acceptedwithout question and developinto gold standards

that authors and reviewers rely on in making decisions.

However, as methodological practices diffuse, they can

evolve into what have become known as ‘statistical and

methodological myths and urban legends’(SMULs)

(Vandenberg, 2006). SMULs involve statistical processes

and decisions that,although possibly originating in sound

science, become distorted and exaggerated as the

approaches diffused throughout the field (Spector, 2006)

such that the intended meaning was lost along the way

(Vandenberg, 2006). Lance (2011: 280–281) has

described such myths as ‘those rules of thumb, maxims,

truisms, and guidelines for research conduct …received

doctrines …that are otherwise passed from generation

to generation. They establish normative research conduct

…inform us about how we should go about our research

…often based,in part, on sound rationale and justification

but also, in part, unfounded lore’.

1

These statistical rules

of thumb, conventional standards, and received

methodological doctrines may not be valid (Lance,

2011) and their application can lead to inappropriate

findings and inferences (Vandenberg, 2006; Lance and

Vandenberg, 2009).

We consider the presence and implications of SMULs

in strategic management research which is a relatively

young academic field (Boyd et al., 2013). Developed

since the late 1960s, it is a multidisciplinary area of

research and has borrowed heavily from economics,

sociology, psychology, political science, evolutionary

ecology, philosophy, and other disciplines (Durand

et al., 2017). Additionally, the domain of strategic

Correspondence: Ming Li, University of Liverpool Management School,

Chatham Street, Liverpool, L69 7ZH, United Kingdom. E-mail

mll@liverpool.ac.uk

1

Statistical myths and urban legends are apparent in many methodological

decisions.See Lance and Vandenberg (2009,2015) for applications.

European Management Review, Vol. 16, 209–220, (2019)

DOI: 10.1111/emre.12319

©2018 European Academy of Management