Machine learning methods in finance: Recent applications and prospects

• Published date: 01 November 2023
• Author: Daniel Hoang,Kevin Wiegratz
• Date: 01 November 2023
• DOI: http://doi.org/10.1111/eufm.12408

DOI: 10.1111/eufm.12408

ORIGINAL ARTICLE

Machine learning methods in finance:

Recent applications and prospects

Daniel Hoang |Kevin Wiegratz

Institute for Finance, Karlsruhe Institute

of Technology (KIT), Karlsruhe,

Germany

Correspondence

Daniel Hoang, Institute for Finance,

Karlsruhe Institute of Technology,

Kaiserstr. 12, 76131 Karlsruhe, Germany.

Email: daniel.hoang@kit.edu

Abstract

We study how researchers can apply machine learning

(ML) methods in finance. We first establish that

the two major categories of ML (supervised and

unsupervised learning) address fundamentally differ-

ent problems than traditional econometric approaches.

Then, we review the current state of research on ML in

finance and identify three archetypes of applications:

(i) the construction of superior and novel measures,

(ii) the reduction of prediction error, and (iii) the

extension of the standard econometric toolset. With

this taxonomy, we give an outlook on potential future

directions for both researchers and practitioners. Our

results suggest many benefits of ML methods com-

pared to traditional approaches and indicate that ML

holds great potential for future research in finance.

KEYWORDS

artificial intelligence, big data, machine learning

JEL CLASSIFICATION

C45, G00

Eur Financ Manag. 2023;29:1657–1701. wileyonlinelibrary.com/journal/eufm

|

1657

EUROPEAN

FINANCIAL MANAGEMENT

This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial‐NoDerivs License, which permits

use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or

adaptations are made.

© 2022 The Authors. European Financial Management published by John Wiley & Sons Ltd.

We appreciate helpful comments and suggestions made by John A. Doukas (the editor), two anonymous referees,

Renée Adams, Andreas Benz, Francesco D'Acunto, Martin Ruckes, Fabian Silbereis, Michael Weber, and participants

at the 2022 European Conference of the Financial Management Association (Lyon).

1|INTRODUCTION

Artificial intelligence is increasingly entering our day‐to‐day life with impressive applications:

face detection enables safe and efficient airport travel, voice recognition allows for seamless

communication with personal assistants on smartphones and smart home devices, and ever

more firms are using chatbots for quick customer support. Almost everyone interacts with

modern artificial intelligence many times per day.

The main technology behind artificial intelligence is machine learning (ML). ML methods

enable machines to conduct such complex tasks as detecting faces, understanding speech, or

answering messages. Given the power of ML technology, it is natural to ask whether ML

methods can also be applied elsewhere. This paper addresses the use of ML to solve problems in

finance research.

Several overview papers indicate the potential of ML in finance. Varian (2014) describes ML

as an appropriate tool in the economic analysis of big data and presents some ML methods with

examples in economics. He further hints at potential ML applications in econometrics.

Mullainathan and Spiess (2017) identify prediction problems as the main use case of ML in

economics and present different categories of existing and potential future applications. Athey

and Imbens (2019) illustrate the most relevant ML methods from an econometric perspective.

They also provide an overview of ML's potential beyond pure prediction, especially for causality

in economic questions.

While the usage of ML in finance research is still in its infancy, the number of applications

that exploit the potential of ML has grown tremendously over the last few years. In 2018, the

number of ML publications more than tripled compared to the yearly average of the years 2010

to 2017. In 2019, the increase was already more than fivefold. In 2020, the increase was almost

sevenfold, and in 2021, there were almost 11 times as many publications using ML than before.

Even though the universe of ML applications in finance has greatly expanded recently, it is still

mostly unclear where and how to apply ML to solve research problems in finance.

The contribution of this paper is threefold. First, we present a high‐level primer on ML for

financial economists. We illuminate the different types of ML, their purposes and

functionalities, and the available methods for each type. Given our focus on finance, we place

special emphasis on the difference between traditional econometric methods and ML. We also

demonstrate the benefits of ML over traditional linear methods (particularly for prediction

problems) by applying ML to a high‐dimensional asset pricing problem in finance. Our

introduction allows researchers in the field to quickly grasp the essentials of ML that are

relevant for applications in finance without assuming any prior knowledge of ML.

Second, we construct a taxonomy of current and future ML applications in finance. Given

the increasing number of recent studies, earlier classifications do not capture existing

applications well. We review the up‐to‐date literature in the field and divide it into three

distinct archetypes. Our taxonomy allows researchers to better understand the current state of

the literature and how different contributions relate to each other. Furthermore, it serves as

guidance for future ML applications in finance.

Third, we study future prospects of ML applications in finance. We systematically analyse

ML applications in finance and how their publication success differs by research field (asset

pricing, corporate finance, financial intermediation, household finance) and application type.

Our results not only suggest a high potential for ML applications in general but also provide

researchers with indications of the most promising future directions.

1658

|

EUROPEAN

FINANCIAL MANAGEMENT

HOANG AND WIEGRATZ

Traditional econometrics aims to provide causal explanations for economic phenomena by

analysing relationships between economic variables. ML, in contrast, allows researchers to obtain

unique insights from high‐dimensional data. There are two major types of high‐dimensional data

for which ML offers benefits over traditional methods such as linear regression. First, ML can deal

with high‐dimensional, numerical data, that is, data consisting of a high number of variables

relative to the number of observations. Such high‐dimensional data arises if there is a plethora of

economically relevant variables or if nonlinearities and interaction effects play an important role.

ML methods leverage the informational content of such data for predictions with small out‐of‐

sample prediction errors. Second, in contrast to traditional methods, ML allows the exploitation of

unconventional data (such as text, images, or videos), which are inherently high‐dimensional. ML

methods can extract economically relevant information from such data, which then serves as a

starting point for further economic analyses.

ML is strongly related to the concept of big data. Big data consists of a high number of

observations, a high number of variables, or both (Stock & Watson, 2020, p. 515). In general,

data with a high number of observations improve the accuracy of ML predictions (in a similar

way to how they improve the precision of parameter estimates of ordinary least squares [OLS]

regressions). If the data exhibit a high number of variables (relative to the number of

observations), ML outperforms simpler, traditional methods such as linear regression. Applying

ML to data with high numbers of observations and variables combines both benefits as it can

yield high prediction accuracy as well as outperformance over traditional methods.

Based on our review of the finance literature, we classify ML applications into three distinct

archetypes: (1) construction of superior and novel measures, (2) reduction of prediction error in

economic prediction problems, and (3) extension of the existing econometric toolset.

First, researchers can use ML to construct superior and novel measures. For instance, when

applied to exploit unconventional data, the extracted information can serve as a superior or

novel measure of an economic variable. Superior ML measures may exhibit lower measurement

error and, therefore, can enable more precise estimates of economic relationships than

traditional measures can. Novel ML measures enable analyses with previously unmeasurable

economic variables.

Second, researchers can use ML to reduce prediction error in economic prediction

problems. For instance, the fundamental problem of pricing financial or real assets is the

prediction of adequate market prices. Given that a main functionality of ML is prediction, ML

methods can provide better results than traditional approaches in solving such economic

prediction problems.

Third, researchers can use ML to extend the existing econometric toolset. Econometric tools

often contain a prediction component. For instance, the first stage of an instrumental variable

design is effectively a prediction problem. ML methods can enhance such existing econometric

tools by improving the performance of their prediction component. Furthermore, some ML

methods themselves directly serve as new econometric tools. For instance, ML‐based clustering

methods extend the set of existing clustering methods from econometrics.

To demonstrate the benefits of ML over traditional methods at a typical prediction problem,

we apply ML to real estate asset pricing, which is particularly relevant in the areas of household

finance and real estate economics.

1

Real estate asset pricing is an inherent high‐dimensional

problem due to the large number of property characteristics, nonlinearities, and interaction

1

Our exemplary application cannot yield generalisable results about the performance of ML compared to traditional methods, but

illustrates how to apply ML to a typical problem in finance with high‐dimensional data.

HOANG AND WIEGRATZ EUROPEAN

FINANCIAL MANAGEMENT

|

1659