Evaluating the Rating of Stiftung Warentest: How Good Are Mutual Fund Ratings and Can They Be Improved?

• Published date: 01 March 2014
• Date: 01 March 2014
• Author: Martin Weber,Sebastian Müller
• DOI: http://doi.org/10.1111/j.1468-036X.2011.00632.x

European Financial Management,Vol. 00, No. 0, 2012, 1–29

doi: 10.1111/j.1468-036X.2011.00632.x

Evaluating the Rating of Stiftung

Warentest: How Good Are Mutual

Fund Ratings and Can They

Be Improved?

Sebastian M¨

uller

Lehrstuhl f¨

ur Bankbetriebslehre, Universit¨

at Mannheim, L 5, 2, 68131 Mannheim, Germany

E-mail: mueller@bank.BWL.uni-mannheim.de

Martin Weber

Lehrstuhl f¨

ur Bankbetriebslehre, Universit¨

at Mannheim, L 5, 2, 68131 Mannheim, Germany and

CEPR, London, UK

E-mail: weber@bank.BWL.uni-mannheim.de

Abstract

We test the abilities of the Stiftung Warentest fund rating system to predict future

fund performance among German registered funds for six equity categories:

is a consumer protection agency and a major provider of fund ratings in

Germany. Our empirical analysis documents predictive abilities of the rating

system. The reason is that measures of past performance are positively related

to future performance in several of these markets, even after controlling for

momentum. Measures of fund activity (e.g.,Active Share) are also helpful to predict

performance, in particular to identify likely future losers. Hence, they should be

taken into consideration by consumers when selecting a fund and rating agencies

when revising current rating methodologies.

Keywords: mutualfunds,performance evaluation,performance persistence,mutual

fund ratings,active management

JEL classification: G11, G12, G1

The authors appreciate helpful comments and suggestions from an anonymous referee, John

Doukas (the editor), Jieyan Fang, Stephan Jank, Stefan Ruenzi, seminar participants at the

University of Mannheim and the 17th annual meeting of the German Finance Association

(DGF) in Hamburg. We also thank Stiftung Warentest for providing data on the ratings

and fund classification, Morningstar for data on mutual fund portfolio holdings and total

expense ratios, and Nico Hemker for excellent research assistance. Special thanks goes

to Andrew Patton for providing the code for the monotonic relationship (MR) tests in

Patton and Timmermann (2010) on his web-site. Financial support from the Deutsche

Forschungsgemeinschaft and Boerse Hamburg and Hannover is gratefully acknowledged.

Correspondence: Sebastian M¨

uller.

C

2012 Blackwell Publishing Ltd

Germany,Euro-Zone,Europe,North America, Pacificand World. Stiftung Warentest

2Sebastian M¨

uller and Martin Weber

1. Introduction

Focusing on the US equity fund market a tremendous amount of academic research has

examined whether measures of past portfolio performance are informative about future

performance. The debate is still ongoing and there are several papers that find some

degree of persistence (see e.g., Kosowski et al., 2006). Nonetheless, the results of many

papers suggest that persistence in equity funds’ risk-adjusted returns is rather weak after

controlling for survivorship bias and for momentum in stock returns (see e.g., Busse

et al., 2010; Carhart, 1997; Malkiel, 1995; Jensen, 1968). Further, recent studies by

Barras et al. (2010), Fama and French (2010), and Cuthbertson et al. (2010) highlight

the problem of identifying truly skilled fund managers if one has to rely on a limited

sample of historical returns. The results of these papers suggest that even funds in the

extreme right tail of the cross-sectional estimated alpha distribution have very often been

rather lucky than skilled. Consequently, it is not surprising that studies analysing the

value of the Morningstar rating system, which is based on past portfolio performance,

find the rating to be a rather poor predictor of future mutual fund performance for US

funds (see e.g., Blake and Morey, 2000; Morey, 2005 and Gerrans, 2006). Moreover, as

shown by Kr¨

aussl and Sandelowsky (2007), the forecasting abilities of the Morningstar

approach have declined over time. However, recent academic work by Amihud and

Goyenko (2009), Cremers and Petajisto (2009), Kacperczyk et al. (2005) and Wermers

(2003) shows that measures which quantify the degree of active portfolio management

are associated with higher risk-adjusted fund returns. These results suggest that investors

should consider the extent to which an open-end actively managed fund really pursues

an active strategy in order to select well-performing funds.

In this paper, we provide evidence on the value of a mutual fund rating system and

other measures of past performance as performance predictors in an international context.

Especially,we test whether the mutual fund rating system of ‘Stiftung Warentest’ is able

to differentiate between outperforming and underperforming German registered funds

that invest in one of the following six equity categories: Germany, Euro-Zone, Europe,

North America, Pacific, and World. In addition, we analyse whether measures of fund

activity (Active Share,Tracking Error and R2) also predict future fund performance

outside the US fund market. Given the results of the abovementioned studies, the degree

of active management appears to be one candidate to better differentiate between luck

and skill in the mutual fund industry. If so, ratings whichare based on historical por tfolio

returns could potentially be improved by taking these additional measures into account.

Analysing the quality of the fund rating system of Stiftung Warentestis interesting for

several reasons. First, Stiftung Warentest is a major fund rating provider in Germany and

it covers the entire German fund market. This allows us to analyse the performance of

funds that invest outside the local German market and even funds that invest worldwide,

whereas previous studies on mutual fund performance and performance persistence

outside the US vastly focus on funds solely investing in their domestic market (see

e.g., Otten and Bams, 2002; Griese and Kempf, 2003; Korkeamaki and Smythe, 2004;

Stotz, 2007 and Cuthbertson et al., 2010). Second, Stiftung Warentest is a consumer

protection agency, which aims at providing independent information on products and

services in a broad range of different fields in order to protect consumer interest.

Since Stiftung Warentest receives f inancial support from the German government and

its constitution prohibits any advertisements, its mutual fund recommendations should

be free of any conflicts of interest which have been documented for other financial

advisers (see Reuter and Zitzewitz, 2006). Third, information intermediaries like

C

2012 Blackwell Publishing Ltd