Market-Implied ESG Score: An Alternative Scoring Method
A recent paper proposes a novel approach to derive sustainability scores from fund holdings data
A common criticism of ESG ratings by providers such as Refinitiv, Bloomberg and MSCI is that they tend to disagree, leading to confusion among asset owners and investors. For instance, provider A may give Tesla a high ESG score but provider B may rate Tesla poorly if it doesn’t like the company’s governance approach.
This is a problem for fund managers with a mandate to maximise a fund’s sustainability impact: whose ratings should they trust?
A recent paper published by Rosella Giacometti, Gabriele Torri, Marco Bonomelli and Davide Lauria from Italy’s University of Bergamo explores a novel approach to score companies based on sustainable funds’ holdings data. The logic is simple: if a company appears disproportionately in funds with strict sustainability mandates, that tells us something about how professionals view it.
Let’s begin!
SFDR Article 6, 8 and 9 Funds
Before we move on to the methodology, a key part to understanding the authors’ approach is the Sustainable Finance Disclosure Regulation (SFDR) framework.
The SFDR is a set of rules introduced by the European Commission in 2019 requiring funds sold in Europe to disclose their sustainability approach.
The SFDR requires asset managers to classify their funds into different classes based on their sustainability objectives:
Article 9: Funds that have sustainable investment as their objective (dark green)
Article 8: Funds that promote environmental or social characteristics (light green)
Article 6: Funds without a sustainability scope
As the authors aimed to study the relationship between ESG scores and SFDR funds’ categorisation, one would naturally assume that “dark green funds” (i.e., Article 9 funds) would invest heavily in companies with better ESG scores since they have sustainable investment as their objective (spoiler alert: this is not the case - we will dive deeper later).
A follow-up goal, then, would be to investigate if we could use the holdings data of the “dark green funds” to infer companies with better sustainability characteristics? This also seems reasonable. After all, if more “dark green funds” own company X, the company must not be doing too badly in terms of promoting sustainable investing goals. A logical fund manager of a “dark green fund” would not want to load up on polluters or companies with subpar governance practices.
This thinking forms the core of the authors’ approach, which is to identify holdings that are over- or under-invested in SFDR 9 funds compared to other funds.
An interesting side note about SFDR: concerns about greenwashing in the first years after the implementation of the SFDR resulted in many funds that were initially labelled as Article 9 downgrading to Article 8 or 6. Another interesting point to note is that the SFDR is currently under revision and the Article 8 and 9 labels will be eventually replaced by two new categories: Sustainable Products and Transition Products. For the purpose of this study, however, we will still be looking at funds data with the SFDR 6, 8 and 9 labels.
Data and Methodology
The authors obtained quarterly holdings data for European-focused equity funds and ETFs from 2002-2023. Each fund is then associated with its respective SFDR label (SFDR 6, 8 or 9) as of 2023.
As a sharp-eyed reader, you may notice that the SFDR guidelines have only been introduced relatively recently, while the fund data goes back to 2002. Indeed, the authors introduced a caveat that the study is “retrospective, and assumes that the investment goals of funds currently labelled as SFDR 9 were characterised by a high level of sustainability also in the previous years.” Whether that is accurate or not may be debatable, but at least we now have a starting point.
Recall that the core of the authors’ approach is to identify which assets are over- or under-invested in SFDR 9 funds compared to other funds. The construction is as such: for each quarter, the authors compared the percentage of SFDR 9 funds that own a certain company X, relative to the percentage of SFDR 6 and 8 funds that own said company. The difference, then, represents the degree of over- or under-representation of certain assets in SFDR 9 funds.
For example, 86% of SFDR 9 funds own Siemens, compared to 56% of SFDR 6 and 8 funds, resulting in a difference of 30%, which makes it a reasonable assumption that in the eyes of SFDR 9 fund managers, owning Siemens does help to promote sustainable investment goals. The difference ranges from -1 (an extreme case when 0% of SFDR 9 funds own a company that is owned by 100% of other funds) to +1 (vice versa).
The authors called the derived difference the SMIS score (which stands for SFDR Market-Implied Sustainability score).
SMIS scores versus ESG scores
With the quarterly-computed SMIS scores, the authors can now compare these to ESG scores provided by an external data provider (for the study, ESG scores from Refinitiv Eikon were used).
The charts below show the SMIS score vs. the ESG score for three dates (in 2010, 2016 and 2022).
As mentioned above, other than some notable exceptions, we don’t see a positive association between the two. The plot seems noisy and we can observe plenty of companies with high ESG score that are under-represented in SFDR 9 funds and vice versa — over-representation of companies with below-average ESG score.
The authors attributed this mismatch to potentially “the differences in the goals and features considered by each scoring methodology” as well as “indicative of greenwashing practices by companies”. In simpler terms, these managers either do not follow the Refinitiv score or they have better BS detection systems in place that can tell when a company with high ESG score is greenwashing.
The authors also presented a table below showing the best and worst performers according to the SMIS score (left) and Refinitiv ESG score (right).
We see a few energy or mining companies with high ESG scores but are somewhat under-represented (e.g., Shell, TotalEnergies and Rio Tinto). These could be due to the exclusion policies that prevent these SFDR 9 funds from owning these companies.
Summary
The authors presented a novel way of calculating a “market-implied” ESG score by looking at holdings data of sustainable funds. The finding that there is a weak correlation between this market-implied score (SMIS) and a vendor-provided score contributed further to the notion that ESG data may be “noisy” and subjective and fund managers may have their own set of rules in choosing sustainable companies.
This disconnect is striking. Professional managers running funds with explicit sustainability mandates are not, in aggregate, buying the companies that score highest on vendor ESG ratings (Refinitiv was used as a benchmark in this paper, but one could easily imagine a similar outcome with other vendors).
Either these managers are using different criteria than the rating agencies, or they’re identifying cases where high ESG scores don’t reflect genuine sustainability. The authors can’t fully disentangle these explanations, but either interpretation suggests that vendor ESG scores capture something different from what sustainability-focused investors actually value.
One evidence of managers using different criteria (as pointed out by the authors) is that companies with high implied SMIS scores are typically involved in sectors or activities relevant for the green transition (such as utilities or technology companies), whereas companies with low implied SMIS scores are the ones in ‘controversial’ activities such as mining and fossil fuels.
For individual investors, the takeaway is nuanced. Optimising a portfolio for the highest vendor ESG score may not align with how professional sustainability managers actually allocate capital. That doesn’t mean ESG scores are useless, but it suggests they capture something different from the revealed preferences of managers with skin in the game.
This is definitely an interesting paper for me as a market-based score adds interesting information about how portfolio managers really think about a company. There is also an incentive dimension to this as managers are ‘voting’ on companies with their asset allocation decisions.
Once again, the link to the paper is here if you want to dive deeper1!
The authors also went to some length covering non-linear regression and portfolio tilting
This is a good proposition and one can envision its operationalization. I think a key challenge with ESG reporting is that most disclosures are not universally mandatory, leading to inconsistent participation. In addition, the lack of harmonization across rating agencies creates wide variation in how companies are evaluated.
Refinitiv's ESG scores are sector relative. Managers of sustainable funds may underweight or completely avoid certain sectors, contributing to the difference between Refinitiv's ESG company score and the SMIS score (see Enel, TotalEnergies, and Rio Tinto in the table above).