• Navigating the various ESG scores of mutual funds and ETFs from rating firms can be confusing.
• Pay attention to the metrics being measured, including materiality.
• "Material ESG issues are the ones which will cause a company a financial penalty if not handled correctly," says Jon Hale, head of sustainability research for the Americas at Morningstar.

Sustainability ratings are useful tools for assessing the environmental, sustainability and corporate governance-worthiness of mutual funds and exchange-traded funds.

But how do you navigate between different ratings providers, especially when they can have different results for the same fund?

CNBC looked under the hood at some of the most pertinent issues.

What's being measured?

"[Ratings] don't have to agree, because the underlying methodologies are different," said Larry Lawrence, executive director, ESG products, for MSCI.

To compare fund ratings in an apples-to-apples fashion, one must not only compare methodologies (here is MSCI's as an example), but also at how the underlying holdings are treated in terms of rating distribution; carbon intensity; transition risks to clean energy, such as companies' preparedness to do so; diversity metrics and so on, he said.

More from Impact Investing:
Climate funds hold less than 1% of 401(k) money. Here's why
BlackRock, $7 trillion money manager, puts climate at center of strategy
Here are some investing options for 2021 — with climate change in mind

Another thing to consider when comparing ESG ratings is materiality.

"Material ESG issues are the ones which will cause a company a financial penalty if not handled correctly," said Jon Hale, head of sustainability research for the Americas at Morningstar. "A typical sustainable fund may avoid investing in companies that don't rate highly on their handling of ESG issues, but the focus is on which issues are material to a particular company."

For example, he said, material issues for Exxon include green house gas emissions (environmental), community impact (social), and political lobbying and spending (governance). In the case of Facebook, social material issues have more to do with product and stakeholder impacts, diversity issues and data privacy.

Morningstar's fund sustainability ratings, illustrated as a scale of 1 to 5 globes, represent the aggregated material ESG risks of the companies in a fund's portfolio, Hale said.

He added that investors have recently begun to evaluate the ESG impact of sustainable funds, noting that more and more of these are issuing impact reports.

Other ratings providers focus on specific ESG issues right up front.

"We flag the companies that are burning down the Amazon, for example, [while other firms] are rating them," said Andy Behar, CEO of As You Sow, a non-profit in Berkeley, California, that performs shareholder advocacy focused on ESG issues. "We're splitting it apart issue by issue."

A fund could have a sustainable name but may own companies that run private prisons or produce banned weapons — and still perform sustainably, he said.

As You Sow rates funds according to performance on seven issues: fossil fuels, deforestation, military weapons, civilian weapons, gender, private prisons, and tobacco. The organization also features seven separate databases, highlighting each issue area, such as fossil-free funds or gender equality funds.

Some are concerned about the lack of verifiable ESG data being collected.

"Most rating firms are relying on company-reported data or secondary data, which can't be verified or audited," said Maneesh Sagar, CEO of RS Metrics, which provides specialized environmental data to ratings firms. "It can also be biased and outdated.

"Right now there is no primary data on the asset level," he added. "Moreover, raters tend to highlight whoever has the best reporting mechanisms."

Dealing with divergence

"Nobody does a great job of analyzing sustainability at the fund level," said Theresa Gusman, chief investment officer, First Affirmative Financial Network, headquartered in Colorado Springs, Colorado. "You'd be surprised at how different the scores could be from each vendor."

To deal with this, First Affirmative combines different ratings, equally weighted, from MSCI, Morningstar, Sustainalytics and their own proprietary score based on Corporate Knights data. This combined data enables Gusman to ask deeper questions of the fund managers; for example, when many small cap funds have low ESG scores because they report less data.

"It's expensive to the companies to report all these data — it's a lot of work — so the scoring gets really skewed,' she said. "The key is not to use the scores as the definitive answer to whether the fund is sustainable or not.

"The proliferation of data has made it far more difficult for individual investors to determine if a fund or ETF meets their individual impact preferences," Gusman added. "They should look at prospectuses in light of these questions."

ESG ratings from different providers disagree substantially, and these differences are driven both by what is measured and by how it is measured, according to an MIT Sloan School of Management working paper comparing results among six rating providers.

Furthermore, the report suggested that this can negatively influence adoption of ESG principles, stating that "[t]he divergence of the ratings disperses the effect of these preferences on asset prices. …[and] hampers the ambition of companies to improve their ESG performance, because they receive mixed signals from rating agencies about which actions are expected and will be valued by the market."

Standardizing ESG reporting

Disclosure on ESG issues is not standardized, and several experts referenced the work the Sustainable Accounting Standards Board is doing to address this. See the graphic below to see how they fit into the ESG research ecosystem.