Methodology
- Introduction
- Concept
- Criteria
- Universe
- Sources
- Screening & coding of information
- Quantifying information
- Calculating EthicalQuote and Rankings – 2009 Update
The methodology used by Covalence has been developed since 2001. The outcomes – curves measuring the ethical reputation of multinationals, EthicalQuote – have been published since the beginning of 2004 on Covalence website as well as through various publications.
Covalence started by considering the challenges that come when assessing the ethical performance of multinational enterprises (MNEs) in the modern world:
- Limited means of direct observation
- Restricted access to internal data
- Social complexity, cultural diversity, ethical pluralism, scientific uncertainty
To overcome these difficulties, Covalence innovates by using an information system for quoting the reputation of companies on ethical issues instead of a questionnaire-based rating scheme. Using media monitoring and the Internet allow to capture numerous, diverse information pieces that can be confronted in order to give a detailed, evolutionary picture of how companies are perceived.
EthicalQuote is an information system developed by Covalence. This online database measures the ethical reputation of multinational companies. Considering the challenges, a choice was made to consider the company as a black box, focus on information inflows and outflows and then construct an image of how companies are perceived, following systemic theories.
Covalence gathers information from numerous and various sources without normative a priori, recognizing that the modern world is characterized by social complexity, cultural diversity, ethical pluralism and scientific uncertainty.
This methodology is built around the notion of exchange of information, rooted in the classical economic model of offer and demand. Each document is coded regarding its orientation towards the company’s behavior in ethical terms: “ethical demand” (information on what the company should do for society) or “ethical offer” (information on what the company does for society). Ethical demand refers to information formulated by actors in public-interest organizations concerning the social or environmental consequences of a multinational’s activities (e.g. environmental impact of production). Ethical offer refers to corporate actions taken by multinationals to deliver on such demands (e.g. eco-innovative product).
Various documents coded as “ethical demand†or “ethical offer†are accounted and confronted, “offers†receiving + 1 and “demands†– 1. The EthicalQuote of companies emerges as a curve given by the cumulative addition of coded documents: when the curve is going up, the ethical reputation is improving, and vice versa.
Covalence has conceived 45 criteria for their capacity to capture various information on the contribution of multinational enterprises to human development globally, with particular attention paid to the realities and needs of developing countries. Rather than definitive moral judgments, the criteria should be seen as open boxes allowing to store and organize information on a barometer, case-by-case basis.
See: List of criteria; Long list of criteria with legal references.
In the process of setting the list of 45 criteria, Covalence has undertaken discussions with, and used feedback from the following NGOs: ADAP (Association pour le Développement des Aires Protégées), AGSI (Association Geste Solidaire Immédiat), GRAD (Groupe de Réalisation et d’Animation pour le développement). The creation of Covalence in 2001 benefited from the support of the Graduate Institute of Development Studies (Geneva, Switzerland).
The 45 criteria have the following characteristics:
- General legal framework consisting of six major international treaties
- Based on widely accepted principles, not on specific ethical choices, to cope with diversity and pluralism
- Capacity to cover changing aspects of companies’ operations
- Capacity to cover diverse actions led by stakeholders and media coverage
The 45 criteria are classified into 4 groups: I. Working conditions; II. Impact of production; III. Impact of product; IV. Institutional impact.
Covalence ethical ranking combines the EthicalQuote score (measure of popularity) and the scores calculated in each criteria group (measure of diversified performance).
Here is how the criteria groups are defined:
- I. Working Conditions
This group covers labor issues within the company as well as with subcontractors and suppliers, e.g. Labor Standards, Wages, Diversity or Social Benefits.
- II. Impact of Production
The second group covers direct and indirect effects of the company’s operations on people and the environment, e.g. Job creation, Fiscal contributions or Environmental Impact of Production.
- III. Impact of Product
The third group of criteria looks at the direct and indirect impact of the company’s products and services on people and the environment, e.g. Product Social Utility or Product Environmental Risk.
- IV. Institutional Impact
The last group of criteria aims at capturing information on the relations of companies with governments and communities, e.g. Human Rights Policy, United Nations Policy or Anti-corruption Policy.
Legal references of Covalence 45 criteria are: the Universal Declaration of Human Rights, the OECD Guidelines for Multinational Enterprises, the ILO Declaration of Principles concerning MNEs and Social Policy, the Rio Declaration on Environment and Development, the agreements of the World Summit for Social Development, the UN Global Compact, and the UN Millenium Goals.
Covalence criteria are not sector-specific. Rather, they are designed to cover any multinational company and to allow cross-sector comparisons.
Covalence covers a universe of 581 companies within 18 sectors following the Dow Jones Sector Titans.
> List of sectors and companies covered
> Universe
Covalence gathers online information using search engines, individual websites and correspondents:
Search engines
Search engines are the main providers of news agregated by Covalence. Google and Meltwater News are used to gather information from millions of potential sources among Companies, Media, Blogs, NGOs, Consultants, Trade Unions, International Organisations, Governments and Academia.
Individual websites
Covalence follows individual websites that regularly publish relevant content.
Correspondents
Covalence’s ethical quotation system allows interested parties to submit information, thus feeding the database and influencing the EthicalQuote of companies, in the spirit of a journal’s readers section. Information can be submitted using the contact form or by email to info@covalence.ch.
The objective is to integrate publicly available material with documents and opinions received from a variety of observers.
A study by SustainAbility (UK) and Mistra Foundation, “Values for Money: Reviewing the Quality of SRI Research†(2004), places Covalence on their short-list and mentions Covalence’s approach as among the best & most innovative practices for its “inclusive source model that allows stakeholders to include themselves in information gathering processes.â€
Covalence’s network of local informers in developing countries is supported by the Swiss Agency for Development and Cooperation.
Languages
Information is searched for in four languages: English, French, German, Spanish. The objective is to gather the largest quantity of relevant information with the spirit of journalistic investigation associated with scientific rigor.
Neutrality
Covalence does not see some sources as more reliable than others. Any source is considered equally. Covalence does not validate information sources, neither the content of information. What we do is collect, confront and synthesize the maximum of relevant documents from different sources. Our policy is to put ourselves in the position of an independent newspaper in front of statements, opinions, readers letters: publish any information provided it has relevance and an identified author, without endorsing its content.
6. SCREENING & CODING INFORMATION
Covalence methodology employs a bottom-up approach: information on specific issues and cases is selected, coded and entered into a database which produces aggregated results and an image of the company’s ethical performance as it is perceived.
The team of analysts at Covalence is composed of three regular members of staff, 8 to 10 in-house intern analysts and 10 distant intern analysts. Over the last years, hundreds students and young graduates from various countries have gathered and analyzed data for Covalence.
New analysts receive a training booklet presenting the work of collecting and coding relevant documents. There is a methodological argument in favor of the use of interns: multiple and temporary analysts reduce subjectivity biases in the process of gathering and coding qualitative information.
Covalence offers a variety of students and young graduates the possibility to directly apply their knowledge in social sciences as well as to learn more about Corporate Social Responsibility issues and case studies. See list of present and past intern analysts .
Intern analysts’ job consists in screening incoming news alerts and newsletters, selecting relevant documents, reading documents, identifying relevant quotations, coding positive or negative orientation, coding criteria and entering information into the database.
What do we mean by “relevant information�
- Information content must be related to 1 or 2 of the 45 Covalence “Criteria of business contribution to human development”. 1 criteria is used if there is little information (usually a short document), and 2 criteria are used if the document provides detailed information (longer document). For the purpose of coherence and stability, 2 criteria is the maximum allowed.
- Information must have an orientation to be coded as “ethical demand” (“information on what the company should do for societyâ€, negative news) or “ethical offer” (“information on what the company does for societyâ€, positive news). Explicit positive or negative words have to be found for demonstrating an orientation and allow the document to be coded and accounted. Examples of negative words: “predatorâ€, “underminingâ€. Examples of positive words: “contributingâ€, “helpingâ€.
- Information must have an identified source
- Information must be characterized by new content relatively to already entered material (simple copies don’t make it)
The work of collecting and coding documents is corrected and validated. The objective is to ensure documents are coded correctly and in a coherent way compared to work done previously.
Here is how relevant documents are accounted and weighed:
- A document gets 1 or 2 points, whether it is coded with 1 or 2 criteria.
- A document gets a positive or a negative sign according to the orientation coded by the operator. “Ethical demands†are negative, “ethical offers†are positive. So documents may weigh -1, -2, +1 or +2.
- A document may be entered twice if both orientations are found: for example, the same press article can be coded once as an offer with criteria 3 and 24 (weigh: +2), and once as a demand with criteria 34 (weigh:-1). The cumulated weigh of the document will therefore be +1.
In order to favor the diversity of sources and issues covered by Covalence, a limit has been established at a maximum of 5 points per company per criteria per source per orientation per year.
Covalence follows a principle of equal weighing of individual sources. The “size†of source (audience, quantity of readers / viewers) is not taken as a weighing criteria, neither is placement in print press.
Why is Covalence applying a principle of equal weighing of individual sources?
Following are our arguments for applying such an equal weighing approach:
a) The modern world is characterized by social complexity, cultural diversity, ethical pluralism and scientific uncertainty: considering “small†sources at the same level as “large†ones is a way to cope with such complexity and diversity.
b) It is technically difficult to measure the size, or popularity of sources and find a weighing factor for such an heterogeneous ensemble of sources as large medias, specialized NGOs, individual correspondents and multinational companies’ headquarters.
c) Western and Anglo Saxon sources are overrepresented in Covalence database, because such sources are more numerous online and are more easily accessible than others. Applying a weighing factor could amplify the already existing overrepresentation of Western and Anglo Saxon sources.
d) Google alerts that Covalence uses only cover pages with the highest popularity according to Google (page rank): we naturally select the most popular pages.
e) Echoes, repetition make weigh. Often one particular issue is covered by different sources. This produces several points in Covalence database, and this is how a weighing process is naturally working: the system measures the noise made by news, the echoes generated by a story among numerous sources. Rather than one particular document, it is the aggregation of a large number of documents that gives a significant picture of reality.
8. CALCULATING ETHICALQUOTE AND RANKINGS – Methodological Update January 2009
In January 2009 Covalence is implementing a renovated methodology for calculating its EthicalQuote score and rankings. Three innovations are introduced: an ethical rate to control the effect of size; an erosion factor to control the effect of time; and the use of criteria groups to control the effect of trends.
These innovations are meant to allow a better comparison of companies of different sizes, to offer a more dynamic measure of ethical reputation, and to get closer to the real ethical performance of companies, which however remain considered an invisible, inaccessible reality (black box).
Universe and benchmarking: from 200 in 2001 to 540+ in 2009
From 2002 to 2007 Covalence has been monitoring 200 companies within 10 sectors. Those companies would be ranked according to their EthicalQuote, an absolute measure of ethical popularity given by the cumulative addition of positive news (+1) and negative news (-1). Results have shown that the biggest companies would attract most of the news count, both positive and negative, and therefore be ranked as leaders or losers in their sector. Smaller companies would register smaller amounts of news and occupy intermediate ranks.
In 2008, Covalence has extended its focus to 540+ companies in 18 broad sectors (~30 companies per sector). The enlargement of Covalence universe of analysis is taking effect in 2009.
Calculating the EthicalQuote
Covalence has renovated the EthicalQuote calculation methodology for allowing the inclusion and benchmarking of a larger number of companies with a reduced size bias. This is achieved by introducing an ethical rate.
The score is the aggregation of positive and negative news and translates as following:
S = A – B
S = Score
A = Positive news (ethical offers)
B = Negative news (ethical demands)
Ethical rate: controlling the effect of size
The rate (or ethical rate) is an expression of the net ethical performance of the company. It’s the score confronted to the overall volume of news affecting the company; it translates like this:
R = S / V
R = Rate
S = Score = A –B
V = Volume = A + B
While the rate provides a relative, qualitative measure of the ethical reputation of a company, the score offers an absolute, quantitative view. How do you compare two companies with close scores but broad differences in ethical rates?
To solve this problem and allow the most tangible benchmarking, a rate-adjusted score has been created. What is a score of + 100? In order to validate these 100 points, the rate is used. The logic behind it is simple and striking. To fully validate a score of + 100, a rate of 100% is needed. As a consequence, a rate of 50% would halve the score to + 50, and so on.
The rate adjusted score calculates as following:
raS = S * ¦R¦
raS = rate adjusted Score
R = Rate
S is multiplied by the absolute value of R. S has always the same sign (-/+) as R. There cannot be a positive score with a negative rate, and vice versa, on a given unit of time (week, month, year).
Erosion factor: controlling the effect of time
When founded in 2001, Covalence has identified a lack of Corporate Social Responsibility (CSR)-related information in the media. As a result, it has become a mission and a social utility of the company to promote corporate social and environmental news through its business model, products (EthicalQuote, reports) and communication tools (website, newsletters, rankings).
As a small volume of information was registered, the value of information was considered as persistent; in other words, information published five years ago would have the same weight as information published in the current morning newspaper.
Today, CSR has gained space in the media and has become a very important domain of corporate and stakeholders’ communications. The volume of information within the EthicalQuote database has grown at a 15% yearly rate since its inception in 2001. Covalence has also grown with it and in time, has come to reconsider the weight of past information in the calculation of EthicalQuote scores. This is being done by using an erosion factor.
The erosion factor (or value discount on past news) is a means to give less importance to old news than to more recent news – positive news (company PR, initiatives & reporting) and negative news (criticisms, crisis). The objective is to offer a more dynamic measure of ethical reputation allowing companies to progress while dissuading them to “sleep on their laurelsâ€.
As a convention the factor of erosion has been set at 2% per month. A sixty months period (or 5 years) basis has led to 1.66% rate of erosion; this has been rounded to 2%. For instance, 100 points would be valued 80 after 12 months, 49 after 36 months, 30 after 60 months and 10 after 120 months or 10 years.
Here is how the EthicalQuote will form as of 2009:
EQ(t) = [S * ¦R¦] + [EQ(t-1)*{1-E}]
EQ = EQ(t) = EthicalQuote S = Score R = Rate E = Erosion factor
Calculating Covalence Ethical Rankings
As described above the ‘new’ EthicalQuote formula incorporates the notion of rate, allowing to adjust the quantitative score with a qualitative rate or measure of ethical performance and thus to control the effect of size. The EthicalQuote formula also integrates an erosion factor allowing to control the effect of time.
Up to 2008 Covalence has been calculating its Ethical Rankings (annual, quarterly, weekly) using the EthicalQuote score without considering differences among criteria and criteria groups. A company performing strongly on certain criteria could compensate weaknesses registered in other criteria and register a high ranking. This measure is very sensitive to trends, fashion and campaigns (e.g. the environment, climate change).
Criteria groups: controlling the effect of trends
News items are classified according to 45 criteria embedded in 4 groups. The four groups of criteria are: A. Working Conditions, B. Impact of Production, C. Impact of Product and D. Institutional Impact.
In our view an ethical ranking should favor companies that show good social and environmental performance in various domains, not only regarding a few hot topics. The effect of trends should be controlled, while a balanced approach of sustainability should be encouraged.
From 2009 on Covalence Rankings combine the EthicalQuote (measure of popularity) and the scores calculated in each criteria group (measure of diversified performance).
The formula used for calculating Covalence Ethical Rankings is the following:
Ethical Ranking = R RankGroup = Rg Rank EthicalQuote = Req
A = Group A B = Group B C = Group C D = Group D
Rg (A,B,C,D); highest EthicalQuote ranks #1, lowest EthicalQuote #n;
Rg = [Rg(A) + Rg(B) + Rg(C) + Rg(D)] / 4 ; lowest Rg ranks #1, highest Rg ranks #n;
Req; highest ranks #1, lowest #n;
R = (Rg + Req) / 2; lowest ranks #1, highest ranks #n.