Rankings are a beautiful thing. They’re important in so many different arenas, from helping us figure out what to buy on Amazon to informing policy decisions of governments. Our discussion here will lean much more toward the latter case. I hope that the case study that follows sheds light on some of the ways statistics are used to determine rankings. Indeed, a commenter on a previous blog entry of mine expressed some interest in this subject.
In the last decade, the United Arab Emirates (UAE) economy has thrived and grown in what can only be described as extraordinary fashion (literally and figuratively). The Arabian Gulf country, thus, takes its "competitiveness" in the international arena quite seriously. Recently they established a Ministry of Happiness, but for many years the UAE's Federal Competitiveness and Statistics Authority (FCSA) has been working hard to inform the government on how to align the country's progression with a vision of "long term prosperity with a balance between productivity and quality of life for the nation."
Above is a figure generated by the FCSA. It displays the UAE's world rankings over the years as generated by a number of reports and indices that measure the strength of certain sectors of a society, such as trade (WEF-GETR), information technology (WEF-GITR), and travel and tourism (WEF-TTR). Some reports measure competitiveness directly, like the WEF-GCR (solid green dots on the graph), and it is this report I would like to focus on here. How can "competitiveness," being a concept, be statistically measured?
WEF-GCR, or World Economic Forum - Global Competitiveness Report, provides an abridged methodology, as well as a detailed methodology in the report's appendix, describing their approach to constructing their report for each country. They combine 114 indicators (measures of different concepts) that matter for productivity, grouping them into numerous categories that comprise the twelve pillars in the figure above. The pillars are distributed over a hierarchy of three sub-indices, in line with three main stages of development. For example, Pillar 1 "Institutions" comprises 21 indicators such as property rights, irregular payments and bribes, efficiency of legal framework in settling disputes, and protection of minority shareholders’ interests. Already we can see how complex the report gets; how is it that 114 indicators like these are quantitatively measured?
The report methodology states that its main source of measurements is from answers from the Executive Opinion Survey (EOS). The EOS is a survey that the WEF conducts annually, collecting information on a broad range of socio-economic issues. The key point is that the respondents to the survey comprise one class of individuals only: business executives. They answer questions such as:
In your country, how strong is the protection of property rights, including financial assets? [1 = extremely weak; 7 = extremely strong]. (Property rights)
In your country, how common is it for firms to make undocumented extra payments or bribes in connection with (a) imports and exports; (b) public utilities; (c) annual tax payments; (d) awarding of public contracts and licenses; (e) obtaining favorable judicial decisions? In each case, the answer ranges from 1 [very common] to 7 [never occurs]. (Irregular payments and bribes)
In your country, how efficient is the legal framework for private businesses in settling disputes? [1 = extremely inefficient; 7 = extremely efficient]. (Efficiency of legal framework in settling disputes)
In your country, to what extent are the interests of minority shareholders protected by the legal system? [1 = not protected at all; 7 = fully protected]. (Protection of minority shareholders' interests)
In the Technical Notes and Sources for the GCR, the WEF further states: "Indicators that are not derived from the Survey are sourced from international agencies and national authorities." And in the appendix, "The computation of the GCI is based on successive aggregations of scores from the indicator level (i.e., the most disaggregated level) all the way up to the overall GCI score. Unless noted otherwise, we use an arithmetic mean to aggregate individual indicators within a category."
So generally, a measurement is calculated for each indicator based the arithmetic mean of all the answers, a score is calculated for each category based on the arithmetic mean of all its indicators' measurements, a score is calculated for each pillar based on the arithmetic mean of all its categories' scores, and a score is calculated for each sub-index based on the arithmetic mean of all its pillars' scores. Categories and pillars are weighted based on how many others there are in the group (e.g. 4 categories in a pillar = each category is weighed at 25%, regardless of the number of indices it is comprised of). These are fixed weights; however, the weight put on each of the three sub-indices (basic requirements, efficiency enhancers, and innovation and sophistication factors) is not fixed:
This is really only a short summary explaining some of the main points of how the World Economic Forum conducts its Global Competitiveness Report. There are several other reports that may be explored, many taking a similar approach in terms of having weighted pillars, categories, etc. Is the approach we've seen here the perfect way to calculate "competitiveness?" What I would think is: how can something so complex be perfect? But it is an approach that no doubt takes advantage of the best survey and statistical methods we have developed. I should not fail to mention that the GCR has been rigorously tested for statistical validity, and it holds astonishingly well in that regard.
But people in the United Arab Emirates don't need to worry about all this. All they have to do is live in their villas, play with their pet lions, and drive their Ferraris on what the WEF-GCR says are the highest quality roads in the world.