Empirical Project 4 Working in R

Download the code

To download the code chunks used in this project, right-click on the download link and select ‘Save Link As…’. You’ll need to save the code download to your working directory, and open it in RStudio.

Don’t forget to also download the data into your working directory by following the steps in this project.

R-specific learning objectives

In addition to the learning objectives for this project, in this section you will learn how to convert (‘reshape’) data from wide to long format and vice versa.

Getting started in R

For this project you will need the following packages:

You will also use the ggplot2 package to produce accurate graphs, but that comes as part of the tidyverse package.

If you need to install these packages, run the following code:

install.packages(c("readxl", "tidyverse", "reshape2"))

You can import these libraries now, or when they are used in the R walk-throughs below.

library(readxl)
library(tidyverse)
library(reshape2)

Part 4.1 GDP and its components as a measure of material wellbeing

Learning objectives for this part

  • check datasets for missing data
  • sort data and assign ranks based on values
  • distinguish between time series and cross sectional data, and plot appropriate charts for each type of data.

The GDP data we will look at is from the United Nations’ National Accounts Main Aggregates Database, which contains estimates of total GDP and its components for all countries over the period 1970 to present. We will look at how GDP and its components have changed over time, and investigate the usefulness of GDP per capita as a measure of wellbeing.

To answer the questions below, download the data and make sure you understand how the measure of total GDP is constructed.

R walk-through 4.1 Importing the Excel file (.xlsx or .xls format) into R

First, use setwd to tell R where the datafile is stored. To avoid having to repeatedly use setwd to tell R where your files are, keep all the files you need in that folder, including the Excel sheet you just downloaded. Replace ‘YOURFILEPATH’ with the full filepath which points to the folder with your datafile. If you don’t know how to find the path to that folder, see the ‘Technical Reference’ section.

setwd("YOURFILEPATH")

Then use the function readxl (part of the tidyverse suite of packages) to import the datafile. Before importing the file into R, open the file in Excel to see how the data is organized in the spreadsheet, and note that:

  • There is a heading that we don’t need, followed by a blank row.
  • The data we need starts on row three.
## # A tibble: 6 x 50
##   CountryID Country  IndicatorName      `1970` `1971` `1972` `1973` `1974`
##                                   
## 1         4 Afghani~ Final consumption~ 5.56e9 5.33e9 5.20e9 5.75e9 6.15e9
## 2         4 Afghani~ Household consump~ 5.07e9 4.84e9 4.70e9 5.21e9 5.59e9
## 3         4 Afghani~ General governmen~ 3.72e8 3.82e8 4.02e8 4.21e8 4.31e8
## 4         4 Afghani~ Gross capital for~ 9.85e8 1.05e9 9.19e8 9.19e8 1.18e9
## 5         4 Afghani~ Gross fixed capit~ 9.85e8 1.05e9 9.19e8 9.19e8 1.18e9
## 6         4 Afghani~ Exports of goods ~ 1.12e8 1.45e8 1.73e8 2.18e8 3.00e8
## # ... with 42 more variables: `1975` , `1976` , `1977` ,
## #   `1978` , `1979` , `1980` , `1981` , `1982` ,
## #   `1983` , `1984` , `1985` , `1986` , `1987` ,
## #   `1988` , `1989` , `1990` , `1991` , `1992` ,
## #   `1993` , `1994` , `1995` , `1996` , `1997` ,
## #   `1998` , `1999` , `2000` , `2001` , `2002` ,
## #   `2003` , `2004` , `2005` , `2006` , `2007` ,
## #   `2008` , `2009` , `2010` , `2011` , `2012` ,
## #   `2013` , `2014` , `2015` , `2016` 
  1. You can see from the tab ‘Download-GDPconstant-USD-countr’ that some countries have missing data for some of the years. Data may be missing due to political reasons (for example, countries formed after 1970) or data availability issues.
Country Number of years of GDP data
   
   
   
   

Number of years of GDP data available for each country.

Figure 4.1 Number of years of GDP data available for each country.

 

R walk-through 4.2 Making a frequency table

We want to create a table showing how many years of Final consumption expenditure data are available for each country.

Looking at the dataset’s current format, you can see that countries and indicators (for example, Afghanistan and Final consumption expenditure) are row variables, while year is the column variable. This data is organized in ‘wide’ format (each individual’s information is in a single row).

For many data operations and making charts it is more convenient to have indicators as column variables, so we would like Final consumption expenditure to be a column variable, and year to be the row variable. Each observation would represent the value of an indicator for a particular country and year. This data is organized in ‘long’ format (each individual’s information is in multiple rows).

To change data from wide to long format, we use the melt command from the package reshape2. The melt command is very powerful and useful, as you will find many large datasets are in wide format. In this case, it takes the data in Column 4 to the last column (these columns indicate the years) and uses them to create two new columns: one column (variable) contains the name of the row variable (the year) and the other column (value) contains the associated value. Compare long_UN to wide_UN to understand how the melt command works. To learn more about organizing data in R, see the R for Data Science website.

library(reshape2)

wide_UN  UN
# Keep all data except for column 1 (CountryID)
wide_UN = wide_UN[, -1]

# id.vars are the names of the column variables.
long_UN = 
  melt(wide_UN, id.vars = c("Country", "IndicatorName"),
  value.vars = 4:ncol(UN))

head(long_UN)
##       Country
## 1 Afghanistan
## 2 Afghanistan
## 3 Afghanistan
## 4 Afghanistan
## 5 Afghanistan
## 6 Afghanistan
##   IndicatorName
## 1 Final consumption expenditure
## 2 Household consumption expenditure (including Non-profit institutions serving households)
## 3 General government final consumption expenditure
## 4 Gross capital formation
## 5 Gross fixed capital formation (including Acquisitions less disposals of valuables)
## 6 Exports of goods and services
##   variable      value
## 1     1970 5559066266
## 2     1970 5065088737
## 3     1970  372478456
## 4     1970  984580895
## 5     1970  984580895
## 6     1970  112390156

Our new ‘long’ format dataset is called long_UN. During the reshaping process, a new variable called variable was created which contains years. We will use the names function to rename it as Year.

names(long_UN)[names(long_UN) == "variable"]  "Year"

To create the required table, we only need Final consumption expenditure of each country, which we extract using the subset function.

cons = subset(long_UN,
  IndicatorName == "Final consumption expenditure")

Now we create the table showing the number of missing years by country, using the piping operator (%>%) from the tidyverse package. This operator allows us to perform multiple functions, one after another.

# Use the pipe operator (%>%) from the tidyverse package.
# This means: use the result of the current line
# as the first argument in the next line's function.

missing_by_country = cons %>%
  group_by(Country) %>%
  summarize(available_years=sum(!is.na(value))) %>%
  print()
## # A tibble: 220 x 2
##    Country             available_years
##                             
##  1 Afghanistan                      47
##  2 Albania                          47
##  3 Algeria                          47
##  4 Andorra                          47
##  5 Angola                           47
##  6 Anguilla                         47
##  7 Antigua and Barbuda              47
##  8 Argentina                        47
##  9 Armenia                          27
## 10 Aruba                            47
## # ... with 210 more rows

Translating the code in words: Take the variable cons (cons %>%) and group the observations by country (group_by(Country)), then take this result (%>%) and produce a table (summarize(...)) that shows the variable available_years (which is the sum (sum(...)) of the variable !is.na(value)).

To understand what !is.na(value) means, recall that value contains the numerical values for the variable of interest. When an observation is missing, it is recorded as NA. The function is.na(value) will return a value of 1 (or TRUE) if the value is missing and 0 (or FALSE) otherwise. We add a ! in front since we want the function to return a 1 if the observation exists and a 0 otherwise. For R, ! means ‘not’ so we get a 1 if the particular observation is not missing.

Now we can establish how many of the 220 countries in the dataset have complete information. A dataset is complete if it has the maximum number of available observations (max(missing_by_country$available_years)).

## [1] 179

If you add up the data on the right-hand side of this equation, you may find that it does not add up to the reported GDP value. The UN notes this discrepancy in Section J, item 17 of the ‘Methodology for the national accounts’: ‘The sums of com­ponents in the tables may not necessarily add up to totals shown because of rounding’.

There are three different ways in which countries calculate GDP for their national accounts, but we will focus on the expenditure approach, which calculates gross domestic product (GDP) as:

Gross capital formation refers to the creation of fixed assets in the economy (such as the construction of buildings, roads, and new machinery) and changes in inventories (stocks of goods held by firms).

  1. Rather than looking at exports and imports separately, we usually look at the difference between them (exports minus imports), also known as net exports. Choose three countries that have GDP data over the entire period (1970 to the latest year available). For each country, create a variable that shows the values of net exports in each year.

R walk-through 4.3 Creating new variables

We will use Brazil, the US, and China as examples.

Before we select these three countries, we will calculate the net exports (exports minus imports) for all countries, as we need that information in R walk-through 4.4. We will also shorten the names of the variables we need, to make the code easier to read.

# Shorten the names of the variables we need
# When a string straddles two lines of code
# we need to wrap it into the 'strwrap' function

long_UN$IndicatorName[long_UN$IndicatorName == 
  strwrap("Household consumption expenditure (including 
    Non-profit institutions serving households)")]  
  "HH.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "General government final consumption expenditure"]  
  "Gov.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Final consumption expenditure"] 
  "Final.Expenditure"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Gross capital formation"] 
  "Capital"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Imports of goods and services"] 
  "Imports"

long_UN$IndicatorName[long_UN$IndicatorName == 
  "Exports of goods and services"] 
  "Exports"

long_UN still has several rows for a particular country and year (one for each indicator). We will reshape this data using the dcast function to ensure that we have only one row per country and per year. We then add a new column called Net.Exports containing the calculated net exports.

# We need to cast (reshape) the long_UN data to a dataframe
# We use the dcast function (used for dataframes)
table_UN  dcast(long_UN, Country + Year ~ IndicatorName)

# Add a new column for net exports (= exports – imports)
table_UN$Net.Exports  
  table_UN[, "Exports"]-table_UN[, "Imports"]

Let us select our three chosen countries to check that we calculated net exports correctly.

sel_countries = c("Brazil", "United States", "China")

# Using our long format dataset, we get imports, exports, 
# and year for these countries.
sel_UN1 = subset(table_UN, 
  subset = (Country %in% sel_countries), 
  select = c("Country", "Year", "Exports",
    "Imports", "Net.Exports"))

head(sel_UN1)
##      Country Year     Exports     Imports  Net.Exports
## 1223  Brazil 1970 12337240060 20187929130  -7850689070
## 1224  Brazil 1971 13016975734 24162976191 -11146000457
## 1225  Brazil 1972 16162334455 29025977711 -12863643256
## 1226  Brazil 1973 18466228366 34950588132 -16484359766
## 1227  Brazil 1974 18897150611 44821362355 -25924211744
## 1228  Brazil 1975 21084064715 42838470795 -21754406080

Now we will create charts to show the GDP components in order to look for general patterns over time and make comparisons between countries.

  1. Evaluate the components over time, for two countries of your choice.

R walk-through 4.4 Plotting and annotating time series data

Extract the relevant data

We will work with the long_UN dataset, as the long format is well suited to produce charts with the ggplot package. In this example, we use the US and China (saved as the dataset comp).

# Select our chosen countries
comp = subset(long_UN, 
  Country %in% c("United States", "China"))

# value in billion of USD
comp$value = comp$value / 1e9

comp = subset(comp, 
  select = c("Country", "Year",
    "IndicatorName", "value"),
  subset = IndicatorName %in% c("Gov.Expenditure", 
  "HH.Expenditure", "Capital", "Imports", "Exports"))

Plot a line chart

We can now plot this data using the ggplot library.

library(ggplot2)

# ggplot allows us to build a chart step-by-step.
pl = ggplot(subset(comp, Country == "United States"),
  # Base chart, defining x (horizontal) and y (vertical) 
  # axis variables
  aes(x = Year, y = value))

# Specify a line chart, with a different colour for each 
# indicator name and line size = 1
pl = pl + geom_line(aes(group = IndicatorName, 
  color = IndicatorName), size = 1)

# Display the chart
pl

The US’s GDP components (expenditure approach).

Figure 4.2 The US’s GDP components (expenditure approach).

There are plenty of problems with this chart:

  • we cannot read the horizontal axis, because it labels every year
  • the vertical axis label is uninformative
  • there is no chart title
  • the grey (default) background makes the chart difficult to read
  • the legend is uninformative.

To improve this chart, we add features to the already existing figure pl.

pl = pl + scale_x_discrete(breaks=seq(1970, 2016, by = 10))
pl = pl + scale_y_continuous(name="Billion US$")
pl = pl + ggtitle("GDP components over time")

# Change the legend title and labels
pl = pl + scale_colour_discrete(name = "Components of GDP",
  labels = c("Gross capital formation", "Exports",
    "Government expenditure", "Household expenditure",
    "Imports")) 

pl = pl + theme_bw()

pl = pl + annotate("text", x = 37, y = 850,
  label = "Great Recession")

pl

The US’s GDP components (expenditure approach), amended chart.

Figure 4.3 The US’s GDP components (expenditure approach), amended chart.

We can make a chart for more than one country simultaneously by repeating the code above, but without subsetting the data:

# Repeat all steps without subsetting the data

# Base line chart
pl = ggplot(comp, aes(x = Year, y = value, 
  color = IndicatorName))
pl = pl + geom_line(aes(group = IndicatorName), size = 1)
pl = pl + scale_x_discrete(
  breaks = seq(1970, 2016, by = 10))
pl = pl + scale_y_continuous(name = "Billion US$")
pl = pl + ggtitle("GDP components over time")
pl = pl + scale_colour_discrete(name = "Component")   
pl = pl + theme_bw()

# Make a separate chart for each country
pl = pl + facet_wrap(~Country)
pl = pl + scale_colour_discrete(
  name = "Components of GDP",   
  labels = c("Gross capital formation",  
    "Exports",
    "Government expenditure",
    "Household expenditure",
    "Imports"))
pl

GDP components over time (China and the US).

Figure 4.4 GDP components over time (China and the US).

  1. Another way to visualize the GDP data is to look at each component as a proportion of total GDP. Use the same countries that you chose for Question 3.

R walk-through 4.5 Calculating new variables and plotting time series data

Calculate proportion of total GDP

We will use the comp dataset created in R walk-through 4.4. First we will calculate net exports, as that contributes to GDP. As the data is currently in long format, we will reshape the data into wide format so that the variables we need are in separate columns instead of separate rows (using the dcast function, as in R walk-through 4.3), calculate net exports, then transform the data back into long format using the melt function.

# Reshape the data to wide format (indicators in columns)
comp_wide  dcast(comp, Country + Year ~ IndicatorName)

head(comp_wide)
##   Country Year  Capital   Exports Gov.Expenditure HH.Expenditure   Imports
## 1   China 1970 67.58221  5.305242        19.30034       107.8411  6.119649
## 2   China 1971 73.79977  6.318662        22.63929       112.3586  6.094361
## 3   China 1972 70.62638  7.740001        23.77126       118.3906  7.510857
## 4   China 1973 80.79658 10.810644        24.34177       126.7546 11.540975
## 5   China 1974 83.38207 12.856408        26.14306       129.3434 16.041397
## 6   China 1975 93.47130 13.309628        27.29335       134.5575 15.859461
# Add the new column for net exports = exports – imports
comp_wide$Net.Exports  
  comp_wide[, "Exports"] - comp_wide[, "Imports"]

head(comp_wide)
##   Country Year  Capital   Exports Gov.Expenditure HH.Expenditure   Imports
## 1   China 1970 67.58221  5.305242        19.30034       107.8411  6.119649
## 2   China 1971 73.79977  6.318662        22.63929       112.3586  6.094361
## 3   China 1972 70.62638  7.740001        23.77126       118.3906  7.510857
## 4   China 1973 80.79658 10.810644        24.34177       126.7546 11.540975
## 5   China 1974 83.38207 12.856408        26.14306       129.3434 16.041397
## 6   China 1975 93.47130 13.309628        27.29335       134.5575 15.859461
##   Net.Exports
## 1  -0.8144069
## 2   0.2243011
## 3   0.2291447
## 4  -0.7303314
## 5  -3.1849891
## 6  -2.5498330
# Return to long format with the HH.expenditure, Capital, and Net Export variables
comp2_wide  
  subset(comp_wide, select = -c(Exports, Imports))

comp2 
  melt(comp2_wide, id.vars = c("Year", "Country"))

Now we will create a new dataframe (props) also containing the proportions for each GDP component (proportion), using the piping operator to link functions together.

props = comp2 %>%
  group_by(Country, Year) %>%
  mutate(proportion = value / sum(value))

In words, we did the following: Take the comp2 dataframe and create groups by country and year (for example, all indicators for China in 1970). Then create a new variable (mutate) called proportion, which divides the variable value of an indicator by the sum of all value for that group (for example, all indicators for China in 1970). The result is then saved in props. Look at the props dataframe to confirm that the above command has achieved the desired result.

Plot a line chart

Now we redo the line chart from R walk-through 4.4 using the variable props.

# Base line chart
pl = ggplot(props, aes(x = Year, y = proportion, 
  color = variable))

pl = pl + geom_line(aes(group = variable), 
  size = 1)

pl = pl + scale_x_discrete(breaks = seq(1970, 2016,
  by = 10))

pl = pl + ggtitle("GDP component proportions over time")

pl = pl + theme_bw()

# Make a separate chart for each country
pl = pl + facet_wrap(~Country)

pl = pl + scale_colour_discrete(
  name = "Components of GDP",
  labels = c("Gross capital formation",
    "Government expenditure",
    "Household expenditure", 
    "Net Exports"))

pl
time series data
A time series is a set of time-ordered observations of a variable taken at successive, in most cases regular, periods or points of time. Example: The population of a particular country in the years 1990, 1991, 1992, … , 2015 is time series data.
cross-sectional data
Data that is collected from participants at one point in time or within a relatively short time frame. In contrast, time series data refers to data collected by following an individual (or firm, country, etc.) over a course of time. Example: Data on degree courses taken by all the students in a particular university in 2016 is considered cross-sectional data. In contrast, data on degree courses taken by all students in a particular university from 1990 to 2016 is considered time series data.

GDP component proportions over time (China and the US).

Figure 4.5 GDP component proportions over time (China and the US).

So far, we have done comparisons of time series data, which is a collection of values for the same variables and subjects, taken at different points in time (for example, GDP of a particular country, measured each year). We will now make some charts using cross-sectional data, which is a collection of values for the same variables for different subjects, usually taken at the same time.

  1. Choose three developed countries, three countries in economic transition, and three developing countries (for a list of these countries, see Tables A–C in the UN country classification document).

R walk-through 4.6 Creating stacked bar charts

Calculate proportion of total GDP

This walk-through uses the following countries (chosen at random):

  • developed countries: Germany, Japan, United States
  • transition countries: Albania, Russian Federation, Ukraine
  • developing countries: Brazil, China, India.

The relevant data are still in the table_UN dataframe. Before we select these countries, we first calculate the required proportions for all countries.

# Calculate proportions
table_UN$p_Capital  
  table_UN$Capital / 
  (table_UN$Capital +
    table_UN$Final.Expenditure +
    table_UN$Net.Exports)

table_UN$p_FinalExp 
  table_UN$Final.Expenditure / 
  (table_UN$Capital +
    table_UN$Final.Expenditure +
    table_UN$Net.Exports)

table_UN$p_NetExports 
  table_UN$Net.Exports /
  (table_UN$Capital +
    table_UN$Final.Expenditure + 
    table_UN$Net.Exports)

sel_countries  
  c("Germany", "Japan", "United States",
    "Albania", "Russian Federation", "Ukraine",
    "Brazil", "China", "India")

# Using our long format dataset, we select imports, 
# exports, and year for our chosen countries in 2015.

# Select the columns we need
sel_2015  
  subset(table_UN, subset =
    (Country %in% sel_countries) & (Year == 2015),
    select = c("Country", "Year", "p_FinalExp",
      "p_Capital", "p_NetExports"))

Plot a stacked bar chart

Now let’s create the bar chart.

# Reshape the table into long format, then use ggplot
sel_2015_m  melt(sel_2015, id.vars = 
  c("Year", "Country"))

g  ggplot(sel_2015_m, 
  aes(x = Country, y = value, fill = variable)) + 
  geom_bar(stat = "identity") + coord_flip() +
  ggtitle("GDP component proportions in 2015") +
  scale_fill_discrete(name = "Components of GDP",
  labels = c("Final expenditure",
    "Gross capital formation",
    "Net Exports")) +
  theme_bw()

plot(g)

GDP component proportions in 2015.

Figure 4.6 GDP component proportions in 2015.

Note that even when a country has a trade deficit (proportion of net exports < 0), the proportions will add up to 1, but the proportions of final expenditure and capital will add up to more than 1.

We have not yet ordered the countries so that they form the pre-specified groups. To achieve this, we need to explicitly impose an ordering on the Country variable using the factor function. The countries will be ordered in the same order we used to define sel_countries.

# Impose the order in the sel_countries object, then use ggplot
sel_2015_m$Country  
  factor(sel_2015_m$Country, levels = sel_countries)

g  ggplot(sel_2015_m, 
  aes(x = Country, y = value, fill = variable)) +
  geom_bar(stat = "identity") + coord_flip() + 
  ggtitle("GDP component proportions in 2015 (ordered)") + 
  scale_fill_discrete(name = "Components of GDP", 
  labels = c("Final expenditure",
    "Gross capital formation",
    "Net Exports")) +
  labels = c("Final expenditure",
    "Gross capital formation",
    "Net Exports")) +
  theme_bw()

plot(g)

GDP component proportions in 2015 (ordered).

Figure 4.7 GDP component proportions in 2015 (ordered).

  1. GDP per capita is often used to indicate material wellbeing instead of GDP, because it accounts for differences in population across countries. Refer to the following articles to help you to answer the questions:

Part 4.2 The HDI as a measure of wellbeing

Learning objectives for this part

  • sort data and assign ranks based on values
  • distinguish between time series and cross sectional data, and plot appropriate charts for each type of data
  • calculate the geometric mean and explain how it differs from the arithmetic mean
  • construct indices using the geometric mean, and use index values to rank observations
  • explain the difference between two measures of wellbeing (GDP per capita and the Human Development Index).

In Part 4.1 we looked at GDP per capita as a measure of material wellbeing. While income has a major influence on wellbeing because it allows us to buy the goods and services we need or enjoy, it is not the only determinant of wellbeing. Many aspects of our wellbeing cannot be bought, for example, good health or having more time to spend with friends and family.

We are now going to look at the Human Development Index (HDI), a measure of wellbeing that includes non-material aspects, and make comparisons with GDP per capita (a measure of material wellbeing). GDP per capita is a simple index calculated as the sum of its elements, whereas the HDI is more complex. Instead of using different types of expenditure or output to measure wellbeing or living standards, the HDI consists of three dimensions associated with wellbeing:

We will first learn about how the HDI is constructed, and then use this method to construct indices of wellbeing according to criteria of our choice.

The HDI data we will look at is from the Human Development Report 2016 by the United Nations Development Programme (UNDP). To answer the questions below, download the data and technical notes from the report:

  1. Refer to the technical notes and the spreadsheet you have downloaded. For each indicator, explain whether you think it is a good measure of the dimension, and suggest alternative indicators, if any. (For example, is GNI per capita a good measure of the dimension ‘a decent standard of living’?)
  1. Figure 4.8 shows the minimum and maximum values for each indicator. Discuss whether you think these are reasonable. (You can read the justification for these values in the technical notes.)
Dimension Indicator Minimum Maximum
Health Life expectancy (years) 20 85
Education Expected years of schooling (years) 0 18
Mean years of schooling (years) 0 15
Standard of living Gross national income per capita (2011 PPP $) 100 75,000

Maximum and minimum values for each indicator in the HDI.

Figure 4.8 Maximum and minimum values for each indicator in the HDI.

United Nations Development Programme. 2016. ‘Technical notes’ in Human Development Report 2016: p. 2.

We are now going to apply the method for constructing the HDI, by recalculating the HDI from its indicators. We will use the formula below, and the minimum and maximum values in the table in Figure 4.8. These are taken from page 2 of the technical notes, which you can refer to for additional details.

The HDI indicators are measured in different units and have different ranges, so in order to put them together into a meaningful index, we need to normalize the indicators using the following formula:

Doing so will give a value in between 0 and 1 (inclusive), which will allow comparison between different indicators.

  1. Refer to Figure 4.8 and calculate the dimension index for each of the dimensions:

Find out more The natural log: What it means, and how to calculate it in R

The natural log turns a linear variable into a concave variable, as shown in Figure 4.9. For any value of income on the horizontal axis, the natural log of that value on the vertical axis is smaller. At first, the difference between income and log income is not that big (for example, an income of 2 corresponds to a natural log of 0.7), but the difference becomes bigger as we move rightwards along the horizontal axis (for example, when income is 100,000, the natural log is only 11.5).

Comparing income with the natural logarithm of income.

Figure 4.9 Comparing income with the natural logarithm of income.

The reason why natural logs are useful in economics is because they can represent variables that have diminishing marginal returns: an additional unit of input results in a smaller increase in the total output than did the previous unit. (If you have studied production functions, then the shape of the natural log function might look familiar.)

When applied to the concept of wellbeing, the ‘input’ is income, and the ‘output’ is material wellbeing. It makes intuitive sense that a $100 increase in per capita income will have a much greater effect on wellbeing for a poor country compared to a rich country. Using the natural log of income incorporates this notion into the index we create. Conversely, the notion of diminishing marginal returns (the larger the value of the input, the smaller the contribution of an additional unit of input) is not captured by GDP per capita, which uses actual income and not its natural log. Doing so makes the assumption that a $100 increase in per capita income has the same effect on wellbeing for rich and poor countries.

The log function in R calculates the natural log of a value for you. To calculate the natural log of a value, x, type log(x). If you have a scientific calculator, you can check that the calculation is correct by using the ln or log key.

Now that you know about the natural log, you might want to go back to Question 3(c) in Part 4.1, and create a new chart using the natural log scale. The natural log is used in economics because it approximates percentage changes i.e. log(x) – log(y) is a close approximation to the percentage change between x and y. So, using the natural log scale, you will be able to ‘read off’ the relative growth rates from the slopes of the different series you have plotted. For example, a 0.01 change in the vertical axis value corresponds to a 1% change in that variable. This will allow you to compare the growth rates of the different components of GDP.

geometric mean
A summary measure calculated by multiplying N numbers together and then taking the Nth root of this product. The geometric mean is useful when the items being averaged have different scoring indices or scales, because it is not sensitive to these differences, unlike the arithmetic mean. For example, if education ranged from 0 to 20 years and life expectancy ranged from 0 to 85 years, life expectancy would have a bigger influence on the HDI than education if we used the arithmetic mean rather than the geometric mean. Conversely, the geometric mean treats each criteria equally. Example: Suppose we use life expectancy and mean years of schooling to construct an index of wellbeing. Country A has life expectancy of 40 years and a mean of 6 years of schooling. If we used the arithmetic mean to make an index, we would get (40 + 6)/2 = 23. If we used the geometric mean, we would get (40 × 6)1/2 = 15.5. Now suppose life expectancy doubled to 80 years. The arithmetic mean would be (80 + 6)/2 = 43, and the geometric mean would be (80 × 6)1/2 = 21.9. If, instead, mean years of schooling doubled to 12 years, the arithmetic mean would be (40 + 12)/2 = 26, and the geometric mean would be (40 × 12)1/2 = 21.9. This example shows that the arithmetic mean can be ‘unfair’ because proportional changes in one variable (life expectancy) have a larger influence over the index than changes in the other variable (years of schooling). The geometric mean gives each variable the same influence over the value of the index, so doubling the value of one variable would have the same effect on the index as doubling the value of another variable.

Now, we can combine these dimensional indices to give the HDI. The HDI is the geometric mean of the three dimension indices (IHealth = Life expectancy index, IEducation = Education index, and IIncome = GNI index):

  1. Use the formula above and the data in the spreadsheet to calculate the HDI for all the countries excluding those in the ‘Other countries or territories’ category. You should get the same values as those in Column C, rounded to three decimal places.

R walk-through 4.7 Calculating the HDI

We will use read_excel to import the data file, which we saved as ‘HDR_data.xlsx’ in our working directory. Before importing, look at the Excel file so that you understand its structure and how it corresponds to the code options used below. We will save the imported data as the dataframe HDR2018.

# File path
HDR2018  read_excel("HDR_data.xlsx",
  # Worksheet to import
  sheet = "Table 1",
  # Number of rows to skip
  skip = 2)

head(HDR2018)
## # A tibble: 6 x 15
##   X__1   X__2         `Human Development ~ X__3  `Life expectancy a~ X__4 
##                                             
## 1 HDI r~ Country      Value                NA    (years)              
## 2             2015                 NA    2015                 
## 3    VERY HIGH H~                  NA                     
## 4 1      Norway       0.94942283449106446  NA    81.710999999999999   
## 5 2      Australia    0.93867953564660933  NA    82.537000000000006   
## 6 2      Switzerland  0.93913086905938037  NA    83.132999999999996   
## # ... with 9 more variables: `Expected years of schooling` ,
## #   X__5 , `Mean years of schooling` , X__6 , `Gross
## #   national income (GNI) per capita` , X__7 , `GNI per capita
## #   rank minus HDI rank` , X__8 , `HDI rank` 
str(HDR2018)
## Classes 'tbl_df', 'tbl' and 'data.frame':  264 obs. of 15 variables:
##  $ X__1                                  : chr  "HDI rank" NA NA "1" ...
##  $ X__2                                  : chr  "Country" NA "VERY HIGH HUMAN DEVELOPMENT" "Norway" ...
##  $ Human Development Index (HDI)         : chr  "Value" "2015" NA "0.94942283449106446" ...
##  $ X__3                                  : logi  NA NA NA NA NA NA ...
##  $ Life expectancy at birth              : chr  "(years)" "2015" NA "81.710999999999999" ...
##  $ X__4                                  : chr  NA NA NA NA ...
##  $ Expected years of schooling           : chr  "(years)" "2015" NA "17.671869999999998" ...
##  $ X__5                                  : chr  NA "a" NA NA ...
##  $ Mean years of schooling               : chr  "(years)" "2015" NA "12.746420000000001" ...
##  $ X__6                                  : chr  NA "a" NA NA ...
##  $ Gross national income (GNI) per capita: chr  "(2011 PPP $)" "2015" NA "67614.353480000005" ...
##  $ X__7                                  : chr  NA NA NA NA ...
##  $ GNI per capita rank minus HDI rank    : chr  NA "2015" NA "5" ...
##  $ X__8                                  : logi  NA NA NA NA NA NA ...
##  $ HDI rank                              : chr  NA "2014" NA "1" ...

Looking at the HDR dataframe, there are rows that have information that isn’t data (for example, all the rows with an ‘NA’ in the first column), as well as variables/columns that do not contain data (for example, most columns beginning with an ‘X_’, though columns labelled X_1 and X_2 contain the HDI rank and the country names respectively).

Cleaning up the dataframe can be easier to do in Excel by deleting irrelevant rows and columns, but one advantage of doing it in R is replicability. Suppose in a year’s time you carried out the analysis again with an updated spreadsheet containing new information. If you had done the cleaning in Excel, you would have to redo it from scratch, but if you had done it in R, you can simply rerun the code below.

Firstly, we eliminate rows that do not have any numbers in the HDI rank column (or X_1 column).

# Rename the first column, currently named X_1
names(HDR2018)[1]  "HDI.rank"

# Rename the second column, currently named X_2 
names(HDR2018)[2]  "Country"

# Rename the last column, which contains the 2014 rank
names(HDR2018)[names(HDR2018) == "HDI rank"]  
  "HDI.rank.2014"

# Eliminate the row that contains the column title
HDR2018  subset(HDR2018,
  !is.na(HDI.rank) & HDI.rank != "HDI rank")

Then we eliminate columns that contain notes in the original spreadsheet (names starting with ‘X_’).

# Check which variables do NOT (!) start with X_
sel_columns  !startsWith(names(HDR2018), "X_")

# Select the columns that do not start with X_
HDR2018  subset(HDR2018, select = sel_columns)

str(HDR2018)
## Classes 'tbl_df', 'tbl' and 'data.frame':  188 obs. of 9 variables:
##  $ HDI.rank                              : chr  "1" "2" "2" "4" ...
##  $ Country                               : chr  "Norway" "Australia" "Switzerland" "Germany" ...
##  $ Human Development Index (HDI)         : chr  "0.94942283449106446" "0.93867953564660933" "0.93913086905938037" "0.9256689410716622" ...
##  $ Life expectancy at birth              : chr  "81.710999999999999" "82.537000000000006" "83.132999999999996" "81.091999999999999" ...
##  $ Expected years of schooling           : chr  "17.671869999999998" "20.43272" "16.040410000000001" "17.095939999999999" ...
##  $ Mean years of schooling               : chr  "12.746420000000001" "13.1751" "13.37" "13.18762553" ...
##  $ Gross national income (GNI) per capita: chr  "67614.353480000005" "42822.19627" "56363.957799999996" "44999.647140000001" ...
##  $ GNI per capita rank minus HDI rank    : chr  "5" "19" "7" "13" ...
##  $ HDI.rank.2014                         : chr  "1" "3" "2" "4" ...

Let’s change some of the long variable names (those in columns 3–8) to shorter ones.

names(HDR2018)[3]  "HDI"
names(HDR2018)[4]  "LifeExp"
names(HDR2018)[5]  "ExpSchool"
names(HDR2018)[6]  "MeanSchool"
names(HDR2018)[7]  "GNI.capita"
names(HDR2018)[8]  "GNI.HDI.rank"

Looking at the structure of the data, we see that R thinks that all the data are chr (character or text variables) because the original datafile contained non-numerical entries (these rows have now been deleted). Apart from the Country variable, which we want to be a factor variable (containing categories), all variables should be numeric.

HDR2018$HDI.rank  as.numeric(HDR2018$HDI.rank)
HDR2018$Country  as.factor(HDR2018$Country)
HDR2018$HDI  as.numeric(HDR2018$HDI)
HDR2018$LifeExp  as.numeric(HDR2018$LifeExp)
HDR2018$ExpSchool  as.numeric(HDR2018$ExpSchool)
HDR2018$MeanSchool  as.numeric(HDR2018$MeanSchool)
HDR2018$GNI.capita  as.numeric(HDR2018$GNI.capita)
HDR2018$GNI.HDI.rank  as.numeric(HDR2018$GNI.HDI.rank)
HDR2018$HDI.rank.2014  as.numeric(HDR2018$HDI.rank.2014)
str(HDR2018)
## Classes 'tbl_df', 'tbl' and 'data.frame':  188 obs. of 9 variables:
##  $ HDI.rank     : num  1 2 2 4 5 5 7 8 9 10 ...
##  $ Country      : Factor w/ 188 levels "Afghanistan",..: 125 9 163 64 47 151 120 81 76 32 ...
##  $ HDI          : num  0.949 0.939 0.939 0.926 0.925 ...
##  $ LifeExp      : num  81.7 82.5 83.1 81.1 80.4 ...
##  $ ExpSchool    : num  17.7 20.4 16 17.1 19.2 ...
##  $ MeanSchool   : num  12.7 13.2 13.4 13.2 12.7 ...
##  $ GNI.capita   : num  67614 42822 56364 45000 44519 ...
##  $ GNI.HDI.rank : num  5 19 7 13 13 -3 8 11 20 12 ...
##  $ HDI.rank.2014: num  1 3 2 4 6 4 6 8 9 9 ...

Now we have a nice clean dataset that we can work with.

We start by calculating the three indices, using the information given. For the education index we calculate the index for expected and mean schooling separately, then take the arithmetic mean to get I.Education. As some mean schooling observations exceed the specified ‘maximum’ value of 18, the calculated index values would be larger than 1. To avoid this, we use pmin to replace these observations with 18 to obtain an index value of 1.


HDR2018$I.Health  
  (HDR2018$LifeExp - 20) / (85 - 20)

HDR2018$I.Education  
  ((pmin(HDR2018$ExpSchool, 18) - 0) / 
  (18 - 0) + (HDR2018$MeanSchool - 0) / 
  (15 - 0)) / 2

HDR2018$I.Income 
  (log(HDR2018$GNI.capita) - log(100)) /
  (log(75000) - log(100))

HDR2018$HDI.calc  
  (HDR2018$I.Health * HDR2018$I.Education * 
    HDR2018$I.Income)^(1/3)

Now we can compare the HDI given in the table and our calculated HDI.

HDR2018[, c("HDI", "HDI.calc")]
## # A tibble: 188 x 2
##   HDI   HDI.calc
##    
## 1 0.949 0.949
## 2 0.939 0.939
## 3 0.939 0.939
## 4 0.926 0.926
## 5 0.925 0.925
## 6 0.925 0.927
## 7 0.924 0.924
## 8 0.923 0.923
## 9 0.921 0.921
## 10 0.920 0.920
## # ... with 178 more rows

The HDI is one way to measure wellbeing, but you may think that it does not use the most appropriate measures for the non-material aspects of wellbeing (health and education).

Now we will use the same method to create our own index of non-material wellbeing (an ‘alternative HDI’), using different indicators. You can find alternative indicators to measure health and education on the UNDP website, in the file ‘Download 2018 Human Development Data Bank’ (left-hand side of the page). (The first column of the spreadsheet, ‘dimension’, tells you whether the indicators refer to ‘Health’, ‘Education’, or other categories).

  1. Create an alternative index of wellbeing. In particular, propose alternative Education and Health indices in (a) and (b), then combine these with the existing Income index in (c) to calculate an alternative HDI. Examine whether the changes caused substantial changes in country rankings in (d).

R walk-through 4.8 Creating your own HDI

Merge data and calculate alternative indices

This example uses the following indicators:

  • Education: Literacy rate, adult (% ages 15 and older); Gross enrolment ratio, tertiary (% of tertiary school-age population); Primary school teachers trained to teach (%)
  • Health: Child malnutrition, stunting (moderate or severe) (% under age 5); Mortality rate, female adult (per 1,000 people); Mortality rate, male adult (per 1,000 people).

First, we import the data and check that it has been imported correctly. You can see that each row represents a different country and indicator (indicator_name), and each column represents a different year.

# Filename
allHDR2018  read_excel("2018_all_indicators.xlsx",
  # Sheet to import
  sheet = "Data")

head(allHDR2018)
## # A tibble: 6 x 34
##    dimension indicator_id indicator_name iso3 country_name `1990` `1991`
##                                 
## 1  Composit-         14206 HDI rank      AFG   Afghanistan    NA   NA
## 2  Composit-         14206 HDI rank      ALB   Albania    NA   NA
## 3  Composit-         14206 HDI rank      DZA   Algeria    NA   NA
## 4  Composit-         14206 HDI rank      AND   Andorra    NA   NA
## 5  Composit-         14206 HDI rank    AGO   Angola     NA   NA
## 6  Composit-         14206 HDI rank    ATG   Antigua and~   NA   NA
## # ... with 27 more variables: `1992` , `1993` , `1993` ,
## #   `1994` , `1995` , `1996` , `1997` ,`1998` ,
## #   `1999` , `2000` , `2001` , `2002` , `2003` ,
## #   `2004` , `2005` , `2006` , `2007` , `2008` ,
## #   `2009` , `2010` , `2011` , `2012` , `2013` ,
## #   `2014` , `2015` , `2016` , `2017` , `9999` 
str(HDR2018)
## Classes 'tbl_df', 'tbl' and 'data.frame':  25636 obs. of 34 variables:
##  $ dimension      : chr  "Composite indices" "Composite indices" "Composite indices" "Composite indices" ...
##  $ indicator_id   : num  146206 146206 146206 146206 ...
##  $ indicator_name : chr  "HDI rank" "HDI rank" "HDI rank" "HDI rank" ...
##  $ iso3           : chr  "AFG" "ALB" "DZA" "AND" ...
##  $ country_name   : chr  "Afghanistan" "Albania" "Algeria" "Andorra" ...
##  $ 1990           : num  NA NA NA NA ...
##  $ 1991           : num  NA NA NA NA ...
##  $ 1992           : num  NA NA NA NA ...
##  $ 1993           : num  NA NA NA NA ...
##  $ 1994           : num  NA NA NA NA ...
##  $ 1995           : num  NA NA NA NA ...
##  $ 1996           : num  NA NA NA NA ...
##  $ 1997           : num  NA NA NA NA ...
##  $ 1998           : num  NA NA NA NA ...
##  $ 1999           : num  NA NA NA NA ...
##  $ 2000           : num  NA NA NA NA ...
##  $ 2001           : num  NA NA NA NA ...
##  $ 2002           : num  NA NA NA NA ...
##  $ 2003           : num  NA NA NA NA ...
##  $ 2004           : num  NA NA NA NA ...
##  $ 2005           : num  NA NA NA NA ...
##  $ 2006           : num  NA NA NA NA ...
##  $ 2007           : num  NA NA NA NA ...
##  $ 2008           : num  NA NA NA NA ...
##  $ 2009           : num  NA NA NA NA ...
##  $ 2010           : num  NA NA NA NA ...
##  $ 2011           : num  NA NA NA NA ...
##  $ 2012           : num  NA NA NA NA ...
##  $ 2013           : num  NA NA NA NA ...
##  $ 2014           : num  NA NA NA NA ...
##  $ 2015           : num  NA NA NA NA ...
##  $ 2016           : num  NA NA NA NA ...
##  $ 2017           : num  NA NA NA NA ...
##  $ 9999           : num  NA NA NA NA ...

Then we follow the same process as in R walk-through 4.7—getting the data for the indicators we want, reshaping it so that each indicator is in a different column (instead of a different row), and giving each indicator a shorter name. We will save this data as HDR2018w. Note that the variable 9999 refers to the latest year available, or the average taken over a range of years (the Excel file contains information on which year(s) were used).

indicators  c(
  "Literacy rate, adult (% ages 15 and older)", 
  "Gross enrolment ratio, tertiary (% of tertiary school-age population)", 
  "Primary school teachers trained to teach (%)", 
  "Child malnutrition, stunting (moderate or severe) (% under age 5)", 
  "Mortality rate, female adult (per 1,000 people)", 
  "Mortality rate, male adult (per 1,000 people)"

HDR2018l  allHDR2018[
  allHDR2018$indicator_name %in% indicators, ]

HDR2018l  subset(HDR2018l, 
  # Indicate which variables to keep.
  select = c("indicator_name", "country_name", "9999"))

HDR2018w  dcast(HDR2018l, country_name ~ indicator_name, 
  value.var = "9999")
names(HDR2018.Edu)[1]  "HDI.rank"

# Rename the second column, currently named 'Very high …'

names(HDR2018w)[1]  "Country"
names(HDR2018w)[2]  "Child.Malnu"
names(HDR2018w)[3]  "Tert.Enrol"
names(HDR2018w)[4]  "Adult.Lit"
names(HDR2018w)[5]  "Mortality.Female"
names(HDR2018w)[6]  "Mortality.Male"
names(HDR2018w)[7]  "Prim.Teacher"

str(HDR2018w)
## 'data.frame': 194 obs. of 7 variables:
##  $ Country         : chr  "Afghanistan" "Albania" "Algeria" "Andorra" ...
##  $ Child.Malnu     : num 40.9 23.2 11.7 NA 37.6 NA NA 9.4 2 NA ...
##  $ Tert.Enrol      : num 8 61 43 NA 9 22 86 51 122 83 ...
##  $ Adult.Lit       : num 31.7 97.2 75.1 100 66 NA 98.1 99.7 NA NA ...
##  $ Mortality.Female: num 202 50 83 NA 203 106 74 74 45 46 ...
##  $ Mortality.Male  : num 245 79 106 NA 277 151 151 174 76 86 ...
##  $ Prim.Teacher    : num NA NA 100 100 47 65 NA NA NA NA ... 

Looking at the structure (str( )), we can see that all indicators are correctly in numerical (num) format.

Before we can calculate indices, we need to set minimum and maximum values, which we base on the minimum and maximum values in the sample.

summary(HDR2018w)
##    Country            Child.Malnu       Tert.Enrol        Adult.Lit   
##  Length:194         Min.   : 1.80   Min.   :  2.00   Min.   : 15.50  
##  Class :character   1st Qu.:10.78   1st Qu.: 13.50   1st Qu.: 68.55  
##  Mode  :character   Median :22.20   Median : 37.00   Median : 92.25  
##                     Mean   :22.84   Mean   : 40.36   Mean   : 80.11  
##                     3rd Qu.:32.65   3rd Qu.: 63.00   3rd Qu.: 97.35  
##                     Max.   :55.90   Max.   :122.00   Max.   :100.00  
##                     NA's   :62      NA's   :43         NA's   :60
##  Mortality.Female Mortality.Male   Prim.Teacher 
##  Min.   : 34.00    Min.   : 63.0   Min.   : 15.00  
##  1st Qu.: 66.75    1st Qu.:121.0   1st Qu.: 70.00  
##  Median :100.50    Median :184.0   Median : 86.00 
##  Mean   :130.86    Mean   :195.1   Mean   : 81.23  
##  3rd Qu.:181.50    3rd Qu.:253.5   3rd Qu.: 99.00  
##  Max.   :463.00    Max.   :555.0   Max.   :100.00  
##  NA's   :14       NA's    :14      NA's   :77

As we want the observations to be inside the [min, max] interval, we choose the following [min, max] pairs for the education indicators: Adult.Lit: [15, 100], Tert.Enrol: [2, 122], and Prim.Teacher: [15, 100]. You may want to research why there can be countries with a tertiary enrolment ratio larger than 100%.

Let’s calculate the alternative education index (I.Education.alt), taking an arithmetic average just as we did for I.Education in R walk-through 4.7.

HDR2018w$I.Adult.Lit 
  (HDR2018w$Adult.Lit-15) / (100-15)

HDR2018w$I.Tert.Enrol
  (HDR2018w$Tert.Enrol-2) / (122-2) 

HDR2018w$I.Prim.Teacher 
  (HDR2018w$Prim.Teacher-15) / (100-15)
  
HDR2018w$I.Education.alt  
  (HDR2018w$I.Adult.Lit +
    HDR2018w$I.Tert.Enrol + 
    HDR2018w$I.Prim.Teacher) / 3

summary(HDR2018w$I.Education.alt)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.1628  0.4222  0.5781  0.5862  0.7181  0.8973     113

You can see that we could not calculate this index for 113 countries, as at least one of the three values was missing.

We repeat this procedure to calculate an alternative health index (I.Health.alt). The [min, max] pairs we use are: Child.Malnu: [1, 56], Mortality.Female: [34, 463], Mortality.Male: [63, 555].

HDR2018w$I.Child.MalNu 
  (HDR2018w$Child.MalNu - 1) /
  (56 - 1)

HDR2018w$I.Mortality.Female 
  (HDR2018w$Mortality.Female - 34) /
  (463 - 34) 

HDR2018w$I.Mortality.Male 
  (HDR2018w$Mortality.Male - 63) /
  (555 - 63)

HDR2018w$I.Health.alt 
  (HDR2018w$I.Child.MalNu + 
  HDR2018w$I.Mortality.Female + 
  HDR2018w$I.Mortality.Male) / 3

# Note that these are all 'bad' indicators in the sense that higher numbers indicate worse outcomes.

# For all other indicators, larger numbers mean better outcomes. So we need to 'flip' the values of this indicator.

HDR2018w$I.Health.alt  (1 - HDR2018w$I.Health.alt)

summary(HDR2018w$I.Health.alt)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.1370  0.5235  0.7088  0.6655  0.8274  0.9766      64

Now we use the merge function to merge this variable into our existing HDR2018 dataframe.

HDR2018  merge(HDR2018, HDR2018w)

Calculate an alternative HDI

Looking at HDR2018, you will see that alternative health and education indices have been added. Now we are in a position to calculate our own HDI (HDI.own).

HDR2018$HDI.own  
  (HDR2018$I.Health.alt *
    HDR2018$I.Education.alt *
    HDR2018$I.Income)^(1/3) 

summary(HDR2018$HDI.own)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##  0.2647  0.4541  0.5705  0.5870  0.7460  0.8469     121

We have a substantial number of missing observations, leaving us with around 70 countries for which we could calculate the alternative HDI.

Calculate ranks

To compare the ranks of the two indices (the original HDI and our alternative HDI), we should only rank the countries that have observations for both indices. We will create a dataframe called HDR2018_sub that contains this subset of countries.

HDR2018_sub  
  subset(HDR2018, !is.na(HDI) & !is.na(HDI.own)) 

Let’s calculate the rank for our index. The rank function will assign rank 1 to the smallest index value, but we want the largest (best) index value to have the rank 1. We add - in front of the variable name to obtain the desired effect.

HDR2018_sub$HDI.own.rank 
  rank(-HDR2018_sub$HDI.own, na.last = "keep") 
HDR2018_sub$HDI.rank 
  rank(-HDR2018_sub$HDI, na.last = "keep") 

Now we will use the ggplot function to make a scatterplot comparing the rank of the HDI with that of our own index.

ggplot(HDR2018_sub, aes(x = HDI.rank, y = HDI.own.rank)) +
  # Use solid circles
  geom_point(shape = 16) +
  labs(y = "Alternative HDI rank", x = "HDI rank") +
  ggtitle("Comparing ranks between HDI and HDI.own") +
  theme_bw()

Scatterplot of ranks for HDI and alternative HDI index.

Figure 4.10 Scatterplot of ranks for HDI and alternative HDI index.

You can see that in general the rankings are similar. If they were identical, the points in the scatterplot would form a straight upward-sloping line. They do not form a straight line, but there is a very strong positive correlation. There are, however, a few countries where the alternative definitions have caused a change in ranking, so let’s use the head and tail functions to find out which countries these are.

temp  HDR2018_sub[
  order(HDR2018_sub$HDI.rank -
  HDR2018_sub$HDI.own.rank),
  # Show selected variables 
  c("Country", "HDI.rank", "HDI.own.rank")]

# Show the countries with the largest fall in rank
head(temp, 5)
##        Country HDI.rank HDI.own.rank
## 98     Lesotho       52           70
## 164  Sri Lanka       13           29
## 104  Madagascar      54           68
## 57     Eswatini      41           51
## 154  Seychelles       7           17
# Show the countries with the largest increase in rank
tail(temp, 5)
##           Country HDI.rank HDI.own.rank
## 111    Mauritania       53           45
## 171      Tanzania       49           41
## 31       Cambodia       43           34
## 165         Sudan       59           50
## 120       Myanmar       45           33
  1. Compare your alternative index to the HDI:
Classification HDI
Very high human development 0.800 and above
High human development 0.700–0.799
Medium human development 0.550–0.699
Low human development Below 0.550

Classification of countries according to their HDI value.

Figure 4.11 Classification of countries according to their HDI value.

United Nations Development Programme. 2016. ‘Technical notes’ in Human Development Report 2016: p.3.

We will now investigate whether HDI and GDP per capita give similar information about overall wellbeing, by comparing a country’s rank in both measures. To answer Question 7, first add (merge) this data to the dataframe that contains your HDI calculations, making sure to match the data to the correct country. (R walk-through 4.8 shows you how to extract the indicator(s) you need and merge them with another dataset.)

  1. Evaluate GDP per capita and the HDI as measures of overall wellbeing:
HDI
Low High
GDP Low
High

Classification of countries according to their HDI and GDP values.

Figure 4.12 Classification of countries according to their HDI and GDP values.

  1. The HDI is one way to measure wellbeing, but there are many other ways to measure wellbeing.