Empirical Project 6 Solutions
These are not model answers. They are provided to help students, including those doing the project outside a formal class, to check their progress while working through the questions using the Excel, R, or Google Sheets walk-throughs. There are also brief notes for the more interpretive questions. Students taking courses using Doing Economics should follow the guidance of their instructors.
Part 6.1 Looking for patterns in the survey data
-
Interviews were conducted by teams of students from the countries surveyed who had business experience and training. The managers interviewed were middle managers, since they were familiar with day-to-day operations as well as the management practices of the firm (for example hiring and firing decisions, performance reviews).
Some measures taken to improve reliability and validity of the data:
- Interviews were conducted in the manager’s native language, which would minimize misunderstandings.
- Managers were not told they were being assessed on management practices, only that they were being interviewed on management, so they would be more likely to give honest responses.
- Managers were also not shown the assessment criteria, so they could not have altered their responses in the hopes of getting a higher score.
- The firms selected were medium-sized firms that were rarely given press coverage, so interviewers were unlikely to have prior knowledge about firms that could bias their scoring.
- To check for consistency between interviewers, each firm was scored independently by two people.
-
While these three aspects of management practices are not comprehensive, they are relatively easy to measure, and the resulting definition of ‘good’ and ‘bad’ practice applies to firms across industries and countries. It is therefore possible to make cross-country and/or cross-industry comparisons.
There are many possible aspects of management that are not included, such as:
- leadership qualities that managers possess (although Bloom et al. (2012) admit that this is difficult to quantify)
- management practices that support innovation
- strategic decisions such as pricing or takeover decisions (these affect firm performance and survival, but whether a decision was ‘good’ or ‘bad’ depends on the context).
-
Solution figures 6.1 and 6.2 provide the tables with countries listed in alphabetical order.
Countries tend to have similar ranks across individual criteria, and ranks for individual criteria are generally similar to overall management rank. We can therefore say that countries with a higher overall rank are better managed across all aspects.
Country | Overall management (mean) | Monitoring management (mean) | Targets management (mean) | Incentives management (mean) |
---|---|---|---|---|
Argentina | 2.76 | 3.08 | 2.68 | 2.56 |
Australia | 3.02 | 3.29 | 3.02 | 2.74 |
Brazil | 2.71 | 3.06 | 2.69 | 2.55 |
Canada | 3.17 | 3.55 | 3.07 | 2.94 |
Chile | 2.83 | 3.14 | 2.72 | 2.67 |
China | 2.71 | 2.90 | 2.63 | 2.69 |
France | 3.03 | 3.43 | 2.97 | 2.74 |
Germany | 3.23 | 3.57 | 3.22 | 2.98 |
Greece | 2.73 | 2.97 | 2.66 | 2.58 |
India | 2.67 | 2.91 | 2.66 | 2.63 |
Italy | 3.03 | 3.26 | 3.10 | 2.76 |
Japan | 3.23 | 3.50 | 3.34 | 2.92 |
Mexico | 2.92 | 3.29 | 2.88 | 2.71 |
New Zealand | 2.93 | 3.18 | 2.96 | 2.63 |
Poland | 2.90 | 3.12 | 2.94 | 2.83 |
Portugal | 2.87 | 3.27 | 2.83 | 2.59 |
Republic of Ireland | 2.89 | 3.14 | 2.81 | 2.79 |
Sweden | 3.21 | 3.64 | 3.19 | 2.83 |
UK | 3.03 | 3.34 | 2.98 | 2.86 |
United States | 3.35 | 3.58 | 3.26 | 3.25 |
Mean of management scores.
Country | Overall management (rank) | Monitoring management (rank) | Targets management (rank) | Incentives management (rank) |
---|---|---|---|---|
Argentina | 16 | 16 | 17 | 19 |
Australia | 9 | 8 | 7 | 10 |
Brazil | 19 | 17 | 16 | 20 |
Canada | 5 | 4 | 6 | 3 |
Chile | 15 | 13 | 15 | 14 |
China | 18 | 20 | 20 | 13 |
France | 7 | 6 | 9 | 11 |
Germany | 2 | 3 | 3 | 2 |
Greece | 17 | 18 | 19 | 18 |
India | 20 | 19 | 18 | 15 |
Italy | 8 | 11 | 5 | 9 |
Japan | 3 | 5 | 1 | 4 |
Mexico | 11 | 9 | 12 | 12 |
New Zealand | 10 | 12 | 10 | 16 |
Poland | 12 | 15 | 11 | 7 |
Portugal | 14 | 10 | 13 | 17 |
Republic of Ireland | 13 | 14 | 14 | 8 |
Sweden | 4 | 1 | 4 | 6 |
UK | 6 | 7 | 8 | 5 |
United States | 1 | 2 | 2 | 1 |
Rank according to management scores.
- Solution figure 6.3 shows the average overall management score for each country.
- The ranking of countries in Figure 1 of Bloom et al. (2012) is slightly different, indicating that the mean values are not the same as in Solution figure 6.1. This difference is due to the number of observations used: Figure 1 uses 9,079 observations, whereas the data we have has 9,207 observations.
(Note: It is likely that Bloom et al. (2012) were using an earlier version of this survey data to create their Figure 1. The earlier version had not been updated to include data from the latest round of interviews.)
- Chile is used as an example in Solution figure 6.4.
US | Frequency | Proportion of firms (%) | Chile | Frequency | Proportion of firms (%) |
---|---|---|---|---|---|
1.00 | 0 | 0.00 | 1.00 | 0 | 0.00 |
1.20 | 0 | 0.00 | 1.20 | 0 | 0.00 |
1.40 | 1 | 0.08 | 1.40 | 1 | 0.32 |
1.60 | 0 | 0.00 | 1.60 | 3 | 0.95 |
1.80 | 10 | 0.82 | 1.80 | 6 | 1.89 |
2.00 | 9 | 0.73 | 2.00 | 24 | 7.57 |
2.20 | 28 | 2.29 | 2.20 | 15 | 4.73 |
2.40 | 46 | 3.76 | 2.40 | 30 | 9.46 |
2.60 | 58 | 4.73 | 2.60 | 25 | 7.89 |
2.80 | 96 | 7.84 | 2.80 | 49 | 15.46 |
3.00 | 139 | 11.35 | 3.00 | 52 | 16.40 |
3.20 | 118 | 9.63 | 3.20 | 28 | 8.83 |
3.40 | 164 | 13.39 | 3.40 | 27 | 8.52 |
3.60 | 111 | 9.06 | 3.60 | 21 | 6.62 |
3.80 | 145 | 11.84 | 3.80 | 20 | 6.31 |
4.00 | 114 | 9.31 | 4.00 | 9 | 2.84 |
4.20 | 60 | 4.90 | 4.20 | 6 | 1.89 |
4.40 | 60 | 4.90 | 4.40 | 1 | 0.32 |
4.60 | 31 | 2.53 | 4.60 | 0 | 0.00 |
4.80 | 31 | 2.53 | 4.80 | 0 | 0.00 |
5.00 | 4 | 0.33 | 5.00 | 0 | 0.00 |
Frequency tables for the US and Chile.
- Solution figure 6.5 provides the column chart for the US and Chile.
-
Similarities: The data is not evenly distributed across the scale: the proportion of observations tends to decrease for scores further away from the mean. The spread of the data and the range look similar.
Differences: The distribution of the US is more right-centred than that of Chile, with a larger proportion of observations at higher values.
- Solution figure 6.6 shows box and whisker plots for the US and Chile.
- From the box and whisker plots, we can now see that the mean and median for the US is higher than that of Chile. In fact, all quartiles of the distribution of the US are higher than those for Chile. The width of the boxes and length of the whiskers indicates that the shapes of the two distributions are similar. There is also one outlier for the US, which was not clearly visible from the column chart.
- Solution figures 6.7 and 6.8 show the tables for hospitals and schools.
Country | Average of management | Average of monitoring | Average of targets | Average of people |
---|---|---|---|---|
Canada | 2.52 | 2.82 | 2.44 | 2.17 |
France | 2.40 | 2.59 | 2.29 | 2.03 |
Germany | 2.64 | 2.85 | 2.55 | 2.45 |
Italy | 2.48 | 2.67 | 2.33 | 2.20 |
Sweden | 2.57 | 2.90 | 2.68 | 2.36 |
UK | 2.82 | 3.07 | 2.71 | 2.62 |
US | 3.00 | 3.21 | 2.87 | 2.92 |
Mean scores for hospitals.
Country | Average of management | Average of monitoring | Average of targets | Average of people |
---|---|---|---|---|
Canada | 2.78 | 2.92 | 2.86 | 2.33 |
Germany | 2.54 | 2.70 | 2.49 | 2.26 |
Sweden | 2.80 | 3.09 | 2.72 | 2.51 |
UK | 2.96 | 3.07 | 2.97 | 2.75 |
US | 2.72 | 2.88 | 2.63 | 2.47 |
Mean scores for schools.
-
Solution figures 6.9 and 6.10 provide separate bar charts for hospitals and schools.
The country rankings for both hospitals and schools are different from that of manufacturing. For example, while the UK ranks below the US, Sweden, and Canada in manufacturing, it ranks above these countries in schools. Similarly, Germany has a high ranking for both manufacturing and hospitals, but a low ranking for schools.
-
There are many possible explanations for the observed patterns, for example:
UK schools have recently undergone some reforms to improve management, such as decentralization (allowing schools autonomy over their management policies) and sharing of better practices across different schools. These policies could explain why the average management score for schools in the UK is relatively high.
Part 6.2 Do management practices differ between countries?
- Chile is used as an example in Solution figure 6.11.
- Chile is used as an example. The width of the 95% confidence interval is 0.066.
Country | Mean | Standard deviation | Number of firms | Width of CI |
---|---|---|---|---|
Chile | 2.83 | 0.60 | 317 | 0.07 |
United States | 3.35 | 0.64 | 1,224 | 0.04 |
Mean management score in manufacturing firms for the US and Chile.
- Solution figure 6.12 shows the column chart for the US and Chile with 95% confidence intervals.
- The confidence interval for Chile is wider than that of the US, indicating the mean for Chile is less precisely estimated. Looking at Solution figure 6.11, the low precision is likely due to the smaller number of observations for Chile.
- The mean score for the US is likely to be different from that of Chile, since both means lie outside each other’s 95% confidence interval. Also, since the confidence intervals are quite far apart, we are quite sure that this result is not just because of our chosen probability of 95%. For example, even if we used 99% confidence intervals, which would be wider, the means would still not be inside each other’s confidence interval. In this case we say our result is robust to the choice of significance level. (In other cases, the results may change depending on the significance level (or probability) chosen.)
- Solution figure 6.13 provides summary tables for both hospitals and schools.
Hospitals | Schools | |||||||
---|---|---|---|---|---|---|---|---|
Country | Average | SD | Number | Average | SD | Number | Width (hospitals) | Width (schools) |
Canada | 2.52 | 0.45 | 175 | 2.78 | 0.39 | 151 | 0.07 | 0.06 |
France | 2.40 | 0.43 | 158 | 0.07 | ||||
Germany | 2.64 | 0.39 | 130 | 2.54 | 0.43 | 143 | 0.07 | 0.07 |
Italy | 2.48 | 0.52 | 166 | 0.08 | ||||
Sweden | 2.57 | 0.44 | 43 | 2.80 | 0.44 | 89 | 0.13 | 0.09 |
UK | 2.82 | 0.43 | 184 | 2.96 | 0.40 | 110 | 0.06 | 0.07 |
US | 3.00 | 0.54 | 327 | 2.72 | 0.45 | 285 | 0.06 | 0.05 |
Mean management score and 95% confidence interval width for hospitals and schools.
- Solution figures 6.14 and 6.15 provide the bar charts for hospitals and schools, and the discussions follow.
Solution figure 6.14 shows sample means for the management scores in different countries along with their 95% confidence intervals.
For hospitals, the mean for the US is likely to be higher than all other countries’ means as its confidence interval clearly does not overlap with any of the other countries’ confidence intervals. We would reach the same conclusion even if wider (99%) confidence intervals were used.
For schools (Solution figure 6.15), the UK is likely to have a higher mean than the US as their respective confidence intervals do not overlap, by a significant margin. For the same reason it is apparent that the average German management score is lower than that in the other countries shown in the figure. Sweden, Canada and the US have overlapping confidence intervals and hence there is no clear evidence that would point at substantial differences in the respective population means.
-
Holding all other things fixed, we would expect the confidence intervals to be wider if the standard deviation was larger (the sample mean is estimated less precisely).
Holding all other things fixed, we would expect the confidence intervals to be narrower if the number of observations was larger (we sampled more of the whole population). The more observations we have, the closer we can approximate the population mean.
Looking at the confidence intervals, standard deviation, and number of observations in our data, we can confirm this relationship.
Part 6.3 What factors affect the quality of management?
- As shown in Solution figures 6.16 and 6.17.
- As shown in Solution figures 6.16 and 6.17.
Private | Public | |||||||
---|---|---|---|---|---|---|---|---|
Country | Mean | SD | Number | Mean | SD | Number | Width (private) | Width (public) |
Canada | 2.78 | 0.79 | 4 | 2.52 | 0.45 | 171 | 1.25 | 0.07 |
France | 2.65 | 0.51 | 20 | 2.37 | 0.41 | 138 | 0.24 | 0.07 |
Germany | 2.61 | 0.39 | 68 | 2.68 | 0.38 | 62 | 0.09 | 0.10 |
Italy | 2.71 | 0.50 | 33 | 2.42 | 0.51 | 133 | 0.17 | 0.09 |
Sweden | 3.10 | 0.07 | 2 | 2.54 | 0.43 | 41 | 0.64 | 0.13 |
UK | 3.00 | 0.39 | 64 | 2.73 | 0.42 | 120 | 0.10 | 0.07 |
US | 3.14 | 0.53 | 164 | 2.87 | 0.52 | 163 | 0.08 | 0.08 |
Mean management score and 95% confidence interval width for private and public hospitals.
Private | Public | |||||||
---|---|---|---|---|---|---|---|---|
Country | Mean | SD | Number | Mean | SD | Number | Width (private) | Width (public) |
Canada | 2.76 | 0.45 | 21 | 2.78 | 0.38 | 129 | 0.21 | 0.07 |
Germany | 2.73 | 0.49 | 16 | 2.51 | 0.41 | 127 | 0.26 | 0.07 |
Sweden | 3.07 | 0.63 | 23 | 2.71 | 0.31 | 66 | 0.27 | 0.08 |
UK | 2.89 | 0.41 | 11 | 2.97 | 0.40 | 99 | 0.28 | 0.08 |
US | 2.66 | 0.48 | 74 | 2.75 | 0.44 | 211 | 0.11 | 0.06 |
Mean management score and 95% confidence interval width for private and public schools.
- Solution figures 6.18 and 6.19 provide the bar charts for hospitals and schools, and the discussions follow.
Hospitals: After we condition on ownership type, the US still has the highest average score for public and private hospitals while France has the lowest average score for public and Germany for private hospitals. Compared to Question 2b in Part 6.2, the rankings in the middle have changed. For example, among private hospitals, Sweden ranks higher than in the overall rankings (though we should interpret this result with caution as there are only two observations for private hospitals).
In most countries, private hospitals are, on average, better managed than public hospitals (except for Germany). This is most obvious in the US and the UK, where the respective confidence intervals are non-overlapping. But in most countries reported the private and public confidence intervals are overlapping. In these countries, the data is compatible with a hypothesis that private and public hospitals have equally effective management.
Note the very wide confidence intervals for Canada and Sweden, which have very few observations.
Schools: Looking across countries, there is no clear difference in the patterns of management practices in public vs private schools: in some countries public schools have a higher mean, in others, private schools have a higher mean. In fact, in the US, UK, Germany and Canada the confidence intervals for private and public schools are well overlapping, indicating that the data is compatible with a hypothesis of equal average management scores in private and public schools. The exception is Sweden, where the data indicates the possibility that the average management quality in private schools is higher than that in public schools. The confidence interval for the management score in private schools does not overlap with the confidence interval for public schools.
- The solution depends on your software and is not shown.
- Brazil and Canada are used as examples in Solution figure 6.20. Note that when there is only one observation in a group, there is no standard deviation.
Larger | Smaller | |||||||
---|---|---|---|---|---|---|---|---|
Country; ownership type | Mean | SD | Number | Mean | SD | Number | Width (Larger) | Width (Smaller) |
Brazil | ||||||||
Dispersed shareholders | 3.48 | 0.73 | 45 | 3.06 | 0.67 | 28 | 0.22 | 0.26 |
Family owned, external CEO | 2.99 | 0.69 | 10 | 2.82 | 0.73 | 8 | 0.49 | 0.61 |
Family owned, family CEO | 2.70 | 0.64 | 41 | 2.50 | 0.67 | 80 | 0.20 | 0.15 |
Founder | 2.66 | 0.59 | 72 | 2.35 | 0.52 | 124 | 0.14 | 0.09 |
Government | 2.44 | 1.18 | 2 | 4.00 | 1 | 10.59 | ||
Managers | 2.51 | 0.63 | 7 | 2.64 | 0.57 | 23 | 0.58 | 0.25 |
Other | 3.01 | 0.54 | 29 | 2.57 | 0.40 | 13 | 0.21 | 0.24 |
Private equity | 3.23 | 0.59 | 5 | 0.73 | ||||
Private individuals | 2.94 | 0.52 | 42 | 2.69 | 0.71 | 39 | 0.16 | 0.23 |
Canada | ||||||||
Dispersed shareholders | 3.52 | 0.58 | 53 | 3.43 | 0.60 | 53 | 0.16 | 0.16 |
Family owned, external CEO | 3.31 | 0.49 | 9 | 2.90 | 0.47 | 6 | 0.37 | 0.49 |
Family owned, family CEO | 3.02 | 0.61 | 14 | 2.75 | 0.55 | 25 | 0.36 | 0.23 |
Founder | 3.01 | 0.69 | 14 | 2.86 | 0.56 | 37 | 0.40 | 0.19 |
Government | 3.00 | 1 | ||||||
Managers | 3.01 | 0.57 | 5 | 3.17 | 0.49 | 5 | 0.70 | 0.61 |
Other | 3.33 | 0.40 | 12 | 3.15 | 0.44 | 16 | 0.26 | 0.24 |
Private equity | 3.12 | 0.58 | 21 | 3.34 | 0.67 | 11 | 0.26 | 0.45 |
Private individuals | 3.46 | 0.45 | 37 | 2.90 | 0.60 | 66 | 0.15 | 0.15 |
United States | ||||||||
Dispersed Shareholders | 3.50 | 0.56 | 295 | 3.45 | 0.56 | 158 | 0.06 | 0.09 |
Family owned, external CEO | 3.45 | 0.54 | 22 | 2.86 | 0.63 | 6 | 0.24 | 0.66 |
Family owned, family CEO | 3.44 | 0.58 | 42 | 2.96 | 0.68 | 73 | 0.18 | 0.16 |
Founder | 3.14 | 0.51 | 28 | 3.14 | 0.61 | 60 | 0.20 | 0.16 |
Government | 4.06 | 1 | ||||||
Managers | 3.80 | 0.73 | 6 | 3.57 | 0.66 | 6 | 0.77 | 0.69 |
Other | 3.48 | 0.48 | 31 | 3.06 | 0.74 | 21 | 0.17 | 0.33 |
Private equity | 3.50 | 0.43 | 27 | 3.34 | 0.48 | 27 | 0.17 | 0.19 |
Private individuals | 3.40 | 0.68 | 68 | 3.07 | 0.61 | 93 | 0.16 | 0.13 |
Table of mean management score and 95% confidence interval width, according to ownership type.
- As shown in Solution figure 6.20.
- Solution figures 6.21, 6.22 and 6.23 provide the bar charts for Brazil, Canada and the US, and the discussions follow.
For Brazil: On average, larger firms tend to be managed better, regardless of ownership type. However, confidence intervals are largely overlapping and hence the data is compatible with the hypothesis that management quality does not correlate with organization size. (We cannot say anything meaningful about government-run firms because there are only a few observations.) Firms owned by shareholders (dispersed or private equity) have a higher mean management score than family-owned firms.
For Canada: No clear pattern between firm size and management practice is apparent: larger firms are managed better (on average) for some ownership types, while smaller firms are managed better for other types. For most ownership types, it is quite likely that we would observe the differences shown, under a hypothesis that the populations have the same mean using the rule of thumb. As with Brazil, shareholder-owned firms have higher average management scores than family-owned firms, though the differences are smaller in absolute terms and the respective confidence intervals are largely overlapping.
For the US: Here there is a clearer pattern linking firm size and management practices, with larger firms being managed better (for founder-owned firms, the means are the same). For larger firms, the means for family-owned firms, shareholder-owned firms and most other ownership types are very similar. The overlapping confidence intervals indicate that the data provides little evidence that the underlying population means are different.
Whenever you use confidence intervals in the above manner, to decide whether there are differences in subgroup populations, remember that 95% confidence intervals may not always contain the respective population means, so your conclusions are never definite.
- Education level of managers: Better-educated managers may have a broader knowledge of management practices (for example, ideas learned from business school) and can therefore improve the management of the firm. However, it is possible that a well-managed firm would require managers to acquire more education (for example, it could be company policy for employees to have a degree before they can be promoted to managerial positions).
- Number of competitors: The threat of being driven out by competition could motivate firms to seek better management practices (for example, adopting modern production techniques to cut costs, or retraining employees to increase productivity). However, if good management practices were easy to replicate in a given market, this would attract more firms into the market, resulting in more competition.
- Firm size: Better management practices might lead to larger firms because this would enable firms to grow while remaining productive. However, larger firms could lead to better management practices because managers have greater incentives to research and implement better management practices in order to find the most efficient way to manage their employees. (The efficiency gains from doing so increase with the number of employees.)
- In a randomized field experiment, subjects (in this case, firms in India) are randomly assigned to either a treatment or a control group. Researchers try to ensure that there are no other differences between the groups besides the treatment (as in laboratory experiments where researchers only change one thing). Since the assignment is random, we are more confident that any observed differences between the treatment and control groups after the treatment phase are due to the treatment itself (improved management practices), rather than other variables. As long as the randomization has been done properly, we can make causal statements such as ‘improved management practices caused a productivity increase’.
- The treatment and control groups had similar productivity before the management changes. After receiving ‘treatment’, the treatment group’s productivity increased whereas the control group’s productivity remained at roughly the same level. Using the rule of thumb, we can say that the improvement in productivity due to the treatment was unlikely to be due to chance after Week 32 (approximate estimate).