Megaprojects are innately risky
The European Perspective on megaprojects, part 2 of 3.
Seven stadiums built for the EURO 2004 had delays and cost overruns. Why?
In part 1 we outlined the problematic stadiums built in Portugal for the EURO 2004. All of them suffered cost overruns and delays. Five of them are heavily underused and have high maintenance costs. We ended up wondering: what went wrong and how can it be prevented?
If you haven’t read part 1 yet, follow the link below:
Today, we will dig into the academic work of Bent Flyvbjerg (pronounced floo-bierg) to find out the reasons of the delays and cost deviations. We will be looking into cost estimates, who is responsible for them, what are the roles for the public and private sectors, what are the actual reasons for the deviations, and how to get them right.
We start with a bang.
Cost estimates are typically wrong
Flyvbjerg looks at projects worth hundreds of millions. Examples of these are large bridges, high-speed train connections, nuclear power plants, or oil and gas pipelines. He has documented how often these projects don't perform well, not only financially, but also socially and environmentally. Flyvbjerg curates an extensive database with thousands of project costs and schedules. He notes that "cost overruns of 50 to 100 percent are common. Cost overruns above 100 percent are not uncommon" and his first conclusion is "don't trust cost estimates".
Let's loop back to the EURO 2004. The Portuguese Court of Auditors explained in its 2005 report that the original estimate for the renovation of the works didn’t take into account the specificity of the locations, the materials, or the environments. It was simply based on this:
350€ per seat in a renovated stadium
998€ per seat in a new stadium
The cost estimate of the Municipal Stadium of Braga was 30 M €. During the works, the city mayor admitted publicly that he never believed the original estimates1, and that the costs were in reality much higher. At that time he acknowledged 73 M €; the actual cost is reaching 195 M €.
Three cracks that break a project's viability
Wrong estimates distort the project from the get-go
In Portugal, the national Government had committed to cover 25% of the estimated costs of the EURO 2004 stadiums. The real cost started to grow, but the Government financing did not.
Sunk costs can sink a project
When estimates reveal themselves inaccurate, several million euros have already been spent and many more committed. In pure rationality, it would be wise to cut the losses and stop the project. But can you imagine a politician saying "Following the financial analysis, we have seen that this stadium will not be profitable, so we're stopping the building and instead we will demolish it"? This is not likely to happen, because human biases incline us to consider all the money already spent, and push forward. In less mundane words, we fall prey of the sunk cost fallacy.
There are many incentives to provide a wrong estimate
For politicians, a low estimate is an opportunity to promote a project that "is not that expensive after all''. For contractors, the incentive is to bid lower in order to win the tender. Costs and risks are underestimated to make the project look good. When the problems start to be seen, it's too late.
The diffusion of responsibility
The train tunnel below the English channel (Eurotunnel) is another megaproject closely studied by Flyvbjerg. When it started financing, the Directors informed the potential investors that the reasonable amount of budget they were setting apart for contingencies was 10%. To be accurate, it should have been 100%. But who was accountable for that 10% figure?
Some studies cited by Flyvbjerg mention that only 44% of the environmental impact assessments are accurate. But how is it this figure fed back to the planners, for better assessments in the future?
We've seen earlier the simplistic approach used to estimate the budget for the works of the stadiums in the EURO 2004. That tournament was a matter of pride for Portugal, and the extra costs have become an uncomfortable topic since. But who was accountable for the first estimate?
Quoting Flyvbjerg:
The costs of a possible disaster do not fall on the people who made the decision. The costs and criticism fall on other people who say, “We didn’t make the decisions. We’re just here now to administer this project so don’t criticize us.” There’s a diffusion of responsibility
The public and private sector involvement
Flyvbjerg doesn't believe that megaprojects can be done either by the public or the private sector alone. They require both private capital, and clear regulations protecting the public space.
He's critical of states backing up projects financially by using sovereign guarantees. While this seems to reduce project risk, it is in fact just transferring it to the public: if the project fails, the state will cover the expenses. This can result in relaxed financial and project discipline. On top of this, governments have a conflict of interest if they are both project promoters and guardians of the public stakes. The place of the government is not championing projects, but protecting the environment, and protecting the taxpayer against financial risks.
Government should play a more active role in other areas. To start with, involving stakeholders in the planning stage. One of the main project risks identified by Flyvbjerg is the delays and extra costs produced by social protests against megaprojects. Public dissatisfaction with projects may increase simply because stakeholder groups and the public are under-informed and feel left out. The sooner the public complaints can be heard and addressed, the lower the risk.
For the private sector, Flyvbjerg has a clear proposal: let risk capital enter with 15% of the investment. The willingness of risk capital to enter is a good predictor of project profitability. For this to work, contracting must be done on the basis of performance indicators, not just on input. In other words, the requirement should be: "build an infrastructure that moves so many people per year with these constraints", instead of just "build a bridge". If risk capital is unwilling to participate, this may impede the construction of some non-profitable infrastructure, saving taxpayer money in the long run.
Something that the private sector should do less is rent-seeking. Even if a project was completely useless once built, many people would have benefited from it: engineers, architects, land owners, construction companies, and politicians who cut the ribbon. Expert project promoters, aware of the human bias about sunk costs that we saw above, take advantage of it to get their project finished no matter what. We can look again at Portugal at the Stadium of Algarve, the only one not being home of any local club, as an example of an underused structure.
What are the reasons?
Megaprojects costs are underestimated, and their benefits overestimated. Why is this?Flyvbjerg identifies three categories of explanations.
The technical explanation:
“We didn’t have the right data”
“Our forecasting technique was not good enough”
“We didn’t have enough experience to produce an accurate estimation”
“But we will do it better next time”.
The psychological explanation:
“We got carried away by optimism and disregarded a more rational analysis”
“And we might do it again the next time, because it’s only human, unless some people or some process gets us in check”.
The political-economic explanations:
“Let’s tell a very optimistic story of this project to gain the population’s approval”
“Instead of saying it will cost 100 million, let’s say it’s 75 million. It will help us obtain the contract.”
“And we will do it again next time, because it’s intentional, there’s no real accountability, and people love to hear positive stories. After all, aren’t we doing this in the interest of the city/region/country?”
Which of these reasons is better supported by data? In his paper “Survival of the Un-fittest”2, Flyvbjerg compiles data from 258 transportation megaprojects in 20 nations on five continents. These are the results:
Technical explanations don’t fit the data well.
Psychological explanations fit the data better.
“Political-economic explanations and strategic misrepresentation account well for the systematic underestimation of costs and overestimation of benefits found in the data”
If you underestimate the costs and overestimate the benefits, you’ll increase your chances of getting funding for your project. This perverse incentive is the idea behind the title: “survival of the un-fittest”.
How I met Bent Flyvbjerg
Let's take a small detour for a personal story. I am an IT Project Manager. In 2017 I was struggling to convince our customers that the estimations I was doing were correct. Spoiler: they were not.
I had presented a yearly plan in which month by month the project was always gradually advancing. At that time I didn't know it, but the World Bank has a term for this sort of planning: "EGAP - Everything Goes According to Plan". On my side, I insisted that this plan didn't include contingencies, so if new elements came up, they should be added. On the opposite side, the customer was understanding that this planning meant a strong commitment from our side and that delivery dates were carved in stone. This could only blow. When it did, the customer threatened to kick me out of the project, and I vowed to never again produce any estimate of any kind for this project.
But I was also unhappy with my plan. I knew it was going to derail. But how could I provide something more accurate? I received the suggestion to add 20% as a contingency, but I wanted to know why. Why 20% and not 40%? What was the real basis for it?
That's when I learned about "reference class forecasting" in an article by Flyvbjerg3. When producing a plan, I was inadvertently transferring my own biases into it (of being an inexpert manager, about my own knowledge of the technology, my optimism). The way to go was to look outside. To find out about similar projects done by other teams, and see how much it had cost them. It was highly likely that I would end up in similar figures. That’s what "reference class forecasting" is: predicting on the basis of similar cases.
Improving project performance
Throughout this article we have already covered the aspects proposed to improve this situation. Let's highlight them:
More transparency. Megaprojects are the most expensive ventures, and the ordinary citizen is the taxpayer: they have to have a say. Civil society must participate from the early stages. All documents should be released. Experts should be called in, not only citizens, to analyse complex documents.
Performance specs, not input specs. In order to avoid an immature discussion of the technical aspects, it is better to focus in the early stages on the requirements with respect to the economic performance, environmental sustainability, and safety performance. Reaching a consensus, for instance, that the project will be implemented only if it is self-financing and involves minimal risk for taxpayers. Until these parameters have not been established, it is not meaningful to start to identify technical solutions.
Involve risk capital. To avoid building non-profitable infrastructure.
Formulate clearly the regulatory regime. That's the job of the public administration.
Conclusion: Underestimated costs, overestimated benefits.
Megaprojects are by definition an area in which so many elements are involved, that many things can go wrong. Accurate planning is highly problematic. This applies as well to environmental impacts, and regional effects. Proponents of megaprojects claim that they will generate economic growth, but these claims are to be taken with caution. Eurotunnel, a giant investment, was generating very small impacts in the economy, and the directly affected regions were worse off (the benefits were for regions 150km away from the tunnel).
In summary, megaprojects typically have overestimated benefits and underestimated costs and risks. Cost estimates should not be trusted, and no one will be accountable for them. Both the private and the public sectors should reorient their participation. The political-economic factors play a very important role in the wrong estimations. To fix this, a number of elements must be put in place: more transparency, performance specs, limited participation of risk capital, and a clear regulatory regime.
Let's take this knowledge with us and dive in into a hot topic in the energy discussions in Europe these days: the construction of new nuclear power plants. Subscribe now to receive the third part and final part of this series in your mailbox.