Introduction
The Climate Change Minister James Shaw has published a Risk Assessment of the effects of global climate change on New Zealand during the next 5 years.
It is a massive document which which will form the basis of a National Adaptation Plan in the next few months. This Plan is expected to “transform the economy” and have major impacts on private property rights.
But the document is downright weird. It identifies dozens of Bad Things that just might happen some day, but makes no attempt to guess how probable they are. It doesn’t say whether they will happen or not. If any of them were ever to happen, the Report doesn’t say whether that would likely be in 10 years or 100 years or 1000 years.
The Report baldly asserts: “Climate change risk assessment requires more emphasis on consequences than on likelihood” [Emphasis added. The result was 100% focus on consequences and zero focus on likelihood]. It doesn’t say why.
Risk assessments are a part of our everyday lives. We all worry about possible future changes or events that might upset our carefully-laid plans. So we ask ourselves two questions:
- How likely is it that the worrying event will happen?
- If it did happen, how bad would it be?
Without answers to both questions, we can’t begin to assess our options. We can’t decide whether the risk is tolerable, or whether it is so hazardous that we must abandon our hopes and dreams. We are left paralysed.
So why do we have only half of a true risk assessment?
My informed guess is that the probability of all these threats is vanishingly small. I’m prepared to lay odds that there will be no detectable acceleration in sea level rise around New Zealand coasts in the next 50 years, let alone the next five. There has been no recorded increase in New Zealand’s average droughts, floods or storms in the past 50 years and the IPCC expects no increases in the next 50.
Niels Bohr is credited with saying: ““Prediction is very difficult, especially if it’s about the future!”.
Some might feel sympathy for the MfE apparatchiks who were charged with fashioning a really scary future out of whole cloth. But Tailrisk Economics has no such sympathy. It assesses the assessment as a disgraceful charade, with an average evidence quality score of 3 out of 10.
The intention to deceive is made evident by the authors’ reliance on the ultra-extreme RCP8.5 as the most likely scenario for greenhouse gas emissions over the next 80 years. Nobody could or does believe this. Its use underscores the dismal fact that this political Report was never intended to predict what might actually happen in the real world.
Risk Assessments are not Threat Assessments
The redoubtable Ian Harrison of Tailrisk Economics has published a scathing review of the National Climate Change Risk Assessment (NCCRA) prepared by the Climate Commission in 2020. He found the average evidence quality score to be a scandalously low 3.09 out of 10.
This document will be the foundation for further “economic transformation” and policies that will eventually cost New Zealanders hundreds of millions of dollars. Yet it is not fit for purpose. In a recent post, Ian Harrison calls it “A Case of Science Denial”.
In the view of the Commission, it doesn’t matter if climate-related threats are never realised or never even the least bit likely to happen. Their report just talks about how bad (not how likely) a list of improbable things “could” be, or “might” be, or “can’t be ruled out” at some unmentioned time in the indeterminate future.
What is a NCCRA?
Part 1C of the Climate Change Response Act 2002 deals with Adaptation, which New Zealand is to achieve by first identifying significant risk areas and then publishing a plan to deal with them.
Risks are assessed by way of 5-year Risk Assessments (NCCRAs) prepared by the Climate Commission after extensive consultation. However, the First (precedent) report, which is required by s 5ZR to be prepared by the Minister alone, was quietly tabled in Parliament during last year’s election campaign.
There has been no opportunity for any debate on the Report during the present Parliament. Neither the legislation nor the published bureaucratic procedure make any provision for any form of audit or appeal or review prior to 2025.
The Act (s 5ZP(1)(a)) requires the report to “assess the risk … from effects of climate change.”
Although identifying risks is a necessary precursor, the key term is “assess” – meaning “to judge or decide the amount, value, quality, or importance of something”[1].
The next subsection requires the risks to be ranked. The report must “identify the most significant risks to New Zealand, based on the nature of the risks, their severity and the need for co-ordinated steps to respond to those risks in the next 6-year period.”
The meaning of the phrase “most significant risks” is fundamental, and requires the quantification and ranking of potential hazards. A textbook (per Wikipedia) describes risk assessment[2] as involving: “identification of risk (what can happen and why), the potential consequences, the probability of occurrence, the tolerability or acceptability of the risk, and ways to mitigate or reduce probability of the risk.”
Put simply: quantification = consequences x probability;
significance = tolerability + mitigation potential
On my reading of the Ministry for the Environment’s voluminous documentation, the First NCCRA entirely omits the requisite element of “probability”. Without that, the report’s ranking efforts are useless, and the document is not fit for purpose.
What the Ministry did
MfE and its endless consultants have produced reams of paper describing their methodology and reasoning. Transparency is rampant – if only because their stated objective[3] is to capture the shape of NCCRAs for many years into the future. This ambition makes it particularly important that this precedent report’s approach to probability measurement is not open to question.
In 2019, before the new Bill was even reported back, the MfE published its “Framework” for the NCCRA for Aotearoa New Zealand. This was based on the methodology used by the UK Climate Committee[4] in the production of its January 2017 report on the “UK Climate Change Risk Assessment”.
Nine months later, the MfE project team described their own approach in great detail in a “Method Report” published at the same time as the First NCCRA itself. And these are further supplemented by a Technical Report which is said to lay out the evidence base for the team’s conclusions.
This paper blizzard (deliberately?) obscures any quantification of probability. A “process overview” diagram in Method p7 discloses that none of the eight steps involves an analysis of the likelihood component. The closest step seems to be 2.3.3 (Risk development and scoring) which is said to: “Assess exposure and vulnerability[5]”. However, “exposure” does not seem to include likelihood of occurrence, as para 2.3.3 reverts to “magnitude of consequence rating” and admits “the analysis of exposure and vulnerability was mostly qualitative” (ie no quantification).
In Technical, each of seven major risks has an “Exposure” subheading. None of them contain any discussion of probability at all.
The First NCCRA document at p22 sets out “Key “Parameters” of the methodology. They include:
“Climate change risk assessment requires more emphasis on consequences than on likelihood.[6] Risk is framed using the elements of hazard, exposure and vulnerability, with the overlap defining the risk. Risk is a function of climate hazards, the degree to which values are exposed to the hazard and their vulnerability to its effects (Ministry for the Environment, 2019).”
In consequence, the report is dominated by “consequences” and “likelihood” is ignored.
These MfE documents comprise a weighty compendium of alarm about “possible” consequences of temperatures that “might” happen, without the slightest consideration of likelihood[7]. A remote chance is given equal weighting with a near certainty. Uncertainties are unmentioned, unknowns[8] are ignored, and timing is given scant consideration.
In my view, s 5ZP requires the Minister to provide a report that ranks the statistical significance[9] of future risks. The tabled document is not that report.
RCP8.5
The Ministry says that it used the extreme 8.5 scenario to scope the “possible” severity of consequences (Step 1) – but not for risk management proposals[10]. That “worst-case” information is meaningless in the absence of a probability assessment – and it assumes (without justification) that severity is always linear.
In any case, Ms Lemire’s description is misleading. According to the attached Process Overview, RCP8.5 is used to “assess exposure and vulnerability” for both near-term and long term timeframes at Step 2.2 – the core of the whole process.
I will not address here the shortcomings of RCP8.5. Obviously, it should not have been used in any situation where probability mattered[11]. The larger question is whether a NCCRA Report must try to quantify future risks.
Probabilities
A NCCRA should not only rank relative probabilities, but must also quantify the absolute probability of key future events. That goes to tolerability – an outside chance that sea level rise might accelerate is far more tolerable than a near certainty of flooding.
This question raises a series of cascading questions as to the statistical probability that:
- the selected RCP will eventuate
- the CMIP6 models will prove accurate
- downscaling global projections will accurately reveal future local temperatures
- future rates of local relative sea level rise, drought, precipitation, storms, etc will correlate closely with actual temperature increases
None of these questions are easy. As the IPCC has made clear[12]:
“In climate research and modelling, we should recognise that we are dealing with a coupled nonlinear chaotic system, and therefore that the long-term prediction of future climate states is not possible.
The words “probable” or “likely” are not used in the Act. However, the very word “risk” denotes uncertainty, and common sense demands that any risk assessment must distinguish a ‘high risk’ from a ‘low risk’ occurrence. To compare their relative ‘significance’, the risks of each consequence need to be quantified.
Timeframes
An important aspect of probability is the timing of potential consequences. Predictions of events occurring decades in the future have uncertainty levels close to 100% and are worse than useless – they obfuscate and mislead.
NIWA’s projections of future temperature increases are miniscule over the next 10 years and remain insignificant for 20-30 years. Within a normal planning horizon of 20 years, on the basis of RCP4.5, no significant SLR drought or extreme weather is expected. In essence, there is no real need for a NRCCA until the Climate Commission produces a real one in 2025.
The MfE methodology does not establish a planning horizon. The 6-year period of s 5ZP only comes into play when “urgency” is being considered in Step 3.3 (final step).
Public Consequences
The primary purpose of a NCCRA is to facilitate risk management. It must identify significant risks which need to be addressed in a national adaptation plan (s 5ZS). If the report wrongly identifies low-probability risks as being most “significant”, national resources will be squandered. If it fails to identify high-probability risks, people and property will be unnecessarily exposed. The potential scale of such errors is very large.
Mandatory reporting of climate change risks by public companies is about to be introduced in line with the international Task Force on Climate-related Financial Disclosures. Public consultation on Ministry Discussion Papers is complete and regulations are imminent. The starting point for company risk assessments will inevitably be the current national assessment.
The Ministry provides adaptation guidance for local government. If, for example, the NCRRA highlights the risk of local flooding (from accelerated sea level rise or increased precipitation), the Council would be expected to increase the capacity of its storm water pipes – at massive cost[13]. And to direct the progressive migration of seaside homes, with value-destroying and insurance consequences.
A recent amendment to the Resource Management Act 1991 requires all Councils to incorporate all climate change risks into all planning processes[14]. The NCCRA will be the primary, if not the only, resource for ascertaining risks.
Degrees of threat are irrelevant
As a final comment: It’s no longer a surprise that the Commission used the implausible RCP8.5 to identify and appraise risks – why wouldn’t they, when likelihood is not a relevant criterion?
The Commission’s methodology is simple : just list all the bad things you can imagine and then study how bad they could possibly be. It doesn’t matter that they will never happen.
Footnotes:
[2] See also ‘risk management’: Once risks have been identified, they must then be assessed as to their potential severity of impact and to the probability of occurrence.
[3] Method Report p 6: “The current NCCRA lays the groundwork for the next by documenting the assessment and engagement method in detail, and providing the Ministry for the Environment (the Ministry) and the Commission with the tools (spreadsheets and engagement materials) as well as raw data and records of engagement. This gives the Ministry and the Commission the option of building on the outputs from this NCCRA, to develop the information and evidence base for the next one. These records may also be of use for other national and local/regional risk assessments.”
[4] The UK regime was the precedent followed by our 2019 Bill.
[5] Vulnerability is a combination of adaptive capacity and sensitivity.
[6] Emphasis added. The result was 100% focus on consequences and zero focus on likelihood.
[7] It is ‘possible’ that little green men from a distant galaxy will force the next generation to use coal for their electricity needs. But it has low probability.
[8] Including UNK-UNKs
[9] An event that has > 5% chance of occurring should be seen as irrelevant. If > 0.005%, it should not even be mentioned.
[10] Which form no part of the RCCRA, in any event
[11] I understand its likelihood has been put at 2% (ie 98% unlikely)
[12] In AR4 (2007), WG1. See also here.
[13] A friend altering his home driveway has been required by the Council to add extra drainage because climate change will render current drainage inadequate
[14] MfE: Under the RMA local government is required to consider the effects of a changing climate on communities. It is also required to incorporate climate change into existing frameworks, plans, projects and standard decision-making procedures. A climate change perspective is now integrated into activities such as flood management, water resources, planning, building regulations and transport.