You can get new post notifications through RSS, email, Twitter or Facebook
Financial models and climate models
A few years ago we were assured that complex financial models used by banks could be relied on to correctly assess financial risk. This was supposed to ensure that banks would stay solvent, and the financial system stable. It turned out that insufficient back-testing, subtle pressure on those building the models to produce the “right” results, and the intrinsic problems of complex models, meant that the models were far from good enough.
We are also asked to trust complex models of climate change that predict rising temperatures. I am inclined to distrust complex models in general, but leaving that aside, are these likely to suffer the shortcomings of the bank’s risk models?
First, consider back-testing. As far as I have been able to find out, it is limited to, at most, about two or three hundred years of data. For example, this document from the IPCC discusses testing models against 20th century data. Given that these models are being used to make predictions over decades or centuries, this is grossly inadequate. It is be like testing a financial model for risk measurement on a one year horizon against an year or two of data — much worse than the banks actually did.
Given that the models predict drastic change, they really need to be back-tested against periods that include such drastic changes. The problem is we simply do not have the data.
Next the question of pressure. I do not want over-emphasise this as I think it is a less important criticism, but it is worth considering what the reaction would be if anyone produced a climate model that predicted that global warming would not happen. I think its likely that they would be labelled a “denier”, accused of being in the pay of big business (funnily enough the big businesses that have invested heavily in renewable energy, carbon trading, etc. do not ever seem to have anyone in their pay) . Even if this did not actually happen, I think it is enough of a worry to be a significant deterrent. Would you to be the one who risked their career on it?
Then come the intrinsic problems of complex models. The problem is simply that the more complex the model, the higher the model risk. The need to make choices when constructing the model will allow pre-conceptions and bias to push researchers towards a model that gives the right output.
Of course, it is quite possible that reality is even worse than the models. It is often pointed out that there could be feedback effects that will cause accelerating change. On the other hand there could be feedback that damps, or even offsets, the change. Without proper back-testing we just do not know.
Comments disabled
Sorry, comments are closed
Feed