Posted on 01/13/2005 4:20:13 PM PST by neverdem
A prime piece of evidence linking human activity to climate change turns out to be an artifact of poor mathematics.
Progress in science is sometimes made by great discoveries. But science also advances when we learn that something we believed to be true isn't. When solving a jigsaw puzzle, the solution can sometimes be stymied by the fact that a wrong piece has been wedged in a key place.
In the scientific and political debate over global warming, the latest wrong piece may be the "hockey stick," the famous plot (prominently displayed by the IPCC report, 2001), published by University of Massachusetts geoscientist Michael Mann and colleagues. This plot purports to show that we are now experiencing the warmest climate in a millennium, and that the earth, after remaining cool for centuries during the medieval era, suddenly began to heat up about 100 years ago--just at the time that the burning of coal and oil led to an increase in atmospheric levels of carbon dioxide.
I talked about this at length in my December 2003 column. Unfortunately, discussion of this plot has been so polluted by political and activist frenzy that it is hard to dig into it to reach the science. My earlier column was largely a plea to let science proceed unmolested. Unfortunately, the very importance of the issue has made careful science difficult to pursue.
But now a shock: independent Canadian scientists Stephen McIntyre and Ross McKitrick have uncovered a fundamental mathematical flaw in the computer program that was used to produce the hockey stick. In his original publications of the stick, Mann purported to use a standard method known as principal component analysis, or PCA, to find the dominant features in a set of more than 70 different climate records.
But it wasn't so. McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but also it handles data normalization in a way that can only be described as mistaken.
Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called "Monte Carlo" analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!
That discovery hit me like a bombshell, and I suspect it is having the same effect on many others. Suddenly the hockey stick, the poster-child of the global warming community, turns out to be an artifact of poor mathematics. How could it happen? What is going on? Let me digress into a short technical discussion of how this incredible error took place.
In PCA and similar techniques, each of the (in this case, typically 70) different data sets have their averages subtracted (so they have a mean of zero), and then are multiplied by a number to make their average around that mean to be equal to one; in technical jargon, we say that each data set is normalized to zero mean and unit variance. In standard PCA, each data set is normalized over its complete data period; for the global climate data that Mann used to create his hockey stick graph, this was the interval 1400-1980. But the computer program Mann used did not do that. Instead, it forced each data set to have zero mean for the time period 1902-1980, and to match the historical records for this interval. This is the time when the historical temperature is well known, so this procedure does guarantee the most accurate temperature scale. But it completely screws up PCA. PCA is mostly concerned with the data sets that have high variance, and the Mann normalization procedure tends to give very high variance to any data set with a hockey stick shape. (Such data sets have zero mean only over the 1902-1980 period, not over the longer 1400-1980 period.)
The net result: the "principal component" will have a hockey stick shape even if most of the data do not.
McIntyre and McKitrick sent their detailed analysis to Nature magazine for publication, and it was extensively refereed. But their paper was finally rejected. In frustration, McIntyre and McKitrick put the entire record of their submission and the referee reports on a Web page for all to see. If you look, you'll see that McIntyre and McKitrick have found numerous other problems with the Mann analysis. I emphasize the bug in their PCA program simply because it is so blatant and so easy to understand. Apparently, Mann and his colleagues never tested their program with the standard Monte Carlo approach, or they would have discovered the error themselves. Other and different criticisms of the hockey stick are emerging (see, for example, the paper by Hans von Storch and colleagues in the September 30 issue of Science).
Some people may complain that McIntyre and McKitrick did not publish their results in a refereed journal. That is true--but not for lack of trying. Moreover, the paper was refereed--and even better, the referee reports are there for us to read. McIntyre and McKitrick's only failure was in not convincing Nature that the paper was important enough to publish.
How does this bombshell affect what we think about global warming?
It certainly does not negate the threat of a long-term global temperature increase. In fact, McIntyre and McKitrick are careful to point out that it is hard to draw conclusions from these data, even with their corrections. Did medieval global warming take place? Last month the consensus was that it did not; now the correct answer is that nobody really knows. Uncovering errors in the Mann analysis doesn't settle the debate; it just reopens it. We now know less about the history of climate, and its natural fluctuations over century-scale time frames, than we thought we knew.
If you are concerned about global warming (as I am) and think that human-created carbon dioxide may contribute (as I do), then you still should agree that we are much better off having broken the hockey stick. Misinformation can do real harm, because it distorts predictions. Suppose, for example, that future measurements in the years 2005-2015 show a clear and distinct global cooling trend. (It could happen.) If we mistakenly took the hockey stick seriously--that is, if we believed that natural fluctuations in climate are small--then we might conclude (mistakenly) that the cooling could not be a natural occurrence. And that might lead in turn to the mistaken conclusion that global warming predictions are a lot of hooey. If, on the other hand, we reject the hockey stick, and recognize that natural fluctuations can be large, then we will not be misled by a few years of random cooling.
A phony hockey stick is more dangerous than a broken one--if we know it is broken. It is our responsibility as scientists to look at the data in an unbiased way, and draw whatever conclusions follow. When we discover a mistake, we admit it, learn from it, and perhaps discover once again the value of caution.
Richard A. Muller, a 1982 MacArthur Fellow, is a physics professor at the University of California, Berkeley, where he teaches a course called "Physics for Future Presidents." Since 1972, he has been a Jason consultant on U.S. national security.
I downloaded the Keigwin-1996 data. It shows SST estimates on 50 year intervals with warmth estimated at 1.5C warmer 25 years ago than 925 years ago. Admittedly a crude measurement subject to ocean currents, but nonetheless matches the MWP and LIA eras.
Sorry, make that 1.5C warmer 925 years ago than 25 years ago.
Downloaded demenocal-2000 (mentioned by Soon as being "marginally warmer that at present"). The data shows the MWP being from 0.1 to 0.5 C warmer than 88 years ago (not the present). That counters the Keigwin data somewhat and doesn't allow comparison to the 20th century.
Downloaded the Mann-1998 data. The record is as he stated: a large warmup from the LIA. Unfortunately this record starts with the LIA. The other problem is appending measurements from 1981 through 1998 onto the end of the graph containing proxy measurements. Surface temperature measurements (with a documented 0.1 to 0.15 warm bias) are not the same as proxy measurements! The reason is the proxy temperatures measurements are a different kind of measurement which will reflect cold and warm extremes differently. That makes it impossible to say how much warmer the 80's and 90's were to other decades measured by proxies. The evidence is obvious in the data: where they overlap, the proxy and measured temperatures do not match up very well.
We seem to be going in circles. No one argues that the SST of in the Sargasso was warmer than than it is now. What Keigwin put it down to is a shift in the currents. Different currents - different temperatures. I did go through this before with WOSG.
If you want to argue that a warming in the world caused a change in the NAO which caused a warmer current to move then fine. But that is not what Soon says or that you have argued.
Wake me if you find something good.
I am making steady progress. You are the one repeating yourself. There wasn't a 0.4 C warming since 1980, there is a 0.1 to 0.15 degree bias in that estimate. The increase since 1980 is a deviation comparable to many other decades in the past and doesn't show up in some of the proxy measurements (e.g. briffa-1998).
The answers I have gotten so far are:
* there were decades in the MWP (and even in the LIA) warmer than the 80's and 90's.
* There was an increase in temperature during the 1900's as a whole, but mostly concentrated before 1950.
* There was variability between periods throughout history that match the variability in this century.
* The Mann 98 deliberately smoothed out historical data to hide that variability.
* That paper did not smooth the 1980's and 1990's and used surface measurements with bias in place of proxies.
* I can conclude, so far, that hockey stick is an artifact.
* But I can't make the MWP warmth conclusion yet globally, I need to look at more data.
That's essentially what I said as well. Maybe it wasn't clear or maybe you just skimmed it and thought I used it to justify the conclusion that 20th century was unexceptional. No, I did not. Since his data is very coarse and at least 88 years old, it says nothing about the 20th century, exceptional or otherwise.
Well you are welcome to end this discussion without addressing my concerns about Mann. As for your suggestion for me to write a paper, I would accept Soon's with all its errors as far superior to anything I could put together. However since I have not made up my mind like you have, I am willing to keep researching the raw data and commenting on all of it whether it supports my view or not. The example above is just one case, Jones98 was another that could support the hockey stick provided I can figure out his normalization technique for 1960 onwards. Otherwise I must judge it to be inconclusive (or tweaked and therefore useless like the Crowley-Lowery dataset).
But you are also welcome to provide your own analyses of the data. I think it would serve your case better than simply pretending to scientifically review the Soon paper.
Took another look at the demenocal-2000 data and he mentions strong seasonal upwelling. If this data is only measuring strength of that, then I'll ignore it and move on to the next data set.
You then read it and seemed to feel that it was sound. So when I do a critical analysis of it you say that I am "simply pretending to scientifically review the Soon paper." So I would guess that you are simply pretending to admit that there are errors in it?
You have provided no scientific review of the Soon hypothesis and conclusions as I did in post 131. You merely presented a bunch of nitpicks that you found in various GW sites about the errors he made. You claimed a "devastating" attack, but provided no systematic review of the paper. I on the other hand have looked and continue to look at the actual data to see if it supports Soon's hypothesis. Because the bottom line is that irregardless of what Soon said and how he said it, I (and most other people on this forum) want to know whether or not his hypothesis it true.
As for CL, the notes from his readme are as follows:
Removal of all forcing except greenhouse gases from the ~1000 year time series results in a residual with a very large late 20th century warming that closely agrees with the response predicted from greenhouse gas forcing.
His assumption of CO2 forcing is not supported by any raw data in his possession, it is a hypothesis only supported by simulations. He then smoothed his historical data based on assumed statistics about natural mechanisms that also labels "forcing" although they are not the same thing (i.e. particulate cooling is not forcing). So his historical data was smoothed to (conveniently) eliminate any MWP or LIA warming. He did not choose to release his raw data.
he'll find some way to brand it heresy.
the sheeple will baaa right on along.
mark me - you'll see.
I looked at the their link climate2003.com where they talk some more about a growth pulse for bristlecone pines which skewed Mann's 1998 results. I also read the rebuttal at realclimate.org/index.php?p=111 which basically dismisses the critics because they use a linear correlation coefficient. Also says "we've moved on" and they are investigating the forcing that causes global warming.
I have to read more of their arguments, but they sound a lot like yours. Attack the scientific credibility of the critics, point out various errors the most basic of which is that the critics used only 10 chronologies instead of 50 to cover the whole time period (1400 to 1980) instead of step-wise reconstruction using all 50 chronologies. This is in their words "sharply at odds with virtually all other empirical and model-based estimates of hemispheric temperature trends in past centuries". That sounds more like choir practice than science, but I'll have to study this further.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.