Free Republic
Browse · Search
News/Activism
Topics · Post Article

Skip to comments.

GLOBAL WARMING BOMBSHELL: Hockeystick Broken
MIT Technology Review ^ | 15 October 2004 | Richard Muller

Posted on 01/13/2005 4:20:13 PM PST by neverdem

A prime piece of evidence linking human activity to climate change turns out to be an artifact of poor mathematics.

Progress in science is sometimes made by great discoveries. But science also advances when we learn that something we believed to be true isn't. When solving a jigsaw puzzle, the solution can sometimes be stymied by the fact that a wrong piece has been wedged in a key place.

In the scientific and political debate over global warming, the latest wrong piece may be the "hockey stick," the famous plot (prominently displayed by the IPCC report, 2001), published by University of Massachusetts geoscientist Michael Mann and colleagues. This plot purports to show that we are now experiencing the warmest climate in a millennium, and that the earth, after remaining cool for centuries during the medieval era, suddenly began to heat up about 100 years ago--just at the time that the burning of coal and oil led to an increase in atmospheric levels of carbon dioxide.

I talked about this at length in my December 2003 column. Unfortunately, discussion of this plot has been so polluted by political and activist frenzy that it is hard to dig into it to reach the science. My earlier column was largely a plea to let science proceed unmolested. Unfortunately, the very importance of the issue has made careful science difficult to pursue.

But now a shock: independent Canadian scientists Stephen McIntyre and Ross McKitrick have uncovered a fundamental mathematical flaw in the computer program that was used to produce the hockey stick. In his original publications of the stick, Mann purported to use a standard method known as principal component analysis, or PCA, to find the dominant features in a set of more than 70 different climate records.

But it wasn't so. McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but also it handles data normalization in a way that can only be described as mistaken.

Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called "Monte Carlo" analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!

That discovery hit me like a bombshell, and I suspect it is having the same effect on many others. Suddenly the hockey stick, the poster-child of the global warming community, turns out to be an artifact of poor mathematics. How could it happen? What is going on? Let me digress into a short technical discussion of how this incredible error took place.

In PCA and similar techniques, each of the (in this case, typically 70) different data sets have their averages subtracted (so they have a mean of zero), and then are multiplied by a number to make their average around that mean to be equal to one; in technical jargon, we say that each data set is normalized to zero mean and unit variance. In standard PCA, each data set is normalized over its complete data period; for the global climate data that Mann used to create his hockey stick graph, this was the interval 1400-1980. But the computer program Mann used did not do that. Instead, it forced each data set to have zero mean for the time period 1902-1980, and to match the historical records for this interval. This is the time when the historical temperature is well known, so this procedure does guarantee the most accurate temperature scale. But it completely screws up PCA. PCA is mostly concerned with the data sets that have high variance, and the Mann normalization procedure tends to give very high variance to any data set with a hockey stick shape. (Such data sets have zero mean only over the 1902-1980 period, not over the longer 1400-1980 period.)

The net result: the "principal component" will have a hockey stick shape even if most of the data do not.

McIntyre and McKitrick sent their detailed analysis to Nature magazine for publication, and it was extensively refereed. But their paper was finally rejected. In frustration, McIntyre and McKitrick put the entire record of their submission and the referee reports on a Web page for all to see. If you look, you'll see that McIntyre and McKitrick have found numerous other problems with the Mann analysis. I emphasize the bug in their PCA program simply because it is so blatant and so easy to understand. Apparently, Mann and his colleagues never tested their program with the standard Monte Carlo approach, or they would have discovered the error themselves. Other and different criticisms of the hockey stick are emerging (see, for example, the paper by Hans von Storch and colleagues in the September 30 issue of Science).

Some people may complain that McIntyre and McKitrick did not publish their results in a refereed journal. That is true--but not for lack of trying. Moreover, the paper was refereed--and even better, the referee reports are there for us to read. McIntyre and McKitrick's only failure was in not convincing Nature that the paper was important enough to publish.

How does this bombshell affect what we think about global warming?

It certainly does not negate the threat of a long-term global temperature increase. In fact, McIntyre and McKitrick are careful to point out that it is hard to draw conclusions from these data, even with their corrections. Did medieval global warming take place? Last month the consensus was that it did not; now the correct answer is that nobody really knows. Uncovering errors in the Mann analysis doesn't settle the debate; it just reopens it. We now know less about the history of climate, and its natural fluctuations over century-scale time frames, than we thought we knew.

If you are concerned about global warming (as I am) and think that human-created carbon dioxide may contribute (as I do), then you still should agree that we are much better off having broken the hockey stick. Misinformation can do real harm, because it distorts predictions. Suppose, for example, that future measurements in the years 2005-2015 show a clear and distinct global cooling trend. (It could happen.) If we mistakenly took the hockey stick seriously--that is, if we believed that natural fluctuations in climate are small--then we might conclude (mistakenly) that the cooling could not be a natural occurrence. And that might lead in turn to the mistaken conclusion that global warming predictions are a lot of hooey. If, on the other hand, we reject the hockey stick, and recognize that natural fluctuations can be large, then we will not be misled by a few years of random cooling.

A phony hockey stick is more dangerous than a broken one--if we know it is broken. It is our responsibility as scientists to look at the data in an unbiased way, and draw whatever conclusions follow. When we discover a mistake, we admit it, learn from it, and perhaps discover once again the value of caution.

Richard A. Muller, a 1982 MacArthur Fellow, is a physics professor at the University of California, Berkeley, where he teaches a course called "Physics for Future Presidents." Since 1972, he has been a Jason consultant on U.S. national security.


TOPICS: Business/Economy; Canada; Culture/Society; Editorial; Foreign Affairs; Government; News/Current Events; Politics/Elections; US: District of Columbia
KEYWORDS: climatechange; globalwarming; hockeystick; horsehockey; junkscience
Navigation: use the links below to view more comments.
first previous 1-5051-100101-150151-173 last
To: Yelling
I believe that the MWP may well be warmer than the early part of the 20th century, roughly on par with the average temperature of the 20th century and slightly cooler than the recent decades (say 1985 onwards).

Briffa1998 (files labeled 2001) was quoted by Soon as showing the 20th century to be warmer than any of the previous 5. The data is interesting though, it shows NH growing season average temperatures from tree rings as having most of that 20th century warmth in the 1930's and 40's.

So this seems to disprove part of your beliefs above, although you will no doubt cling tightly to them.

151 posted on 01/20/2005 4:51:55 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 145 | View Replies]

To: neverdem

bttt


152 posted on 01/20/2005 4:54:44 AM PST by Wurlitzer (I have the biggest organ in my town {;o))
[ Post Reply | Private Reply | To 1 | View Replies]

To: palmer
First, you are quoting Soon's interpretation which, as was shown above, seems to be a little biased and in some places just plain wrong.

Second, even if Soon is correct I am not sure I follow your point here. My comment was about the MWP and your comment is about a data set that covers the LIA.

My comment still stands.
153 posted on 01/20/2005 6:27:30 AM PST by Yelling
[ Post Reply | Private Reply | To 151 | View Replies]

To: Yelling
You are obviously not following my point. Soon was quoting the paper as showing anomalous 20th century warming (which is not surprising since the data starts in the LIA). I looked at the raw data and drew my own conclusion which is that the warming is during the 1930's and 40's (with cooling since). Your comment was this:

I believe that the MWP may well be warmer than the early part of the 20th century, roughly on par with the average temperature of the 20th century and slightly cooler than the recent decades (say 1985 onwards).

The data shows that there was warming in 1930's and 1940's. Your comment is at least leaning heavily.

154 posted on 01/20/2005 6:46:44 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 153 | View Replies]

To: palmer
Crowley2000 is also mentioned in Mann's paper and has data that has been thoroughly manipulated using a simulation to remove all forcing effects except those from greenhouse gases. There's no testing of the CO2 forcing hypothesis and therefore no proof of anything.

Umm, I think that you may have gotten your papers mixed up. The one that Mann referenced is "How Warm Was the Medieval Warm Period?" by Crowley and Lowery .

The one that you found the data set on was "Causes of Climate Change Over the Past 1000 Years" by Crowley alone.
155 posted on 01/20/2005 7:20:36 AM PST by Yelling
[ Post Reply | Private Reply | To 150 | View Replies]

To: palmer

But there has not been cooling since. There was a period of very slight cooling about 1940 - 1980, followed by a greater increase of warming. The warming since 1980 has been about 0.4C, lots of room for a MWP.


156 posted on 01/20/2005 7:21:56 AM PST by Yelling
[ Post Reply | Private Reply | To 154 | View Replies]

To: Yelling

Nonetheless, it is likely that Crowley used the same data tweaking in both papers. The dataset I downloaded is tweaked and can't be used to compare current proxy measurements with past ones. I'll keep downloading data sets and posting conclusions for all the ones I download no matter what the results, but I can't draw conclusions when data has been manipulated.


157 posted on 01/20/2005 7:25:12 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 155 | View Replies]

To: Yelling

0.4 C warming (subtracting the 0.1 to 0.15 degree measurement bias) is hardly unusual in the climate record. If you look at ANY of the raw data (untweaked) you will see one or two decades of dramatic warming all over the place including the LIA.


158 posted on 01/20/2005 7:27:44 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 156 | View Replies]

To: palmer
Nonetheless, it is likely that Crowley used the same data tweaking in both papers.

LOL

That is a new argument, an appeal to probability.

Go ahead and do your analysis, but please try to make sure that you actually reference the correct papers this time.

Until then.

Y.
159 posted on 01/20/2005 8:00:47 AM PST by Yelling
[ Post Reply | Private Reply | To 157 | View Replies]

To: Yelling

The dataset is from Crowley's paper, but it is the Crowley-Lowery dataset. That's what it says in the readme file. My appeal to probability seems sound in this case.


160 posted on 01/20/2005 8:07:23 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 159 | View Replies]

To: Yelling
For example they use Keigwin to show that the Sargasso had cooler temperatures during the LIA without mentioning that Keigwin thinks this is due to a shift of currents.

I downloaded the Keigwin-1996 data. It shows SST estimates on 50 year intervals with warmth estimated at 1.5C warmer 25 years ago than 925 years ago. Admittedly a crude measurement subject to ocean currents, but nonetheless matches the MWP and LIA eras.

161 posted on 01/20/2005 8:29:17 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 105 | View Replies]

To: palmer

Sorry, make that 1.5C warmer 925 years ago than 25 years ago.


162 posted on 01/20/2005 8:29:59 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 161 | View Replies]

To: palmer

Downloaded demenocal-2000 (mentioned by Soon as being "marginally warmer that at present"). The data shows the MWP being from 0.1 to 0.5 C warmer than 88 years ago (not the present). That counters the Keigwin data somewhat and doesn't allow comparison to the 20th century.


163 posted on 01/20/2005 10:22:16 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 161 | View Replies]

To: Yelling
You then said "Apparently Mann used just nine locations out of all the available data, just 5 for North America." No, that is not correct. First, it depends on which paper you are talking about. In his 98 paper that looked at the global temperature he used over 100

Downloaded the Mann-1998 data. The record is as he stated: a large warmup from the LIA. Unfortunately this record starts with the LIA. The other problem is appending measurements from 1981 through 1998 onto the end of the graph containing proxy measurements. Surface temperature measurements (with a documented 0.1 to 0.15 warm bias) are not the same as proxy measurements! The reason is the proxy temperatures measurements are a different kind of measurement which will reflect cold and warm extremes differently. That makes it impossible to say how much warmer the 80's and 90's were to other decades measured by proxies. The evidence is obvious in the data: where they overlap, the proxy and measured temperatures do not match up very well.

164 posted on 01/20/2005 10:45:40 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 132 | View Replies]

To: palmer

We seem to be going in circles. No one argues that the SST of in the Sargasso was warmer than than it is now. What Keigwin put it down to is a shift in the currents. Different currents - different temperatures. I did go through this before with WOSG.

If you want to argue that a warming in the world caused a change in the NAO which caused a warmer current to move then fine. But that is not what Soon says or that you have argued.

Wake me if you find something good.


165 posted on 01/20/2005 10:52:37 AM PST by Yelling
[ Post Reply | Private Reply | To 161 | View Replies]

To: Yelling
We seem to be going in circles.

I am making steady progress. You are the one repeating yourself. There wasn't a 0.4 C warming since 1980, there is a 0.1 to 0.15 degree bias in that estimate. The increase since 1980 is a deviation comparable to many other decades in the past and doesn't show up in some of the proxy measurements (e.g. briffa-1998).

The answers I have gotten so far are:

* there were decades in the MWP (and even in the LIA) warmer than the 80's and 90's.
* There was an increase in temperature during the 1900's as a whole, but mostly concentrated before 1950.
* There was variability between periods throughout history that match the variability in this century.
* The Mann 98 deliberately smoothed out historical data to hide that variability.
* That paper did not smooth the 1980's and 1990's and used surface measurements with bias in place of proxies.
* I can conclude, so far, that hockey stick is an artifact.
* But I can't make the MWP warmth conclusion yet globally, I need to look at more data.

166 posted on 01/20/2005 11:27:20 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 165 | View Replies]

To: palmer
You are correct I am repeating myself because I had this same argument with WOSG, We first covered Keigwin and I showed that he thinks that the change was caused by a shift in currents. Then you bring him up!

Same with deMenocal. I already posted about him above yet you bring him up again. I even quoted him. Someone asked him about the use of his data to show a MWP and he said "Peter deMenocal, an associate professor at Columbia University's Lamont-Doherty Earth Observatory, used sediment records off the coast of Africa as a proxy for ocean-surface temperatures. He says Mr. Soon and his colleagues could not justify their conclusions that the African record showed the 20th century as being unexceptional. "My record has no business being used to address that question," the Columbia scientist says. "It displays some ignorance putting it in there to address that question."

This is what I mean by going around in circles. When you bring up something new I will consider it.

In regards to your answers, I wish you well and an looking forward to seeing you post something about them. However as you work through the data, please ask yourself if you have corrected for all the things that you accused Mann of not correcting for.

In regards to Mann, if you are convinced then you should publish your results. However nothing you have yet posted backs up one of your assertions.

Now, I am engaged in another good discussion (this one on radiative forcing functions) so good night.

Y.
167 posted on 01/20/2005 4:00:20 PM PST by Yelling
[ Post Reply | Private Reply | To 166 | View Replies]

To: Yelling
Peter deMenocal, an associate professor at Columbia University's Lamont-Doherty Earth Observatory, used sediment records off the coast of Africa as a proxy for ocean-surface temperatures. He says Mr. Soon and his colleagues could not justify their conclusions that the African record showed the 20th century as being unexceptional.

That's essentially what I said as well. Maybe it wasn't clear or maybe you just skimmed it and thought I used it to justify the conclusion that 20th century was unexceptional. No, I did not. Since his data is very coarse and at least 88 years old, it says nothing about the 20th century, exceptional or otherwise.

Well you are welcome to end this discussion without addressing my concerns about Mann. As for your suggestion for me to write a paper, I would accept Soon's with all its errors as far superior to anything I could put together. However since I have not made up my mind like you have, I am willing to keep researching the raw data and commenting on all of it whether it supports my view or not. The example above is just one case, Jones98 was another that could support the hockey stick provided I can figure out his normalization technique for 1960 onwards. Otherwise I must judge it to be inconclusive (or tweaked and therefore useless like the Crowley-Lowery dataset).

But you are also welcome to provide your own analyses of the data. I think it would serve your case better than simply pretending to scientifically review the Soon paper.

168 posted on 01/20/2005 6:46:12 PM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 167 | View Replies]

To: palmer
That's essentially what I said as well.

Sigh, no, you did not. What you said is that the paper shows temperatures that do not support some aspect of warming during the MWP. What I am saying is that the paper can't be used for that since it doesn't measure that. What the author says is that the wind and current pattern shifted to cause cooler currents with more upwelling. This caused sea surface temperatures to drop. It is not a reflection of any surface temperature. You go to a beach on a very warm day with an offshore wind and the water is cooler than you expect. That is upwelling. nothing to do with temperature.

However, I do find it somewhat amusing that you are prepared to accept any aspect of what a paper says to support your ideas and yet you accuse me of being biased. Read what the paper says and then you will be in a much better position to comment on it.

Regarding CL, could you comment on how it was "tweaked" and therefore useless?

Finally, regarding Soon! Lets see. It was presented by WOSG and I showed how it is a very poor attempt. You then read it and seemed to feel that it was sound. So when I do a critical analysis of it you say that I am "simply pretending to scientifically review the Soon paper." So I would guess that you are simply pretending to admit that there are errors in it?

I am not leaving the debate, but I do feel that I have done more than my share of the research. I have had to dig out several papers, read them and try to understand what they show and then present and defend my point of view against people who don't wish to read the papers. While I find reading technical papers somewhat enjoyable it is time consuming. If you wish a real debate about this then post some real comments that I can look at and review.

One final comment - I think you sell yourself short. I feel that you could actually do better than Soon did on that paper. I am pretty sure I could even though I am an engineer, not a climatologist - but then again Soon isn't a climatologist either. That may explain something.

Regards,

Y.
169 posted on 01/21/2005 4:10:33 AM PST by Yelling
[ Post Reply | Private Reply | To 168 | View Replies]

To: Yelling
upwelling

Took another look at the demenocal-2000 data and he mentions strong seasonal upwelling. If this data is only measuring strength of that, then I'll ignore it and move on to the next data set.

You then read it and seemed to feel that it was sound. So when I do a critical analysis of it you say that I am "simply pretending to scientifically review the Soon paper." So I would guess that you are simply pretending to admit that there are errors in it?

You have provided no scientific review of the Soon hypothesis and conclusions as I did in post 131. You merely presented a bunch of nitpicks that you found in various GW sites about the errors he made. You claimed a "devastating" attack, but provided no systematic review of the paper. I on the other hand have looked and continue to look at the actual data to see if it supports Soon's hypothesis. Because the bottom line is that irregardless of what Soon said and how he said it, I (and most other people on this forum) want to know whether or not his hypothesis it true.

As for CL, the notes from his readme are as follows:

Removal of all forcing except greenhouse gases from the ~1000 year time series results in a residual with a very large late 20th century warming that closely agrees with the response predicted from greenhouse gas forcing.

His assumption of CO2 forcing is not supported by any raw data in his possession, it is a hypothesis only supported by simulations. He then smoothed his historical data based on assumed statistics about natural mechanisms that also labels "forcing" although they are not the same thing (i.e. particulate cooling is not forcing). So his historical data was smoothed to (conveniently) eliminate any MWP or LIA warming. He did not choose to release his raw data.

170 posted on 01/21/2005 4:51:12 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 169 | View Replies]

To: palmer
Humm, I did provide several "nit-picks" that showed that Soon was not interpreting the data correctly. And the "nit-picks" that I found were based on papers that were presented to me by you and WSOG. So in fact statistically things aren't looking too good for Soon (and no, you can't accuser me of bias since I had no hand in selecting the papers). Since he has not understood these, how many others has he got wrong? I don't intend to spend my time going through all his references, however I am interested in the ones where he seems to have his facts straight and if you can recommend any (after you have reviewed his paper to ensure that he has his facts right) I would appreciate knowing about them.

I on the other hand have looked and continue to look at the actual data to see if it supports Soon's hypothesis. Because the bottom line is that irregardless of what Soon said and how he said it, I (and most other people on this forum) want to know whether or not his hypothesis it true.

Err, I hate to pop your bubble, but it was you who wanted to accept Soon as given. I, on the other hand, did look at the data and the papers and presented the information. I do find your comments about me posting from a GW site ironic. If I post from somewhere else I generally reference or at least say "from a site" and anyone can ask me if they wish. If you can find any evidence of me posting from a "GW site", please post it. Right or wrong, my comments are my own.

On the otherhand earlier on this thread I came across a post, cut and pasted with out reference from the CO2 Science site. Your comments on that please?

However in regards to the truth. As you are, I am also merely a lone individual in search of scientific truth. Although you seem to have a strong interest in rhetoric as well.

His assumption of CO2 forcing is not supported by any raw data in his possession, it is a hypothesis only supported by simulations. He then smoothed his historical data based on assumed statistics about natural mechanisms that also labels "forcing" although they are not the same thing (i.e. particulate cooling is not forcing). So his historical data was smoothed to (conveniently) eliminate any MWP or LIA warming. He did not choose to release his raw data.

Like a spiral in a spiral and a wheel within a wheel ... and around we go in circles. You are not looking at the dataset or methodology that was given in the paper that I referenced because you are not looking at the relevant paper. There was nothing in that paper talking about CO2 forcings! You came up with that by looking at the data for another paper of his. There is a reference to the data that he used but nothing about the methodology. So without knowing anything about it you say that the data is smoothed, etc. Your comments may be valid as applied to Crowley 2000, but mean nothing as applied to Crowley and Lowery 2000. Your appeal to probability is looking like a long-shot.

Oh, and a small point but in climatology particulate cooling is considered a forcing.

Again, please wake me if you find anything of interest.
171 posted on 01/21/2005 9:07:00 AM PST by Yelling
[ Post Reply | Private Reply | To 170 | View Replies]

To: Lx

he'll find some way to brand it heresy.
the sheeple will baaa right on along.
mark me - you'll see.


172 posted on 01/27/2005 8:11:27 AM PST by King Prout (trolls survive through a form of gastroenterotic oroborosity, a brownian "perpepetual movement")
[ Post Reply | Private Reply | To 7 | View Replies]

To: Yelling
Critique of the Mann et al Northern Hemisphere Average Temperature Reconstruction
BY
* Steven McIntyre Toronto, Ontario
* Ross McKitrick Department of Economics, University of Guelph.
http://www.uoguelph.ca/~rmckitri/research/trc.html

I looked at the their link climate2003.com where they talk some more about a growth pulse for bristlecone pines which skewed Mann's 1998 results. I also read the rebuttal at realclimate.org/index.php?p=111 which basically dismisses the critics because they use a linear correlation coefficient. Also says "we've moved on" and they are investigating the forcing that causes global warming.

I have to read more of their arguments, but they sound a lot like yours. Attack the scientific credibility of the critics, point out various errors the most basic of which is that the critics used only 10 chronologies instead of 50 to cover the whole time period (1400 to 1980) instead of step-wise reconstruction using all 50 chronologies. This is in their words "sharply at odds with virtually all other empirical and model-based estimates of hemispheric temperature trends in past centuries". That sounds more like choir practice than science, but I'll have to study this further.

173 posted on 01/28/2005 3:31:28 AM PST by palmer ("Oh you heartless gloaters")
[ Post Reply | Private Reply | To 171 | View Replies]


Navigation: use the links below to view more comments.
first previous 1-5051-100101-150151-173 last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
News/Activism
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson