Posted on 11/28/2009 7:14:48 PM PST by Sneakyuser
Where’s the data? 27 November 2009
Much of the discussion in recent days has been motivated by the idea that climate science is somehow unfairly restricting access to raw data upon which scientific conclusions are based. This is a powerful meme and one that has clear resonance far beyond the people who are actually interested in analysing data themselves. However, many of the people raising this issue are not aware of what and how much data is actually available.
Therefore, we have set up a page of data links to sources of temperature and other climate data, codes to process it, model outputs, model codes, reconstructions, paleo-records, the codes involved in reconstructions etc. We have made a start on this on a new Data Sources page, but if anyone has other links that we’ve missed, note them in the comments and we’ll update accordingly.
The climate science community fully understands how important it is that data sources are made as open and transparent as possible, for research purposes as well as for other interested parties, and is actively working to increase accessibility and usability of the data. We encourage people to investigate the various graphical portals to get a feel for the data and what can be done with it. The providers of these online resources are very interested in getting feedback on any of these sites and so don’t hesitate to contact them if you want to see improvements.
(Excerpt) Read more at realclimate.org ...
The problem is more than just access to raw data. It is knowing which data has been used in which papers. It is knowing how the data has been manipulated and the justification of that manipulation. This is particularly true in a field that requires as much statistical analysis as paleoclimatology.
Replication of results is key to the scientific method. Data and data analysis should be released with all published papers.
It should be released when requested for academic study. And it should definitely be released when demanded under the FOIA.
Comment by JPSobel — 27 November 2009 @ 9:59 PM
Just a question. Is there anywhere one can view the purloined emails to form one’s own opinion about how damaging they may be? I am deeply concerned for example by Monbiot’s opinions on the content and would like to judge for myself. Thanks.
[Response: The easiest way is through here. But some of the emails are strangely truncated, not sure why.
Please note that these are illegally obtained, though curiosity is a powerful driver.... - gavin]
Comment by Dr. Alan Keller — 27 November 2009 @ 10:10 PM
Agree totally with #1 JP Sobel – I’ve requested data from Briffa and never had a reply to numerous emails. I have never experienced that before in my entire scientific career in the many science fields I work in.
I’ve always had an email back in reply with data given freely and in a friendly, helpful manner. In fact, one of the past posts on this site about this subject now reflects poorly on the group as a whole in light of the leaked CRU emails. Better to post all data, all algorithms and all methodology at time of publication so others can replicate with ease. Then, when all of it has to released under FOI request, there will be no emails, papers or statements that can be “taken out of context”.
[Response: There will always be people to take things out of context - whether these things are released via FOI or stolen. If you want Briffa's data, go to the NOAA Paleoclimate link. - gavin]
Comment by Richard — 27 November 2009 @ 10:19 PM
I recommend including a link to the world data center for greenhouse gases: http://gaw.kishou.go.jp/wdcgg/
Comment by tamino — 27 November 2009 @ 10:42 PM
if you *really* want to see how scientists talk behind closed doors . . . . see the following video! http://www.youtube.com/watch?v=-VRBWLpYCPY&feature=player_embedded Gavin is the co-author on the far left. Comment by Jeffrey Park — 27 November 2009 @ 10:47 PM
Gavin,
all you have to do is go open source–no need to hide and invent excuses, you are obviously not very good at that anyway.
Release all the data (you can keep the 2% of inexplicably protected data that you might have) even if it is just 98% of the total and also release the codes that were used to create/prove the conclusions/graphs that were published.
It is very simple … people do this [public release of specific work] daily .. see linux and OSS, if you don’t know how to do it and how it works.
The context is quite clear when it comes to the e-mails of the CRU folks … the only way out, for your gang, is to go open source.
[Response: GISTEMP has. Why aren't you interested in that? - gavin] Comment by donQ — 27 November 2009 @ 10:51 PM
Gavin,
thank you for [Response: GISTEMP has. Why aren't you interested in that? - gavin] …
unfortunately this is insufficient.
1) There is no source code provided
2) There is no coherent data provided (within the context of how such data is used to provide results)
I also have gone through the codes available in the CRU FOI2009.zip — those source files betray astounding level of incompetence from several angles … 1) management 2) computer-science 3) quality assurance. So … again, release the codes that according to you are used to generate your results.
Perhaps you weren’t aware of it, but the page you provide links to
http://data.giss.nasa.gov/gistemp/sources/GISTEMP_sources.tar.gz which is not available according to your web-site. So again, where is the source code?
[Response: Not sure why that isn't up, it was before. Maybe they are updating for some recent changes in source data. In the mean time, you can see the code here: http://code.google.com/p/open-gistemp/ - gavin] Comment by donQ — 27 November 2009 @ 11:11 PM Wow – and they used to call me a flat earther… Barney Rubble (my best friend) says that if we used techniques like the CRU during our quest for fire (taxing the roots and berries of all our friends for years to fund our preconceived notion that fire is caused by a complete lack of water) then we would have miserably failed.
Instead, we just stoned our detractors regularly.
Eventually, Barney threw a piece of flint at some poor guy, and started a fire. Voila. I credit our discovery not only to the regular stonings of detractors (a technique CRU did apparently use) but mostly to one thing we had back then that seems to be completely lacking at East Anglia – minds that are open to alternate theories.
See, even during the stone age our approaches and techniques were vastly superior to the CRU’s.
- Cheers,
- Fred Flintstone
Comment by F. Flintstone — 27 November 2009 @ 11:22 PM
Could you also add links for radiosonde data?
[Response: Good idea. I added RAOBCORE and HadAT - but I think there are a couple more sources. Links welcome. - gavin]
Comment by Chad — 27 November 2009 @ 11:45 PM
Slightly off topic again but u know me. Has anyone read the preliminary warning from University of Manitoba researcher David Barber. He has recently been around arctica including the areas that appear white on the satellite that was believed to be intact multi-year ice. According to his research it’s only solid on the surface beneath that it’s completely ‘rotten’, indicating to me that the warmer arctic ocean currents are contributing far more to the arctic ice loss that air temps.
There is virtually no more solid multi year ice left. When he was there he witnessed a massive multi year ice floe break up before his eyes. His prediction is that within 6 years there will be no summer sea ice in the arctic. This data should be rushed through to the IPCC for evaluation and presented in Copenhagen ASAP! Comment by Lawrence Coleman — 27 November 2009 @ 11:55 PM
Gavin,
thank you. The google link has code. ;-)
The total amount of it [code + support files] is 5 MB ... is this all that you have?
[Response: Me personally? If you want my code, go here. If you mean why isn't there more data for processing the temperature stations, I'm not sure why you want more. It's not that hard (and I'm sure a determined team to cut it down radically). - gavin] [PS: The other link to the GISTEMP sources (up to date) is now live again. It's all about who you know.... ;) - gavin] Comment by donQ — 27 November 2009 @ 11:57 PM
Long time lurker. Great site to find info to smite Deniers with. Not that they ever realize they’ve been smited, unfortunately.
Anyway, with the links provided, could a skeptic/denier perform an analysis that would disprove man made global warming? (If, in fact, man made global warming WAS just a liberal conspiracy?) If so, why haven’t they already done so? Is there any data missing that would be needed? If so, are there teams of denialists out there working their butts off to gather such data? Or are FOI requests the only way they know how to do research?
Comment by Groucho48 — 28 November 2009 @ 12:05 AM
Please don’t smudge our data — the pixels are still wet.
LOL, after 15 years of stonewalling....
Sorry, I wouldn’t trust any information or data from them. How do we know we’re getting the real thing? They’ve shown themselves willing to lie to further their agenda. Everything they have done or will do is now suspect.
Bet they didn't say that last week.
We can only wonder what they will say when some of the governments who gave them grants under false premises haul their little fannies to court and strip them down to the bare meat!
Anyway, with the links provided, could a skeptic/denier perform an analysis that would disprove man made global warming? (If, in fact, man made global warming WAS just a liberal conspiracy?) If so, why haven’t they already done so? Is there any data missing that would be needed? If so, are there teams of denialists out there working their butts off to gather such data? Or are FOI requests the only way they know how to do research?
Must have been posted by a FReeper.
What I get out of all this is that they are trying to give us access to the real data, knowing that by the time anyone could go through it all and do anything with it, the ship will have long sailed and we will be the slaves who must row.
Oh please.
Anyone who has ever known a university professor well knows that academe refuses to recognize and will shun, demean and destroy those who do not publish in the ‘right’ topics using the ‘appropriate’ methods. Everything else is National Enquiry stuff.
Regression analysis of data may be approved, but the numbers are completely malleable.
One of the items we discussed was whether or not changing the paint on the rooftops from calcium carbonate based "White Wash" to acrylic changed the heat flow from the roof to the interior thereby causing misreporting of the data.
Hmmm.
Some of the AGW fanatics tried to deal with that one but didn't ~ wonder how that will be dealt with at the fraud trials.
Wouldn't trust any of their data until that question is satisfactorily answered.
I would suspect the data that they are releasing is real. The most important thing is to release the methods by which they analyzed the data. You can make raw numbers do anything you want with the proper equations. I would also want to verify that there is "no missing data." You can also get a predetermined outcome by only analyzing the cherry picked data.
As Stalin said it is not important who the people vote for. What is important is WHO COUNTS THE VOTES Unfortunately the bad guys have been counting the votes for 15 years!
Really? Then either the researchers at East Anglia aren't part of the "climate science community" or there's another reason they haven't responded to the FOIA requests they've been given. It doesn't matter in the least what else we have access to if we don't have access to the data on which policy decisions involving hundreds of billions of dollars are based.
The author is no doubt well-meaning but his response is hardly likely to soothe anyone's fully-justified suspicions at this point. Claiming a swimming pool is 99.99% pure is no good if there's a turd floating on the surface - it's 100% corrupted and it isn't merely a matter of perspective.
Yeah, A LOUSY FREEPER TROLL!! LOL!
“Real Climate”
This web site is run by many of the same people THAT WROTE THE E-MAILS and “hid the decline”!!!!!
I agree. If they did have irrefutable proof that anthropogenic warming is the major factor in climate change, they would have thrown open their raw data files long ago. But they didn’t. They know they need to be the only ones interpreting the data due to the weakness of their claim.
Their actions speak louder than their words.
ex animo
davidfarrar
[[The climate science community fully understands how important it is that data sources are made as open and transparent as possible]]
Really? Because, Ahem...
“SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based. It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years....In a statement on its website, the CRU said: “We do not hold the original raw data but only the value-added (quality controlled and homogenised) data.”
http://www.freerepublic.com/focus/f-news/2396128/posts
[[I would suspect the data that they are releasing is real.]]
Donb’t bet on it- see my above post where CRU admits to manipulating the data, then throwing away the origial dats.
The curtain has been pulled back and the charlatan has been exposed. What we have is John Gotti trying to pretend he is a fine upstanding citizen.
I wouldn’t trust a damn thing that I see on RealClimate...
precisely- The two liars take a blood oath to ‘tell the truth’ (which of course means nothign, since they are both liars to begin with- but it leaves the ‘impression’ of honesty in the midns of those who are either guillible enough to fall for their crap oaths, or in on the same lies to begin with)
reminds me of the m ovie liar liar with Jim Carey- one only wishes the CRU and IPCC and the obama administration were FORCED to tell the truth like the actor in liar liar was-
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.