Environment/Nature

It's Climategate 2.0! (…not)

In December, the Goddard Institute for Space Sciences (GISS) published over 200 pages of internal emails as required by a Freedom of Information Act (FOIA) request by the Competitive Enterprise Institute (CEI). The emails involved how the GISS handled responding to a number of requests for information, data, and code from Steve McIntyre, founder of the climate disruption-denier website ClimateAudit.org. Clearly there was no metaphorical “smoking gun” in the emails, because the CEI didn’t crow about a likely Climategate 2.0 following the emails’ release.

However, today it appeared that Judicial Watch and number of large climate denier blogs didn’t get the memo. Judicial watch issued a press release that claimed

“This email traffic ought to be embarrassing for NASA. Given the recent Climategate scandal, NASA has an obligation to be completely transparent with its handling of temperature data. Instead of insulting those who point out their mistakes, NASA scientists should engage the public in an open, professional and honest manner,” stated Judicial Watch President Tom Fitton.

Apparently, neither Anthony Watts of Wattsupwiththat.com nor the aforementioned Steve McIntyre were aware that the emails had been released, since both deniers put up fresh posts repeating the Judicial Watch press release in the last couple of days. Furthermore, Telegraph blogger (and one of the more vocal Climategate pundits) James Delingpole also got caught by the press release, even going so far as to ask

Has Climategate moved to the US? Looks like it from this story at Watts Up With That.

After reading the emails myself, it’s clear to me that Delingpole must have come unmoored if he seriously thinks that these emails show anything even remotely like another climate non-scandal. And while neither Watts nor McIntyre have made the same claim that Delingpole made, both men did link to the Judicial Watch press release and both have been content to permit the comment threads to claim “scandal!” on their behalf.

Put simply, the emails show the GISS scientists acting professionally and in and open and transparent manner with reporters and McIntyre himself. To illustrate, let’s read some of the emails ourselves.

Gavin Schmidt wrote, in reference to responding to McIntyre:

I would suggest being more specific about what was assumed and what you will do now. The stats you had for the number of stations which had positive and negative offsets would be appropriate. You might also want to thank him for bringing this to our attention. The first because he’ll ask you anyway or work it out himself, the second since it doesn’t hurt to be gracious.

Reto Ruedy, in his email response to McIntyre:

..and I’d like to thank you for bringing this to our attention.

Ruedy also contacted the National Geographic Society to make sure that they updated their maps:

I checked what this correction does to your map and it does change the colors somewhat over parts of the US; the rest of the world is unaffected. Even the change over the US is way within the maring of error (0.5 C). So there is little need to make any changes.

Ruedy wrote several emails to Leticia Sorg of Redacao Epoca which show him patiently explaining how McIntyre’s overblown claims about the US have no impact on global climate (a point I made myself back in 2007). Sorg asked “Considering [the 1934/1998 ranking change], would it be possible to say that the planet is becoming hotter and hotter?” to which Ruedy replied

To answer your question, given the existing sampling error (.1-.2C): No – we cannot draw any conclusion about our planet from the US data (much less from the rankings you show below)

In other words, the emails show the scientists discussing their work, politely explaining things to journalists and to McIntyre, and generally being normal people.

McIntyre, however, doesn’t exactly come off nearly as well as the GISS scientists in these emails. McIntyre asked on August 4, 2007,

In addition, could you provide me with any documentaiton (additional to already published material) providing information on the calculation of GISS raw and adjusted series from USHCN versions, including relevant source code.

Ruedy points out that, as of McIntyre’s request, all the source code is already publicly available:

The software we spend close to 100% of our time in developing and which is the real basis or [sic] our work (in addition to general physics and chemistry), is openly available (giss.nasa.gov/tools/modelE) to anybody.

And yet McIntyre, in a later email on August 8, 2007 (four days after his initial request and a day after Ruedy pointed out that the source code was publicly available), asks again for source code:

I would like to assess the impact of these modifications on the US and global averages for myself. I would appreciate a copy of the source code used for these calculations.

Reto Ruedy, responding to a request from a NASA press officer about McIntyre:

The blog [Climate Audit] you attached is a prime example of what gives bloggers a really bad name….

[McIntyre] omits that the global mean time series (which is generally considered the standard measure for global warming) is unaffected [by the small errors in the US record that McIntyre discovered].

He concentrates on US time series which (US covering less than 2% of the world) is so noisy and has such a large margin of error that no conclusions can be drawn from it at this point; showing the plot of annual means before and after the correction would have made the whole article a joke since the differences are barely visible.

He had to use the device of ranking the years rather than showing the plots to make any point at all. The problem with rankings is that there are large clumps of years which are equal within the margin of error and rankings within these clumps are purely accidental.

He finds it astounding that years 1934 and 1998 reversed ranks, not remembering that the corrections only affected years 2000-2006, hence that there is no possible connection there.

And McIntyre had been using a ‘bot to download every bit of data from GISS and complained when his IP was blocked (a point that McIntyre made today in the comments at climateaudit.org). Schmidt wrote on August 16, 2007

Reto and Rob Schmunk have the details. [McIntyre] was using a robot to automatically download pages that robots weren’t allowed to (because of the server demands of interactive scripts) and Rob blocked the IP. After a couple of emails back and forth, he was allowed to continue on weekends/evenings.

The webmaster’s account of McIntyre’s actions goes like this:

On about May 16, around 10:30 or 11:00 p.m., as I was getting ready to leave GISS for the night, I belatedly checked the error logs on the two web servers and discovered that there were several thousand errors in the log on Web2. On a normal day there would be about 500…..

Further investigation revealed that someone had been firing off requests to Web2 since about 2:00 that afternoon for the station data and by the time I looked into the situation, there had been at least 16,000 requests. Perhaps half of these had gone to addresses in the CGI directory, which means that activating CGI scripts to extract data, etc….

Plainly this activity was from an “automated” agent, which in rough parlance is usually called a “robot”….

As the robot on may 16 came from a generic ISP address rather than, say, an academic address and further because it’s “user-agent” tag provided no further information about who was running it, and _also_ because the GISS websites have “robots.txt” files which instruct all well-behaved web robots to stay out of the CGI directories, I cut off access to the ISP in question to the websites on Web2.

The next day I received e-mail from McIntyre asking what was up. He did not identify himself or on whose behalf he was acting….

All I know is that my first contact with him came because he was blasting umpteen thousand requests at the webservers.

I have no idea how much traffic McIntyre’s website gets, and I don’t know that I havve ever even looked at it. His tone in his e-mail was on the arrogant side, so I had no desire to prolong communication with him any longer than was necessary.

In other words, McIntyre created a ‘bot to scrape all the GISS data up and download it and did so in a way that a) interfered with the operation of GISS’ server, b) ignored instructions that the GISS webmaster had put in place to prevent overloading the servers, and c) he got annoyed when he was shut down. If that had been me (and I’ve managed websites from time to time), I’d have suspected an attack and done exactly what the webmaster did.

Columnist Mark Steyn doesn’t come off very well either, given the actual email responses McIntyre got. A Steyn column is quoted at length in a GISS email dated August 13, 2007, from Stephen Volz. The quote says in part:

So why is 1998 no longer America’s record-breaker? Because a very diligent fellow called Steve McIntyre of climateaudit.com labored long and hard to prove that there was a bug in NASA’s handling of the raw data. He then notified the scientists responsible, and received an acknowledgment that the mistake was an “oversight” that would be corrected in the next “data refresh.” The reply was almost as cool as the revised chart listings. (emphasis mine)

All in all, the emails show McIntyre being a jerk, the GISS scientists responding politely even to repeated, unreasonable requests from McIntyre, and the GISS scientists managing quite well under the stress of a media circus based on statistically insignificant differences that had no bearing on the actual error that McIntyre corrected.

So this isn’t Climategate 2.0 (Climategate 1.0 wasn’t Climategate either, for that matter), even though Judicial Watch and James Delingpole both seem to think so. And while Anthony Watts and Steve McIntyre aren’t personally claiming that this a new Climategate, they’ve done nothing as of the writing of this post to correct the record at their websites, making any misperceptions of a scandal by readers Watts’ and McIntyre’s responsibility.

Furthermore, the content of the actual emails shows that the GISS scientists did engage McIntyre and reporters in an open and professional manner. GISS also openly and transparently permitted McIntyre to continue scraping all the data off the site after getting him to agree to do it on off-hours so as not to overload the GISS servers. Amazingly enough, GISS acted in 2007 just as Judicial Watch’s president, Tom Fitton, demanded last week.

Thanks to Eli Rabett for pointing this out, albeit indirectly.

9 replies »

  1. You are kidding, right? Eli Rabett? The emails show that the conspirators thought of him as a jerk for daring to question anything. All he did was place a request for information, but since he’s already shown up the dodgyness of the Hocky Stick, they didn’t want him anywhere near the data or the crummy codes they were using. One just has to read the emails to see that.

    Then, this new scandal.
    BREAKING NEWS: Manipulated Datasets at NASA and NCDC
    Once again, the audacity and brazen manipulation of scientific evidence is shocking. These are very serious allegations.

    “Thermometers moved from cold mountains to warm beaches; from Siberian Arctic to more southerly locations, and from pristine rural locations to jet airport tarmacs.”
    http://joannenova.com.au/2010/01/climategate-arrives-in-the-us/

  2. Sorry, that’s McIntyre I meant by “he”, not Eli Rabett. as having just asked for information he was entitled to and being considered a jerk for it. Just likeTony Soprano they have to denigrate their opponents.

  3. What part of “The software… is openly available (giss.nasa.gov/tools/modelE) to anybody.” did you not understand?

    McIntyre wasn’t a jerk for asking for data, he was a jerk for scraping the site during prime time. He was a jerk for repeatedly asking for code even after he had been pointed to it. He was a jerk for not explaining what he was doing and why to the GISS webmaster.

    McIntyre’s a jerk for trying to leverage a small error he discovered (and that he was publicly given credit for) in the data after 2000 to claim that AGW is all bogus because 1934 was 0.01 degrees C hotter than 1998. He is a manipulator of data and people for not making it clear that the ranking is meaningless, given that the error in the measurements is 10-20x the difference in the two year’s temp anomalies.

  4. McIntyre is a jerk because Hansen said he was a pest at the time. BTW, that link above to “200 pages of internal emails” goes nowhere.

    And the new GISS scandal is much more important that the one in 2007.

    • Thanks for pointing out the link, abt – I fixed it, so it goes to the files at GISS.

      McIntyre was being a pest at the time, abt. How else would you characterize scraping the site during the time of the day when most websites get the most traffic? How else would you characterize repeatedly asking for software that was publicly and freely available even after he was pointed right at it?

      And I notice that you didn’t address my point that McIntyre was manipulating the public and the press with his 1934/1998 ranking thing. McIntyre manipulates data as much as needed to make his point just as often as D’Aleo, Monckton, and Singer do, and all four have been found – repeatedly – to be essentially lying to people with their data and statistics.