Anyway here’s our Op Ed entitled "Statisticians Blast Hockey Stick".
The recently released final report of a panel of three independent statisticians, chaired by an eminent statistics professor, Edward Wegman, Chairman of the U.S. National Academy of Sciences’ Committee on Theoretical and Applied Statistics, has resoundingly upheld criticisms of the famous “hockey stick” graph of Michael Mann and associates.
The Wegman report, which was submitted to the U.S. House Energy and Commerce Committee in July, stated that our published criticisms of Mann’s methodology were “valid and compelling” and stated that ”Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.”
This comes on the heels of an earlier report in June by a National Academy of Sciences panel chaired by Gerald North of Texas A&M, which also endorsed specific criticisms of Mann’s methodology and which concluded that no statistical confidence could be placed in his claims that temperatures in the 1990s exceeded those in the medieval warm period.
Wegman also criticized lack of independence in paleoclimate science at multiple levels – in the selection of proxies, in the reviewing of articles and in the Intergovernmental Panel on Climate Change (IPCC) process itself. In his testimony to the House Energy and Commerce Committee, he sarcastically questioned Mann’s citation of his own articles or articles by his students as supposedly “independent” verification of his results.
Given the importance that the IPCC and others have placed on historical temperature reconstructions, Wegman recommended that qualified statisticians be involved in the analysis and that the work be reviewed by truly independent experts.
In response to the Wegman report, Michael Mann issued a statement saying that it “simply uncritically parrots claims by two Canadians”. However, in testimony to the House Energy and Commerce Committee, the President of the National Academy of Sciences, Ralph Cicerone stated his belief that Dr. Wegman was well qualified to make the statements in his report.
In what follows we simply quote, verbatim, from the report and Wegman’s Congressional testimony. The report and hearings are available at http://energycommerce.house.gov/108/Hearings/07272006hearing2001/Wegman.pdf,
http://energycommerce.house.gov/108/Hearings/07192006hearing1987/hearing.htm and
http://energycommerce.house.gov/108/Hearings/07272006hearing2001/hearing.htm
WEGMAN EXCERPTS
The debate over Dr. Mann’s principal components methodology has been going on for nearly three years. When we got involved, there was no evidence that a single issue was resolved or even nearing resolution. Dr. Mann’s RealClimate.org website said that all of the Mr. McIntyre and Dr. McKitrick claims had been ‘discredited’. UCAR had issued a news release saying that all their claims were ‘unfounded’. Mr. McIntyre replied on the ClimateAudit.org website. The climate science community seemed unable to either refute McIntyre’s claims or accept them. The situation was ripe for a third-party review of the types that we and Dr. North’s NRC panel have done.
While the work of Michael Mann and colleagues presents what appears to be compelling evidence of global temperature change, the criticisms of McIntyre and McKitrick, as well as those of other authors mentioned are indeed valid.
“Where we have commonality, I believe our report and the [NAS] panel essentially agree.…We believe that our discussion together with the discussion from the NRC report should take the ‘centering’ issue off the table. [Mann’s] decentred methodology is simply incorrect mathematics… I am baffled by the claim that the incorrect method doesn’t matter because the answer is correct anyway. Method Wrong + Answer Correct = Bad Science.
The papers of Mann et al. in themselves are written in a confusing manner, making it difficult for the reader to discern the actual methodology and what uncertainty is actually associated with these reconstructions.
It is not clear that Dr. Mann and his associates even realized that their methodology was faulty at the time of writing the [Mann] paper.
We found MBH98 and MBH99 to be somewhat obscure and incomplete and the criticisms of MM03/05a/05b to be valid and compelling.
Overall, our committee believes that Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.
[The] fact that their paper fit some policy agendas has greatly enhanced their paper’s visibility. … The ‘hockey stick’ reconstruction of temperature graphic dramatically illustrated the global warming issue and was adopted by the IPCC and many governments as the poster graphic. The graphics’ prominence together with the fact that it is based on incorrect use of [principal components analysis] puts Dr. Mann and his co-authors in a difficult face-saving position.
We have been to Michael Mann’s University of Virginia website and downloaded the materials there. Unfortunately, we did not find adequate material to reproduce the MBH98 materials. We have been able to reproduce the results of McIntyre and McKitrick
Generally speaking, the paleoclimatology community has not recognized the validity of the [McIntyre and McKitrick] papers and has tended dismiss their results as being developed by biased amateurs. The paleoclimatology community seems to be tightly coupled as indicated by our social network analysis, has rallied around the [Mann] position, and has issued an extensive series of alternative assessments most of which appear to support the conclusions of MBH98/99.
Our findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus ‘independent studies’ may not be as independent as they might appear on the surface.
It is important to note the isolation of the paleoclimate community; even though they rely heavily on statistical methods they do not seem to be interacting with the statistical community. Additionally, we judge that the sharing of research materials, data and results was haphazardly and grudgingly done. In this case we judge that there was too much reliance on peer review, which was not necessarily independent.
Based on the literature we have reviewed, there is no overarching consensus on [Mann’s work]. As analyzed in our social network, there is a tightly knit group of individuals who passionately believe in their thesis. However, our perception is that this group has a self-reinforcing feedback mechanism and, moreover, the work has been sufficiently politicized that they can hardly reassess their public positions without losing credibility.
It is clear that many of the proxies are re-used in most of the papers. It is not surprising that the papers would obtain similar results and so cannot really claim to be independent verifications.”
“We note that the American Meteorological Society has a Committee on Probability and Statistics. I believe it is amazing for a committee whose focus is on statistics and probability that of the nine members only two are also members of the American Statistical Association, the premier statistical association in the United States, and one of those is a recent Ph. D. with an assistant professor appointment in a medical school. The American Meteorological Association recently held the 18th Conference on Probability and Statistics in the Atmospheric Sciences.. Of the 62 presenters at a conference with a focus on statistics and probability, only 8 … are members of the American Statistical Association. I believe that these two communities should be more engaged and if nothing else our report should highlight to both communities a need for additional cross-disciplinary ties."
Especially when massive amounts of public monies and human lives are at stake, academic work should have a more intense level of scrutiny and review. It is especially the case that authors of policy-related documents like the IPCC report, Climate Change 2001: The Scientific Basis, should not be the same people as those that constructed the academic papers.”
