Blog discussions, conference presentations and peer review

A blog post earlier this week about an EGU presentation of Eva Steirou (a researcher in the group of Demetris Koutsoyiannis) on temperature data homogenisation created some stir in the blogosphere after Watts Up With That? and Climate Audit paid attention to it. Koutsoyiannis has now written a guest blog to give some first reactions.

Guest post by Demetris Koutsoyiannis

I believe that science blogs have offered a very powerful means in scientific dialogue, which is a prerequisite of scientific progress. I have very positive personal experiences. In 2008, a poster paper in EGU, “Assessment of the reliability of climate predictions based on comparisons with historical time series”,  was widely discussed at blogs and this was very useful to improve it and produce a peer-reviewed paper, “On the credibility of climate predictions” , which again was widely discussed at blogs. In the follow up paper, “A comparison of local and aggregated climate model outputs with observed data” we incorporated replies to the critiques we have seen in lots of blog comments.

In comparison, the formal peer reviewed system, while in principle encourages post-publication discussion through formal Commentaries and Replies, was able to offer us a single Commentary for the second paper (none for the former), which also gave us the opportunity to clarify our methodology (and feel safer about it) in our reply, “Scientific dialogue on climate: is it giving black eyes or opening closed eyes? Reply to “A black eye for the Hydrological Sciences Journal” by D. Huard”.

In another example, my 2012 paper “Clausius-Clapeyron equation and saturation vapour pressure: simple theory reconciled with practice” received one formal Commentary and was marginally (and off-topic) discussed at a blog. This discussion annoyed me for its eristic tone, but also prompted me to revisit my calculations and spot an error, for which I published a Corrigendum (acknowledging the blog discussion).

Both these examples show that blog discussions can be complementary and synergistic to the more traditional scientific transactions, conference presentations and journal papers, and also extremely useful. Although we do not call such discussions peer reviews, they can provide very useful review comments by peers. Of course, blog discussions are not replacements of formal peer reviews and have their own weaknesses (as all human transactions). A first one is that they are too selective. You must be very lucky and your paper very topical to trigger discussions at blogs.

Eva Steirou’s presentation in the last EGU conference, coauthored by me, entitled “Investigation of methods for hydroclimatic data homogenization” was one of the lucky ones as it became the subject of several blog posts and extensive discussions in the last couple of days. Its fortune was clearly made by Marcel Crok, through his post of 16 July. I must note that the presentation was online since April and no one paid particular attention—and I do not remember to have alerted anyone about it. Throughout June it received 10 visits while in the last few days it received 2500 visits, about half of which come from WUWT. I am grateful to all who discussed it and particularly to Marcel who brought it into light, and I regret that subsequent bloggers did not give him the due credit.

In his second post, Marcel gave the full story of the presentation in an accurate manner. His first post was equally accurate. He knew the details because he attended the EGU session where it was presented.  Evidently, it was not a peer-reviewed paper but a conference presentation (although it is not infrequent to call “paper” a conference presentation or a poster, we should not use the term “peer-reviewed” for conferences that select papers reviewing only the abstracts).

But we plan to produce a peer-reviewed paper (unless we have made a fatal error, which we hope not) and we keep studying the topic more thoroughly. That is why we think that we are lucky to have received all these comments from the blogs. I did not have the time to read them all, let alone to assimilate them, so I will not provide replies here. From first glance I find most of them very useful, whether they are positive or negative.

In his second post, Marcel mentions two of the commenters, Victor Venema and Steven Mosher, and summarizes some points of their critiques. About Steven’s complaint that there is no station list, I think there is no need to complain because the list is contained in the last pages of Eva’s diploma thesis. Although the thesis is in Greek, the list and the accompanying graphs are in English. It’s a pity not to have clarified that, as we, all members of the team Itia, are fans of open information.

About Victor’s critique I had the impression that we had resolved several issues because he emailed me and we had a long exchange (which BTW I authorized him to publicize if he wished). Therefore, I was somewhat surprised to see a rather harsh post in his blog. I will not reply to the essence of his points. We will study them in detail—and I have already told this to him. But I note, when he says that our (prior) estimates of the expected proportions of data corrected upward and corrected downward should be 1/2 is “plainly wrong”, he would be more convincing if he gave his own estimate, rather than telling there are biases. Also, I am surprised to see that he criticizes our statement that homogenization practices often lead to false results. Is he so sure that they always give correct results?

But of course these are scientific disagreements and it is fine if scientists disagree. Some arguments, though, fall into other categories, such as arguments from authority or ad hominem. Well, I am familiar with such arguments within scientific transactions, formal (paper reviews) or informal (in blogs), but they are always saddening and also make it necessary to refer to personal information in order to reply.

Victor has written in several blogs, as well as in his own original blog post, that Eva is a student and that I am new to homogenization, and emphasized that I was the convener of the session, implying that the presentation was kind of approved by myself. I asked him to correct his downgrading of Eva who is a graduate, professional engineer, and write the names of the co-conveners of the session, because I was not alone. He did not exactly do what I asked, but he was kind to change the phrasing in his post.

The following phrase from Victor’s blog post, also quoted by Marcel, is really interesting:

Had I been EGU convener of the homogenization session, I would probably have accepted the abstract, but given it a poster because the errors signal inexperience with the topic and I would have talked to them at the poster.

I do not wish to dispute that Victor is an expert on the topic. I really think he would be an excellent convener of the homogenization session. I would be happy if he chose my abstract to be presented as a poster. Most of my papers, including the one in the top of this post, have been presented as posters. I would fully respect his right to “probably accept” and of course “probably reject” my abstract. Even, I recognize his right to say or imply (in the above phrase as well as in his first email inviting me to “discuss these matters with the experts”) that I am not an expert.  Actually, I take this as a compliment as I prefer to view myself more like a student than an expert. I have no more respect to experts than I have to students and often collaboration is more constructive and fascinating with those who feel students than those who declare themselves to be experts. I do not care (rather I enjoy) being “new to homogenization” or to everything else—except “new to thinking” (to “skeptesthai”, if you allow me to use the Greek word for “thinking”).

But it may not be fair for him to charge me with inexperience. I think I have some experience with hydrometeorological data management, from acquisition to processing:

  1. As an engineering hydrologist I have participated in several professional studies of the design or management of several projects and the production of engineering reports. In hydrological engineering reports, data homogenization and further processing to extract design quantities are routine procedures.
  2. Twenty years ago, I have been the inspirer and project director of a Greek national-level project “Hydroscope: Creation of a National Databank for Hydrological and Meteorological Information”. Among other things, this produced several reports (in Greek; accessible via the above link) and software including methodologies for data homogenization.
  3. Almost 20 years ago, Itia installed and operated, for the first time in Greece, a new-generation meteorological station, which gave free data on the Internet in real time, even before the shaping up of the World Wide Web (in the beginning using the Unix “finger” utility). We continue to provide this information, both in real time and historical.
  4. Itia has produced several software products, again open and free, among which some are for hydrometeorological data processing. Our latest one, Hydrognomon, has been downloaded and used by colleagues internationally, despite its manual being only in Greek (we are now translating it to English).
  5. Itia (with my colleague Nikos Mamassis as project director) has produced and maintains the software used by most (if not all) hydrological and meteorological organizations of Greece to store and process their data.
  6. Recently, Itia (in particular, my colleagues Antonis Christofides and Stefanos Kozanis) has created the openmeteo.org, a public database with free hydro-meteorological data which offers anyone the possibility to upload and store his own data.

Thus, I think I have some experience with data and I have also some experience with stochastics. Given these two, I often have doubts whether what we are doing is correct or not. That is why I directed Eva to make this research, supervised by myself. I understand doubts may be unwelcome for some who prefer dogma over science, but for me it is a way of living—and I have no problem to tell my doubts publicly. Even if they will eventually prove to be groundless, they may be constructive in strengthening their subject.

Clearly, our research is not finished, and what we presented in EGU is our first communication in public. Despite being criticized by Victor for the opposite, I believe we chose an appropriate session to submit our work—and with this I do not of course mean that “his” homogenization session would be inappropriate. Victor may wish to see that Böhm himself, whom he heavily cites in his post, was invited and presented his work, related to a review of two decades of experience in the field of homogenizing instrumental climate data, in our same session two years ago (ignore the slightly changed session title this year). I hope the session will be organized again in next years and I will propose to the colleagues in the EGU HS Subdivision Precipitation and Climate who will be convening this session to also invite Victor to present his work, if he wishes of course. Our subdivision is very active, democratic, open and open-minded.

Share

2 comments to Blog discussions, conference presentations and peer review

  • [...] WUWT blog roll. That said, the post did generate quite a bit of discussion, always a good thing. Marcel writes of a guest post by Koutsoyiannis:  A blog post earlier this week about an EGU presentation of Eva [...]

  • Demetris: “But I note, when he says that our (prior) estimates of the expected proportions of data corrected upward and corrected downward should be 1/2 is “plainly wrong”, he would be more convincing if he gave his own estimate, rather than telling there are biases.”

    To demonstrate that the expectation that there are just as much jumps upward as downward is wrong, it is sufficient to mention one reason why past temperature measurements were biased, I mentioned a range of reasons why the were too warm.

    You cannot give a number for an expected proportion, it will depend on the period and region considered (which technological changes need to be taken into account), it will depend on the (detection power of the) homogenization algorithm and it will depend on the policies of national weather service with respect to merging stations in case of relocations (with respect to difference in distance and height of the two locations). During the last homogenization workshop in Budapest, some participants had a table with causes of inhomogeneities and how often they occur. For every country these fractions were quite different, probably because of the above mentioned causes.

    Demetris: “Also, I am surprised to see that he criticizes our statement that homogenization practices often lead to false results. Is he so sure that they always give correct results?”

    Any number has an uncertainty, is “false”. What was wrong was the assumption that errors in homogenization always lead to artificially stronger trends and that thus the true trend will lie in between the trend of the raw data and the homogenized data.

    Errors in homogenization can just as well lead to underestimates of the true trend. This would maybe even by more probable, because as I stated above we expect the raw data to be biased, e.g. due to insufficient protection against the influence of the sun. If you do not find these breaks, because the homogenization algorithm does not find them, you will underestimate the trend in the true climate.

    At least you wrote in the abstract that the real trend would be between the one for the raw data (0.4°C per century) and the one of the homogenized data (0.7°C per century). The headline of Watt’s up with that made of this: “New paper blames about half of global warming on weather station data homogenization”, which is the lowest value of your range. Another example of typically WUWT misinformation. I would say that the true trend could be both higher and lower than 0.7°C and I find higher a bit more likely.

    By the way, with experts I did not mean me, but all the others at the homogenization session. At a conference it is much easier to communicate as via blogs and you learn much more, at least if you are in the right session. I just started working on this topic some time ago, have not developed a homogenization method, nor homogenized a dataset. It was because of this, that the European homogenization community has asked me to lead their blind validation study.

    The results of this study did not show any bias in the trends after homogenization. There is, of course, a remaining error, but no bias. Another recent validation study by Williams et al. (2012) found, if I remember well, that if inhomogeneities cause a bias in the raw data, homogenization will not remove the full bias. In other words, this study also suggests that the true trend in your selected dataset would be more, not less, than 0.7°C per century.

    I personally do not yet trust studies on trends in severe or even extreme weather. These studies often use daily data and the homogenization of the distribution of daily data is still in its infancy. Also the homogenization of precipitation is still quite difficult. Everyone is welcome to scrutinize the homogenization of the mean temperature, but I doubt :-) it will be productive.

Geef een reactie

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Agenda

Loading...

Donate to support investigative journalism on global warming

My blog list