Climatic Research Unit

For many years monthly average temperature data from Weston Park weather station was collated by the Climatic Research Unit at the University of East Anglia, and processed to produce the CRUTEM gridded land surface temperature dataset. This dataset forms one significant part of the evidence for global climate change. Currently in its fourth version (CRUTEM4), the monthly updates from Weston Park are now processed by the Met Office Hadley Centre for Climate Prediction and Research. However once a year the CRU team updates the CRUTEM4 dataset with climate data that has been made newly available or updated by meteorological agencies around the world.

We spent 2 days visiting CRU and whilst there we spoke to Professor Phil Jones (Director of CRU), David Lister (Research Associate) and Ian Harris (Researcher). We also had chance to speak to a visiting climate historian Clive Wilkinson whose work we discuss in the Archives station.

Read the Data Journey, Culture and Policy tabs, then add a comment to the discussion below.

Once you have finished, follow the Red Line to the Met Office, or select one of the other stations on the map above.

Data Journey-0

Gridded Datasets

The Climatic Research Unit (CRU) has played a key role in creating and updating the CRUTEM gridded dataset of global land surface temperature anomalies. CRU created the first version of the dataset in the 1980s. The latest major update was released in 2012 and is called CRUTEM4. The methodology describing how it was created is explained in recent academic research publications co-authored by the CRU team.

The CRUTEM4 gridded dataset is a representation of the difference between monthly average temperatures and the 1961-1990 average temperature across the land surface of the Earth. The dataset covers the period from January 1850 to the present.

The gridded data are based on an archive of monthly mean temperatures provided by more than 5500 weather stations distributed around the world. Each station temperature is converted to an anomaly from the 1961-90 average temperature for that station, and each grid-box value is the mean of all the station anomalies within that grid box. As well as the mean anomaly, estimates are made of the uncertainties arising from thermometer accuracy, homogenisation, sampling grid boxes with a finite number of measurements available, large-scale biases such as urbanisation and estimation of regional averages with non-complete global measurement coverage. [Met Office Hadley Centre website]

Screen shot showing CRUTEM4 gridded dataset findings for the grid square covering England as published by CRU scientists on Google Earth (Osborn & Jones, 2014). The yellow markers show which stations' data are being used to calculate the average. The red lines show the years when the temperature was warmer than the 1961-1990 average, and the blue lines show the years it was cooler. Click on the image to open CRUTEM4 in Google Earth and explore further.
CRUTEM4 gridded dataset findings for the grid square covering England as published by CRU scientists on Google Earth (Osborn & Jones, 2014). The yellow markers in the grid square show which stations’ data are being used to calculate the average. The red lines show the years when the temperature was warmer than the 1961-1990 average, and the blue lines show the years it was cooler. Click on the image to open CRUTEM4 in Google Earth and explore further.

Collaboration

Whilst CRU did much of the early work on CRUTEM, the current version – CRUTEM4 – is a collaborative effort between the Met Office Hadley Centre and CRU. Each undertake part of the maintenance work required to keep it up-to-date:

The historical data is compiled by the team at UEA [CRU] who, having compiled a database of temperature records for weather stations around the globe, make that data available to use here at the Met Office.  We then use that information at the Met Office to produce the gridded temperature maps and time series, merge that data with the sea-surface temperature dataset [into the combined HadCRUT4 dataset] to produce global temperature estimates and derive the uncertainty estimates in these. [MO_04]

Today, CRU completes its update of the CRUTEM4 gridded dataset on an annual basis. CRU researchers gather and incorporate additional historical data series from stations around the world. CRU researcher David Lister explains a typical part of the update process:

My job at this time of the year tends to be to go to these different sources, like the Chinese source and the Canadian source, etc., and just grab the recent updates. Sometimes it’s just literally a few monthly values, sometimes they’ve revamped it because they run stuff through algorithms, through homogenisation software, etc. And so they might have suddenly had a big rework and so the series could be significantly changed. [CRU_01]

Once the annual update is completed, CRU passes its new version to scientists at the Met Office Hadley Centre who are responsible for undertaking the monthly updates. The updates for each annual update are carefully documented on the Met Office web site.

The Hadley team also use these updated CRUTEM4 data in the generation of HadCRUT4 – a combined land and sea surface temperature gridded dataset. Visit the Met Office station to find out more.

Quality control

During the update any new datasets are processed to match the requirements of the CRUTEM4 dataset. For some stations, the first stage involves converting daily data into monthly average (mean) temperature. For all stations there are then detailed checks to ensure data is accurate and that the correct values are imported. David Lister and his colleague Ian Harris explain some of the quality control processing that might be required:

[CRU_01] If it was a daily file, you could have three different readings for one day, simply because one reading might be what came off the instrument.  And then the next reading might be a QC’d [Quality Controlled] value because they’ve determined through their QC software that this value is wrong.  Or there may have been two or three different values come out of that instrument for the day, or whatever…

[CRU_02] Or one’s been homogenised and one hasn’t. 

[CRU_01] You can get these different values for the same day.  And there are flags, obviously, so you have to then take the flag to know which of those values to use. [CRU_01 & 02]

Data Sharing

In order to improve the transparency of the production of CRUTEM4 since the Unit was hacked in 2010 (see Policy tab), the dataset now only contains data that can be shared publicly:

All the series that go into the gridded product are publicly available on the Hadley Centre website, and that’s the rules of engagement now, publicly available.  So if anyone offered us a load of series that were for the scientific good but could not be released, we can’t use them because they have to be publicly available. [CRU_01]

Once CRUTEM4 has been updated it is uploaded into the British Atmospheric Data Centre repository so it can be used by other academic researchers. It is also made publicly available on the Met Office Hadley Centre web site, along with station data and additional time-series data generated by the Met Office:

So we make all of our data available as ASCII as well and just describe the format unambiguously so that people can access it.  But we still, I get people contacting me all the time for the datasets that I manage, because they’re having trouble opening them in Excel or something. [CRU_02]

In an attempt to improve accessibility of the dataset for the public, CRU have also made CRUTEM4 available via Google Earth.

Culture-1

Quiet Pride

The entrance to the world-leading Climatic Research Unit at the University of East Anglia is typically understated – a sign printed on A4 green paper taped to the door indicates your arrival.

Photograph of the entrance to Climatic Research Unit
Entrance to Climatic Research Unit, University of East Anglia
Photo of team photos in the common room, Climatic Research Unit, UEA.
Team photos in the common room, Climatic Research Unit, UEA.

 

 

 

 

 

 

 

 

 

 

 

There is a sense of quiet pride within the Unit – recent publications, research posters, certificates celebrating achievements and team photographs are modestly displayed on the walls.

The space feels distinctly non-corporate, and somewhat removed from the more recently refurbished School of Environmental Sciences building that it is adjoined to.

Community

A strong sense of community permeates the culture of CRU. Members find time for a daily crossword in the common space, take turns to cook a group lunch once a week, and head off to a local café for lunch every Friday.

Common room at Climatic Research Unit, UEA.
Common room at Climatic Research Unit, UEA.

In recent years the CRU community has faced threats from the University’s management to merge the Unit’s space with that of its larger parent Department and turn their common space into offices. Such a move would likely disrupt the strong community bonds observed within CRU and the team have so far been able to resist these threats to their space.

The sense of community within CRU stretches beyond the protective walls of the Unit, expanding into the networks of global scientific collaboration, and historically towards the founders of the Unit and early climate scientists.

The current work of CRU on the CRUTEM4 dataset is based upon principles of trust and collaboration within the international scientific community:

A lot of it is on we’re trusting their work. [CRU_01]

Everyone’s a link in a chain. [CRU_02]

The founders of CRU were perceived to provide “a strength and a foundation” [CRU_02] from which the Unit could grow. Some Unit members also felt a strong sense of connection to earlier observers of the climate such as British school teacher Guy Stewart Callendar whose climate records and diaries produced in the 1950s-60s were held for many years at CRU.

External threats

For the researchers of CRU, the starting point for their work is the data that has flowed into the Unit over the years from stations such as Sheffield Weston Park.

The output of their work on CRUTEM4 and earlier versions – evidence of warming of the global climate – threatens many established interests, some of whom refuse to accept the findings and attempt to generate skepticism about the results amongst the public and policy makers:

The Koch Foundation. It’s two brothers…They’ve been funding loads of right wing organisations all across America in various things and they have massive input into the Senate and the House of Representatives….They want a status quo, basically, they want to continue burning the fossil fuels and everything. [CRU_4]

Most notably this threat from climate change ‘skepticism’ resulted in the hacking of the Unit’s email system in November 2009, which is discussed in the policy tab.

Integrity

Integrity is a core value and is keenly defended by members of the Unit when it is brought into question by those outside:

A lot of people outside, especially the critics, think we’re all sort of evangelistic environmentalists who are frantically trying to change society and everything else to try and save the planet…But for me, I just want to get to the truth, and it’s reliable datasets, the best we can do at this point in time, that’s my real interest… we have to be unbiased as scientists to do what we do. [CRU_01]

[We’ve had] a little bit of money from the World Wildlife Fund for Nature, I’ve forgotten what it’s called now, over the years to one or two things, but when you do work for them–, well, we’ve also had money from BP and Shell over the years, you give them what you find. You don’t give them what they want, you give them what you find and these sceptics can’t seem to understand that. It’s the same with any government money or research council money, or EU, you’re not giving them what they want, you’re giving them what the data says. [CRU_4]

The public attack on the integrity of the Unit during 2009 was sorely felt:

People were trying to trash our reputation. [CRU_01]

Humour

The results of their data processing – evidence of warming of the global climate and threats from climate change sceptics – carries a burden.

Humour appears to play a role in attempting to lighten this load, as seen in a local newspaper cartoon that hangs on the common room wall which features three unconscious CRU researchers and a newspaper headline reporting praise for the Unit alongside the caption “No wonder the poor beggars fainted – this is the first cheerful news that’s ever come out of this department”.

Humour is also used to dilute the symbolic power of their aggressors:

Their surname is K-o-c-h, so we might call them Koch, but apparently in America they’re pronounced “Coke”. [CRU_4]

Data sharing

Data sharing is perceived to be an important practice within the CRU community. The gridded datasets produced by CRU have always been shared freely with other climate scientists.

In the past, there were restrictions on the sharing of the weather station data underlying the CRUTEM database. These restrictions often resulted from the terms and conditions agreed with meteorological services that provided CRU with access to their station data many years ago. These restrictions were not necessarily taken lightly by CRU members however:

As someone who works in climate, you often despise these attitudes, but you’ve got to be able to see why people might not want them to be publicly available. [CRU_01]

As discussed in the policy section, CRU’s data sharing practices faced scrutiny in 2009-10. The Information Commissioner’s Office ruled that a breach of the Environmental Information Regulations (2004) had occurred in response to a request to the University for access to some of the weather station data underlying the CRUTEM dataset. A House of Commons Committee also reported issues around the transparent publishing of data and methods by CRU – although recognised that practice was “in line with common practice in the climate science community” [Commons Select Committee].

Since the events of 2009-10, CRU has taken an active role to push for the publication of the data sources used to produce the CRUTEM4 dataset and promote more transparency around how the dataset is produced.  CRUTEM4 now only uses station data that can be made publicly available.

Policy-2

As part of a University the Climatic Research Unit is subject to a number of laws regarding the processing of data and information. Two of these laws are the Freedom of Information Act (2000) and the Environmental Information Regulations (2004).

Under these regulations people have the right to request access to information held by public bodies such as Universities, and public bodies are under obligation to publish certain types of information about their activities. However, not all information has to be made available, there are various exemptions.

In 2009, the University of East Anglia received a significant number of requests under these Acts (61 in July alone) regarding the work of CRU, including a request for copies of some of the weather station data underlying CRUTEM that had been sent to Georgia Tech University in the USA. These requests were refused by the University under various exemptions detailed below.

In November 2009, weeks before the UN Copenhagen Summit on Climate Change, the Climatic Research Unit was hacked and over 1000 emails to and from CRU staff were leaked and published on Wikileaks alongside various documents, code and models.

Using this leaked information climate change skeptics worked to bring the reputation of CRU and climate science into question in the media and strengthen the position of climate change skeptics at the 2009 United Nations Climate Change Conference.

The Unit was accused of misrepresenting the data they were processing in order to fit preconceived opinions about climate change resulting from human activity. However, after a thorough inquiry this accusation was found to be “patently” false by the House of Commons Science and Technology Committee. After thorough investigation the Committee found that CRU had “no case to answer”, however there was a potential breach of the Freedom of Information Act whose responsibility lay with the University rather than CRU.

The request for access to the weather station data used by CRU that was received by the University of East Anglia in July 2009 fell under the Environmental Information Regulations 2004.

The Environmental Information Regulations 2004 are transposed from the EU Directive 2003/4/EC on public access to environmental information, which in turn was the EU implementation of the Aarhus Convention (1998) on access to information, public participation in decision-making and access to justice in environmental matters.

The University refused to provide the weather station datasets using the following exemptions:

  • Regulation 6 – most of the information already publicly available and accessible via the Global Historical Climatology Network website.
  • Regulation 12 (5)(a) adverse affect on international relations due to the potential for damaging relations between UK climate research and foreign meteorological organisations  who had  provided data in confidence.
  • Regulation 12(5)(c) adverse affect on intellectual property rights held by the University of East Anglia, national meteorological services and weather stations.
  • Regulation 12(5)(f) adverse affect on the interests of the information provider (national meteorological services) many of whom had supplied the data to CRU under particular terms and conditions and had not given permission for disclosure.

The Information Commissioner’s Office considered the University’s use of these exemptions and in June 2011 it ruled that the decision was in breach of the Regulations. The University was instructed to provide the applicant with the datasets.

Whilst the Information Commissioner’s Office recognised some of the arguments made by the University, it was ruled that it could not be evidenced that any of the issues surrounding the release of the datasets would lead to a “more probable than not…adverse affect” [ICO] on international relations, intellectual property rights or the interests of national meteorological services that provided the data to CRU. Further, the data that was available via the Global Historical Climatology Network was perceived to be too difficult to access without specific technical skills.

Since this time CRU have played an active role in ensuring that the station data used in producing CRUTEM4 is publicly available. Today, only station data that can be published online is used in the production of CRUTEM4.

Archival Sources


We’d like you to take a minute to reflect on some of the things you have read above and add a comment to the discussion below.

Here are some questions to get you started:

  • How do the cultural values of this station compare to some of the other stations you have explored?
  • How do you think the cultural values of the Climatic Research Unit contribute to the quality of the CRUTEM4 gridded dataset?
  • How do you feel about the pressures that CRU have faced in recent years to release the weather station data underlying the CRUTEM dataset?

Once you have finished, follow the Red Line to the Met Office, or select one of the other stations on the map above.

By posting a comment you confirm that you agree to our conditions of participation.

One thought on “Climatic Research Unit”

  1. As an advocate of open data, until I read this I was firmly on the side of making everything open. However, having read about the various issues surrounding the CRU data, it’s admirable that they are able to reconcile these various concerns and release as much as they have.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>