Vast amounts of weather data exist in archives around the world. Climate scientists recognise the value of this data for filling the gaps in the observation data they currently have available, which can help them to understand better changes in the climate over time.
Around the world people are working together through a project called ACRE to try and recover some of this data before it decays or is destroyed. You can read more about the ACRE project in the Met Office R&D tab.
To find out more, we spoke to Clive Wilkinson [CRU_03], a climate historian who has a vast knowledge of shipping records and archives and who works with climate scientists such as [MO_3] and [MO_4] to get them the data they need.
Read the ‘Data Journey’ and ‘Culture’ tabs below, then add a comment to the discussion at the bottom of the page.
Once you have finished follow the Blue line on the right to the Old Weather project. Here you can find out how citizen scientists are helping to transcribe the recovered data and make it usable for climate scientists.
Ship log books
More than two thirds of the earth’s surface is covered by oceans. The collection of marine weather observations has been less systematic than over land, particularly before the advent of satellite technology, due to the often hostile and inaccessible nature of these oceans. There are therefore significant gaps in historic weather observations over the sea.
However, crews of naval and merchant ships have long made weather observations at sea using a variety of methods:
You have observations made by ships, either in very old data from say throwing a bucket over the side of the ship, hauling it up, putting a thermometer into it, and then writing down what the temperature was, or otherwise coming in through water intake into an engine room. [MO_4]
This data was recorded in ships’ log books and many of these are now stored in archives around the world. This data offers climate scientists a possible window through which to develop their understanding of marine weather conditions over the last century and more.
Before climate scientists can analyse this data it needs to be recovered, and in some cases rescued, from the archives.
Filling the gaps
Data recovery and data rescue activities begin with an information need from climate scientists for historical weather data from specific regions and time periods where climate data is sparse.
I usually say to people if you’ve got a weather observation, if it’s south of the equator, we’ll take it. Right. If it’s north of the equator before about 1950 — we’ll take it. If it’s north of the equator, outside 1950, and it’s not in North America, Europe — Japan, or the North Atlantic — we’re probably still interested. [MO_03]
They discuss their needs with historians and archivists such as Clive Wilkinson, who use their skills to locate suitable ship log books in the archives. This can often be a challenging task. In many countries, the archives are simply storehouses with boxes of old records whose contents are unknown. Often these records are kept in poor, often humid, conditions which mean they can decay or be destroyed before they are found.
Watch the video below from the National Maritime Museum in which climate historian Clive Wilkinson discusses the archives of old shipping records he helps recover.
Finding the data
After speaking with the climate scientists, Clive’s first job is to identify appropriate ships and find out which archives hold the required records.
I compiled an inventory of every major Royal Navy vessel from 1800 to 1950, where it was, what its movements were, where its logbook is, and I also compiled reports for them, which are available, concerning logbooks generally, where they are in the archives, what they consist of, what some of the problems are in using them, this sort of thing. [CRU_03]
He then works with the local archivists to negotiate the project’s access to the archive and begin locating the shipping logs that need to be recovered – or, if they are in a particularly poor state, rescued.
Where no cataloguing and digitisation has taken place, finding the shipping logs often requires a bit of detective work and an element of serendipity. Clive finds that the most fruitful place to start looking is often in the relatively un-catalogued sections of the archive.
When it says miscellaneous in my view it’s because it’s got lots of stuff with numbers on and they didn’t know what to do with it. [CRU_03]
Recovering the data
Once suitable records are found, the data recovery process begins with a historian or archivist such as Clive compiling a survey of the archived materials. Clive also uses his vast knowledge of shipping logs to gauge how useful they will be for the climate scientists, and advises them on any problems they are likely to encounter using the data:
Well, my expertise is finding the data, knowing how it was collected, knowing some of the problems of the–, what I’ll do is I’ll look through the logbooks, and I’ve got a lot of experience, I’ve looked at thousands of them, and I anticipate the problems they’re going to have with the data. And sometimes I say, “You’re going to have this problem. This is how you overcome it.” [CRU_03]
Climate scientists then use this survey to select which records are of most use to them. These selected log books are then digitised and uploaded into a central repository.
A preservation process will also be undertaken for records that are in a particularly poor state of repair:
…and the archivist looks at them and says, “How good condition is this record?” You know, “Can we just photograph it, or do we need to repair it first? Is it going to fall to pieces on us? Does it need curating?” So they do the curating as necessary, and then we try and photograph it, and that requires a bit of human expertise because the quality of the ink after 150 years is not always great. Okay. But usually we get fairly legible photographs out of it. [MO_03]
These digitised ship log books are then assessed by a climate scientist, and records are selected and released for transcription by citizen scientists working on projects such as Old Weather.
Making a contribution
Whilst Clive works with a business partner, and in collaboration with a number of different climate scientists and archives, his work has been largely solitary with significant amounts of time spent working in archives around the world.
Despite the solitary nature of a lot of the work, his underlying motivation is social in that he wants to use his skills to make a contribution and help scientists to understand the climate better:
My key motivation is–, well, it may sound rather strange but I actually feel I’m doing something extremely useful….Well, climate change, environmental change, environmental risks, severe weather, I mean, these are all things we need to understand better and I feel I’m making a contribution to help the scientists do their work. So we all like to think we’re doing something useful. [CRU_03]
I can’t save the planet on my own. I can just do my little bit. [CRU_03]
Clive was deeply knowledgeable about shipping logs stored in archives. He gained his skills and knowledge from many years of experience in the field. He combined this experience with a curiosity driven method of discovery:
My favourite research method is serendipity…detective work.[CRU_03]
This combination of experience and enjoyment in his work contributed to a confidence in what he was able to achieve, and a quiet pride in the rare contribution that he was able to make to climate science:
If it floats and has a logbook I’ll find it.[CRU_03]
Whilst some projects he worked on were driven by the information needs of climate scientists, the value of the ship log books found in the archives did not need to be immediate for Clive. For him, the archive records needed to be preserved and recovered for future generations even if their immediate use was not obvious:
“Oh, we won’t digitise that because that’s not of any interest.” And I was saying, “Well, it might be of interest to somebody one day.”[CRU_03]
Through spending a lot of time in archives, Clive was familiar with many historical weather observers on ships and on land and demonstrated an affinity for their work. He talked enthusiastically about the work of early weather observers such as Charles Meldrum and Edmond Halley, and the crews of historical ships:
Back centuries ago they were very good observers but they didn’t necessarily understand what they were observing. But their observations were excellent, make no mistake about that. It didn’t surprise me because the reason mariners took weather observations was for safe and efficient navigation, and so they were very keen.[CRU_03]
Whilst climate scientists recognised the importance and value of Clive’s work, significant frustrations were present in terms of the funding environment for data recovery work:
It’s very difficult to get funding for data recovery because people want scientific results. A lot of the funding bodies or people who make these decisions don’t realise that unless you’ve got the data, you’re not going to do any science, but they don’t want to fund the recovery. … No one’s got any money…with the ACRE initiative, when we have our annual meetings we get people from all over the world, and the general complaint is no one’s got any money. [CRU_03]
When we spent time with Clive, he was working unpaid hoping that his investment of time would result in funding at a later date:
At the moment I’m working for nothing, okay?… But we believe in what we’re doing….and really no one else can do it because I know where all the data is. [CRU_03]
However, he perceived the dependency of the historic climate data infrastructure on voluntary labour as unacceptable:
But again, it’s all voluntary and it’s a charity. Do we want to do climate research on charity?… I have very strong feelings about this. I think it’s terrible that, you know, we’re doing this. [CRU_03]
The frustration that he experienced resulted in a pragmatic response to financing work. Clive reported that he and his colleague had begun looking to the private sector for investment:
My colleague and I are saying, well, where is the money? The money’s in private businesses. How can they benefit from what we’re doing? If we can show them the benefit, if they can have something that will help them manage decisions, then maybe they’ll fund this… with the ACRE initiative, when we have our annual meetings we get people from all over the world, and the general complaint is no one’s got any money. [CRU_03]
However, it is uncertain how reliance on private investments will impact on the shape of data recovery efforts in the future, and whether the funding of private investors will meet the needs of climate scientists.
- Transcript of interview with Clive William Wilkinson [CRU_03]
- Audio recording of interview with Clive William Wilkinson [CRU_03]
- Transcript of interview with MO_4
- Audio recording of interview with MO_4 (pending)
- Transcript of interview with MO_3 (pending)
We’d like you to take a minute to reflect on what you have read above and add a comment to the discussion below.
Here are some questions to get you started:
- Were you surprised to find out how much valuable climate data is stored in archives around the world?
- How do you think the cultural values of the climate historian might impact upon the ability of climate scientists to find the data they need?
- What value do you think collaboration between climate scientists, historians and archivists can bring to our understanding of climate change?
- What are your thoughts about the difficulties climate historians such as Clive have finding funding for their data recovery and rescue work?
Once you have posted a comment, follow the blue line to the Old Weather project by clicking the link on the right, or if you prefer navigate to another station using to map at the top of the page.
By posting a comment you confirm that you agree to our conditions of participation.