Paved with Good Intentions

February 14, 2011

I suppose it takes a certain amount of chutzpah to organise a Policy Forum entitled ‘The Future of Open?’, and then argue, in a session on ‘Open Information’, that the UK’s Freedom of Information Act 2000 and Environmental Information Regulations 2004 currently require universities in England, Wales and Northern Ireland to be rather too open.[1] More precisely, my argument was that, by treating universities like any large public sector institution, both the legislation itself, and the Information Commissioner’s current interpretation of it, risked causing significant damage to the research environment for which UK universities are justly famed. This risk arose from the failure of the legislation to distinguish adequately between universities’ operational information, and information in the form of data that university researchers generate in the course of their research. Some evidence of this risk was apparent in the furore around the work of the Climate Research Unit at the University of East Anglia, and in the case of data relating to tree ring research at Queen’s University, Belfast, although both those examples turn very much on their particular facts. 

That somewhat contrarian position met with muted enthusiasm from the audience. Yet, since the Policy Forum, in the course of recent consultancy work carried out in Higher Education, my initial misgivings have, if anything, become more concrete. Recent pronouncements from both the ICO and the government on FOI suggest that neither fully understand, or are equipped to assess, the potential implications of their current stance for university-based research in the UK. 

Universities and FOI

The Freedom of Information Act 2000 covers a wide array of ‘public authorities’ ranging from central and local government, the police and universities[2] to a host of public bodies as diverse as the Adjudicator for the Inland Revenue and Customs and Excise, the Human Fertilisation and Embryology Authority and the Government Hospitality Advisory Committee for the Purchase of Wine. All in all, some 100,000 public authorities are covered.[3] Any person can make a request for information to a public authority, which must state whether it holds the information specified in the request and, where required, provide that information. 

In this era of Wikileaks, many proponents of freedom of information appear to regard ‘openness’ on the part of public authorities, whether voluntary or involuntary, solely as a ‘public good’. FOI, it is said, allows the public to shine the cold light of day on hitherto invisible organisational structures and decision-making processes, enhancing democratic scrutiny and accountability over public authorities’ activities and, in these straitened times, ensuring that they are providing value for money. Indeed, for many supporters of FOI, the main problem with the UK legislation is that it promises more than it actually delivers. There are a wide range of exemptions available to public authorities, and engagement with the legislation has, particularly in central government departments, often been grudging at best and obstructive at worst. There are undoubtedly aspects of the legislation and its administration that could be adjusted to ensure more effective compliance. The Ministry of Justice’s announcement, in January 2011, about future plans for both FOI and DP frameworks suggest that some change is imminent.[4] 

Those tasked with handling FOI requests often, however, have a different perspective on the value of FOI. In the HE sector, as elsewhere, some FOI managers are sceptical as to the extent to which, for example, any genuine ‘public interest’ is actually served by journalists trawling for a story via round-robin requests to universities, or whether much of the information disclosed as a result of those requests is actually of value in moving public debate forward. There are also problems with what we might term ‘FOI arbitrage’. This takes the form of commercial organisations making blanket requests to all universities about particular subject matter (eg student numbers or commercial contracts) with the aim of then aggregating and marketing the responses. This passes the cost of collating the necessary information to the universities, who are unable to refuse the requests (or if they refuse them, are entangled in time-consuming appeal processes), or to pass on the fair cost of complying with them. Outsourcing the cost of research in this fashion allows the third parties involved to extract a profit they otherwise could not, at the expense of the universities. This ‘freeriding’ is not an issue exclusive to universities and, while universities can ill-afford extraneous costs, some commentators might say that these are relatively fringe issues – a minor cost to be borne for the greater good. Others point out that such ‘arbitrage’ activity is disruptive of regular business; damages morale amongst staff who feel they and their universities are being taken unfair advantage of; and is destructive of the institutional goodwill which facilitates effective FOI compliance. 

This is not to argue that FOI should not apply to UK universities.[5] Universities receive considerable public money and, more importantly, make decisions which affect an increasing percentage of the UK population (eg on fee levels and student recruitment). There is no obvious reason why their internal processes and decision-making should not be open to reasonable scrutiny, and their management and employees held accountable. For example, information requests to universities have been used to assess the extent of workplace bullying or harassment,[6] student dropout rates,[7] and the extent to which universities recruit students from disadvantaged backgrounds.[8] However, it is possible to demonstrate that the availability of FOI is not always an unqualified public good, and that unexpected consequences can arise where FOI legislation is vague as to what legitimate ‘public interest’ goals might be or, as will be discussed below, where it is applied as a ‘one size fits all’ solution. 

University research data and FOI

Given that it covers over 100,000 public authorities of diverse types, sizes and roles, it is perhaps unsurprising that the FOIA 2000 (and the EIR 2004) has a ‘one size fits all’ feel to it. One might imagine that, given the range of controversies to deal with at the drafting stage – which organisations to designate as public authorities, what exemptions to permit – there was little time for consideration of whether university research data was an appropriate set of information to which to apply FOI, or whether the exemptions available would adequately protect the legitimate interests of the universities or academic researchers.[9] The universities themselves were, in the early days of implementation, rather more concerned with producing publication schemes and compliance mechanisms for their organisational data than with the possible implications for the disclosure of research data. While there were some early suggestions that journalists might use FOI requests to obtain access to researchers’ data, prior to that research being published, a paucity of such requests meant that the issue received little attention. 

Two recent cases have pushed the issue back into the limelight. Both cases involved academic research in an area of controversy, specifically, climate change. The first (often referred to in the media as ‘Climategate’) concerned the Climate Research Unit at the University of East Anglia, the second an Emeritus professor at Queen’s University Belfast. For the purposes of this article, the important thing about these cases is not their facts per se, but rather what they reveal about the both the impact of FOI on small research units and the ICO’s attitude towards university research data. 

With regard to the CRU/UEA case, the UEA gave evidence before the House of Commons Science & Technology Committee that: 

In July 2009 UEA received an unprecedented, and frankly administratively overwhelming, deluge of FOIA requests related to CRU. These amounted to 61 requests out of a 2009 total of 107 related to CRU, compared to annual totals of 2 in 2008 and 4 in 2007 (University totals for those years were 204, 72 and 44 respectively).[10] 

At the time the CRU had 16 staff of which just three were full-time posts and the rest largely postgraduate students and post-doctoral researchers.[11] The volume of requests required the input of the UEA’s Information Policy Compliance Manager, Director of Information Services, a pro-VC, and additional administrative/secretarial support. Despite the extra resources, the pivotal role of the three full-time CRU staff in dealing with the requests meant that there was a significant impact on the work of the CRU. In essence, its senior staff were significantly hampered in their research by the administrative overhead of what appears to have been an orchestrated campaign.12]

In his evidence before the Science & Technology Committee, Richard Thomas, the former ICO, stated that: 

The public must be satisfied that publicly-funded universities, as with any other public authority in receipt of public funding, are properly accountable, adopt systems of good governance and can inspire public trust and confidence in their work and operations. The FOIA, by requiring transparency and open access, allows the public to scrutinize the actions and decisions taken by public institutions. Failure to respond or to respond properly to FOIA requests undermines public confidence in public institutions. (my italics) 

It is far from clear to me that requiring researchers to provide their research data, whether ‘raw’ or ‘processed’, can in fact be used meaningfully to achieve the ends Thomas outlines above. Thomas went on to say that: 

The fact that the FOIA requests relate to complex scientific data does not detract from this proposition or excuse non-compliance. The public, even if they cannot themselves scrutinize the data, want to ensure that there is a meaningful informed debate especially in respect of issues that are of great public importance currently and for generations to come. 

Ensuring ‘meaningful informed debate’ is undoubtedly a laudable goal, but it is hard to see why in the context of academic research data, this requires the application of the FOIA. All UK academic researchers undertake research with the aim of eventual publication, firstly because peer review is a vital component of validating research, and secondly because UK Higher Education funding, and thus inevitably academic promotion, is heavily predicated upon producing research publications.  

The apparent inequity of the UK position is demonstrated in the Queen’s University Belfast case. Here, a Professor of Palaeoecology at Queen’s University Belfast collected data relating to tree rings over a number of years. A third party, a science blogger, described by The Guardian as ‘[a]n arch-critic of climate scientists’[13] made a request under the EIR 2004 for all electronic data relating to tree ring research held by QUB. QUB wished to withhold access, on the grounds, amongst others, that the information requested was being used in ongoing research which would result in future publications;[14] that its disclosure would adversely affect QUB’s intellectual property rights;[15] and that the data was commercially confidential.[16] 

The third party complained to the ICO, who rejected all of QUB’s submissions.[17] With regard to future publication, FOIA provides that a public authority may refuse to disclose information where the request relates to material which is still in the course of completion, to unfinished documents or to incomplete data. Despite QUB stating that the information requested was being used in ongoing research which would result in future publications, as well as to update and develop new tree ring chronologies, the results of which would be published via the Internet on the International Tree Ring Data Bank (ITRDB) within the next 12 months (this mechanism being used primarily to prevent deliberate or inadvertent misinterpretation),[18] the ICO held that: 

…the raw data was collected over a period of 40 years, and is now being used for research. This does not suggest to the Commissioner that the data is unfinished or incomplete, rather that, whilst the research utilising this data is ongoing i.e. the analysis of the data, the data itself has already been collected and is therefore not unfinished or incomplete.[19] 

With respect, this appears to be based on a fundamental misunderstanding of how research in universities works. There is, of course, already an expectation that academics will release sufficient research data for others to assess the value of their published research, and in some areas researchers may proactively release research data in advance of publication. Many UK funding bodies require the archiving of research data on completion of projects for access by other researchers. However, the ICO’s position appears to be that once research is completed, but before the researcher has completed their analysis, research data may legitimately be subject to FOI requests. This runs counter to most current academic practice nationally and internationally. 

The ICO also held that QUB had not established the existence of intellectual property rights in relation to the withheld information. While QUB argued that it required protection for its ‘ability to commercially exploit the data i.e. use it to attract funding to undertake further research, through preventing access’, the ICO (likely correctly) noted that this in itself did not engage the exception. With regard to the issue of commercial confidentiality, the ICO noted that while the data ‘could be viewed as commercial information as it is commercially exploited by the University’ this alone, in the absence of a confidence owed to a third party, did not engage the exception – the research was undertaken by QUB staff, and QUB collected the data for itself. Again, while the interpretation of the law may be correct, one can only wonder how, with such an attitude to research data, universities are supposed to engage in the kind of activities which politicians suggest that they must in order to raise revenues: creating spin-out companies based on their knowledge; collaborating with business on research and research-based activities; and attracting international investment in the UK from global research-intensive companies. 

How then, was the QUB tree ring data to be used by the requestor? According to The Guardian, the blogger believed that the data could support the case that there was a widespread medieval warm period on Earth 1,000 years ago – this would question the suggestion that warming in the 20th century is unique in recent history. The QUB scientists say the data is irrelevant, having themselves published a study in 2009 showing that the relevant tree ring data is a poor indicator of temperature change. The blogger, ‘who admits he has no expertise in tree-ring analysis, says that whatever the data may or may not reveal, the university has no right to keep the data secret.’ One might query both whether forcing release of the data in this context adds to ‘a meaningful informed debate’, and whether the apparent underlying premise that university researchers should not be able to control access to their research data is truly in the ‘public interest’. 

Lessons from the US

Certainly in the United States, a nation with considerable experience of FOI legislation, such questions appear to have been answered in the negative. In 1999, Senator Richard Shelby of Alabama requested research data on air quality and health from Harvard University which had been used by the Environmental Protection Agency (EPA) to justify raising air pollution standards. Harvard refused. Shelby responded by adding an amendment on the Omnibus Consolidated and Emergency Supplemental Appropriations Act of 1999, ‘to ensure that all data produced under an award [a federal grant] will be made available to the public through the procedures established under the Freedom of Information Act.’ This specifically targeted only universities, teaching hospitals, and non-profit research centers, and did not apply to federally funded research undertaken by for-profit entities. The Shelby Amendment caused great controversy amongst US academic researchers, who were concerned that: 

it would invite intellectual property searches by industry and scientific competitors ,jeopardize the privacy of research subjects, decrease the willingness of research subjects to participate in studies, expose researchers to deliberate harassment, and increase costs and paperwork.[20] 

The result of that controversy was a considerable scaling back of the impact of the Shelby Amendment. Only published data is covered, with ‘published’ meaning an appearance in a peer-reviewed scientific or technical journal, and/or its use by a federal agency in support of a policy or law: drafts of papers, grant applications, other preliminary information, e-mail, personal notes etc[21] are excluded. FOI requests can be denied if the research affects national security, or exposes proprietary/commercial data, trade secrets, medical and personnel records, law enforcement information, and geological data.[22] Crucially, the final version does not require researchers to make data publicly available while their research is still ongoing. 

Developments in Open Research Data

The argument that UK university research data needs to be more open, and that FOI is an effective mechanism to achieve this is increasingly being undermined by the move in academia to open research data models and planned data sharing. As already noted, the UK Research Councils, and bodies such as the Wellcome Trust, increasingly encourage or even stipulate as a condition of grants that research data (and publications) resulting from funded research will after a certain period be made available to the research community. 

Researchers have a legitimate interest in benefiting from their own time and effort in producing the data but not in prolonged exclusive use of these data. Timescales for data sharing will be influenced by the nature of the data but it is expected that timely release would generally be no later than the release through publication of the main findings and should be in-line with established best practice in the field.[23]  

Additionally, some research communities are voluntarily moving to make their research data available (eg PubChem, a database of chemical molecules and their activities against biological assays).[24] Ironically, such moves may create a backlash from corporations and other vested interests that see the accessibility of such free sources of data as a threat to their commercial research data collections and databases.[25] The important element of these initiatives is that the researchers retain control of when the data is released, and ‘established best practice’ plays a key role in influencing appropriate release times. Use of FOI to ‘unlock’ university-generated research data, in contrast, removes the autonomy of the researcher to:

·         determine with whom to share the data prior to public release;

·         determine the appropriate time for release of the data, taking into account variables such as publication timings and legitimate exploitation; and

·         present analysis of the data without pre-emptive re-interpretation and/or harassment from third parties. 

In the words of Professor Myles Allen of Oxford University:

 ‘There was an assumption within the climate science community that we could use our professional judgment to distinguish between professional scientists and activists or members of the public. The big implication in all this for science is that the [FOIA] is taking away our liberty to use our own judgment to decide who we spend time responding to. And that has a cost.’[26] 

The importance of such autonomy becomes apparent when one considers how groundbreaking research (eg Sir Richard Doll’s research linking smoking to health problems or Herbert Needleman’s research into the neurodevelopmental damage caused by lead poisoning) might have fared had the tobacco and lead industries respectively been able to subject those researchers and their teams to repeated FOI requests in the manner currently available under the UK FOIA/EIR regime. 

The Costs of FOI

While permitting public access to University research data may inspire public trust and confidence (although investment in the public understanding of the processes and practices of research would surely be more fruitful in the long term) and may even improve certain working practices, such as research record keeping,[27] these putative gains may well be outweighed by the costs to the public interest. For example, there is already pressure on researchers and their institutions to produce and administer more documentation, not in direct support of research but in order that managers can achieve legislative targets for responses to requests, thereby avoiding embarrassing PR clashes with the ICO; or develop ‘defensive’ information, such as metadata for research data placed in institutional repositories, or information cached in institutional publication schemes. Universities across the UK already face major cutbacks both in full-time staff and the availability of research funding: adding administrative overhead and disrupting research processes is hardly conducive to the goal of maintaining a university sector with a world-class reputation for research. It is striking that US politicians, despite the attraction of playing to an ingrained suspicion of academia amongst broad sectors of the general public and the interests of some extremely well-funded and influential commercial lobbies, in the end shied away from imposing such an overhead on US universities. 

The financial overhead of applying FOI to university research data may, however, pale beside the damage to independent research and academic freedom. If conducting research into ‘controversial’ topics results in concerted targeting of researchers and their institutions by third parties using FOI requests (and not necessarily UK-based third parties, as access to the mechanisms of the UK FOI regime is not restricted to UK citizens), then there is both a disincentive for university researchers to engage with controversial areas of research, and a rationale for university managers to avoid hiring researchers working in such areas, or to seek to dissuade researchers from following certain lines of research. Equally, if third parties can use FOI to obtain research data before university researchers have completed their analysis of it, then FOI may be used by third parties either to obtain the benefits of research without sharing in the costs – another form of FOI arbitrage – or to pre-emptively attack, or attempt to discredit, lines of research which might be damaging to vested interests. 

These conjectures may seem alarmist, but history suggests that without adequate protection for researchers, research data is all too often misappropriated, or misinterpreted, or misused. The UK FOI/EIR legislation, and the ICO’s current interpretation of it, seems to regard university research data as little different to council spending decisions, MP’s expenses, or Vice Chancellors’ salaries. This is a grossly simplified understanding of the rationales of FOI, and one that manages to both misunderstand and devalue the concept of the ‘public interest’. 

There is thus a need to balance the short-term benefit to the public of access to university research data against the longer-term benefit to the public of a UK university research environment where evolving international practices, including open access publishing and open data sharing, influence how researchers and universities provide access to research data. In such an environment, researchers should be able to make efficient use of their time and resources; tackle controversial topics without fear of pre-emptive attack by vested political and commercial interests; and seek to provide resource for their institutions via exploitation of their research. The ICO’s current dialogue with Universities UK provides some hope for a more reasoned approach, but ultimately, as in the US, it is likely to be the legislature where a final decision will lie. 

Andrew Charlesworth is Reader in IT Law at the University of Bristol:


[1]           Scottish public authorities are subject to the Freedom of Information (Scotland) Act 2002, which will not be discussed in detail here for reasons of space.

[2]           FOIA 2000, s 3(1)(a)(i) and Sch 1, para 53(1).

[3]           Ministry of Justice. FOIA 2000: Designation of additional public authorities Response to Consultation (16 Jul. 2009):

[4]           Ministry of Justice. Opening up public bodies to public scrutiny, Press release (7 Jan. 2011):

[5]           Although some have, see Baldwin, J.F. ‘Bad and mad: the FOI requests undermining our independence’ Times Higher Education Supplement (11 Sep. 2008):

[6]           AcademicFOI.Com. Workplace Bullying & Harassment (2010):

[7]  Drop-out rates (2009):

[8]           ‘Twenty-one Oxbridge colleges took no black students last year’ The Guardian, 6 Dec. 2010:

[9]           It is worth noting that this discussion did take place in Scotland, and as a result the Freedom of Information (Scotland) Act 2002 makes a specific, but limited, concession as regards research data (see s 27(2)).

[10]          House of Commons Science and Technology Committee. The disclosure of climate data from the Climatic Research Unit at the UEA, Eighth Report of Session 2009–10 (31 Mar. 2010) at 33.

[11]          Russell, M. et al. The Independent Climate Change E-mails Review (Jul. 2010) at 18, para.2

[12]          Ibid at 90.

[13]          Climate sceptic wins landmark data victory ‘for price of a stamp’ The Guardian, 20 Apr. 2010

[14]          Reg. 12(4)(d)

[15]          Reg. 12(5)(c)

[16]          Reg. 12(5)(e)

[17]          FOIA 2000 (s.50) EIR 2004 Decision Notice, Public Authority: Queen’s University Belfast, 29 March 2010

[18]          Tree ring data is used in climate science, and both data and studies have been targeted by individuals and organisations wishing to disprove the theory that current climate change trends have an anthropogenic basis.

[19]          Ibid. at para.49

[20]          Science, Technology, and Law Panel, National Research Council. Access to Research Data in the 21st Century, (National Academies Press, 2002) at 2.

[21]          Contrast this with the UK ICO’s opinion that ‘Currently, any information held by a university (including laboratory notebooks, simulations, etc.) is subject to the FOIA / EIR.’ Minutes of Roundtable Meeting between Information Commissioner’s Office and Higher Education Sector to discuss the implications of Freedom of Information for the Sector, 29 September 2010.

[22]          H.A. Neal, T.L. Smith & J.B. McCormick. Beyond Sputnik: U.S. science policy in the twenty-first century, (University of Michigan Press, 2008); also M. Woodbury, A Difficult Decade: Continuing Freedom of Information Challenges for the United States and its Universities, E-Law 10(4) (Dec.2003), paras 30-35.

[23]          BBSRC (2007) ‘BBSRC Data Sharing Policy: version 1.1 (June 2010 update)’:

[24]          PubChem:

[25]          J. Kaiser ‘Science resources. Chemists want NIH to curtail database’. (2005) Science 308 (5723): 774.

[26]          Climate researchers ‘secrecy’ criticised – but MPs say science remains intact, The Guardian, 31 Mar. 2010:

[27]          E.g. Report of the International Panel set up by the University of East Anglia to examine the research of the Climatic Research Unit at p.3: