Disclosure, documentary release and candour in government

All governments restrict access to their documents to some degree, for reasons of state. As a lawyer, I have had a good deal to do with this process, generally but not invariably when representing the government before the courts. But this is not going to be a lecture about current public controversies, or indeed about law, and it is certainly not going to contain any reminiscences of mine. What I want to do is to address the issue of state secrecy, not as a lawyer but as a historian.

There is a difference between the historical and the legal approach to these matters. When questions of state secrecy come before the courts, it is usually because the government wishes to prevent the disclosure or publication of information which it regards as harmful to the current functioning of the state. This can involve a difficult balance between the public interest in the effective functioning of government and some other conflicting constitutional interest. It may be some overriding interest of justice, because the documents are relevant to current litigation; or the public interest in citizens being able to hold governments to account, something which they cannot do without knowing what they have been up to. But in none of these contexts are the courts particularly concerned with the integrity of the historical record.

A century and a half ago, the Victorian sage and future Regius Professor of History at Cambridge, Lord Acton, expressed the view that there was a natural conflict between the interests of historians in recording the past and the interest of officials in maintaining the confidentiality of their records. There was, he said, a “great enmity between the truth of history and reasons of state, between sincere quest and official secrecy.” Contemporary historians have generally been content to echo his views on this point. Early disclosure is essential to their craft. But my purpose this evening is to suggest that while there are undoubtedly some cases in which official secrecy is the enemy of historical research, in yhe longer term they are perfectly complementary.

Leaving aside the rather special case of information whose disclosure would be prejudicial to national security, there are two main reasons why the government might wish to prevent access to its administrative records. One reason, which was once much commoner than it is now, is to protect the historical reputation of the state, by withholding disclosure of documents that might make people think ill of it, until the subject-matter has ceased to be controversial. This may involve keeping the historical record of some event secret for decades or even centuries after the participants are dead, in the interests of the disembodied state. The second reason looks not at the ultimate consequence of a decision, but at the process by which it was originally made. It seeks to protect the process of political or administrative decision-making, by ensuring that documents recording the making of policy are not put into the public domain until a reasonable time has passed. The object is to encourage the frank discussion of the issues among decision-makers and their advisers, which may be inhibited if the discussion is shortly made public. Otherwise, the discussion may not occur at all, or it may occur among a smaller group of people, with damaging consequences for the quality of the decision, or it may occur in a less formal setting where it will be recorded in wholly anodyne terms or not at all. Although there are undoubtedly cases where both considerations apply, it must be obvious that there is a radical difference between these two approaches. The first is in reality a form of indirect censorship. It may require the suppression of documents for very long periods of time. No self-respecting historian is likely to support it. The second motive is concerned with the current functions of the state, and with the quality and proper recording of decisions. It is likely to be justified for a much shorter period. It may be fundamental to the integrity of the historical record. It is an area in which the concerns of historians and officials largely coincide. This is because in the long-term it is more important to the writing of history that a record should exist, and that it should be accurate and complete, than that it should be available soon.

When Lord Acton condemned official secrecy, he was mainly concerned with secrecy imposed for overtly propagandist reasons. In his day, the most notable practitioner of this kind of censorship in his day was his particular bugbear, the papacy. The Vatican Archive, one of the most impressive collections of administrative documents in Europe, containing documents going back more than a millennium, was completely closed to outside researchers until 1883. Indeed, secrecy was so fundamental to the traditions of the archive that an inscription carved over the main entrance to the Archives from the Vatican Library pronounced the automatic excommunication of any stranger to the papal service who might enter there. This is not, as some people think, an invention of Dan Brown. It is not. The reason for the policy was that the popes had for many years regarded the spiritual authority of the Catholic Church as intimately dependent on its historical claim to be the sole authentic voice of a Christian tradition extending back to St. Peter. Such controversial episodes in the history of the papacy as the reign of the Borgia Pope Alexander VI, the trials of Bruno and Galileo for heresy and the inner history of the Council of Trent were thought to have a potential to undermine the Church’s mission.

This kind of attitude was becoming rare in Acton’s time. But it was by no means unique and is not entirely extinct today. The will of Frederick the Great of Prussia, who died in 1786, was treated as a classified document on the directions of the German Foreign Ministry until 1916. Large parts of the central archives of the Italian state, dealing with the unification of Italy and the administration of Cavour in the 1850s, remained closed until well after the First World War. Defending this policy in 1912, the Italian Prime Minister Giolitti observed with disarming frankness that “it would not be right to allow our beautiful legends to be undermined by mere historical criticism.” More recently, access to files covering the Second World War at the Central Archive of the Russian Ministry of Defence at Podolsk outside Moscow was severely curtailed in about 2000 in order to protect the official narrative. In Turkey, it is reported that the Central State Archive, which is part of the secretariat of the Prime Minister, makes available files on sensitive subjects such as the Ottoman Empire’s dealings with the Armenians on a highly selective basis.

It would be satisfying to record that considerations of this kind had never been an issue in Britain. Satisfying but untrue. The public records have always offered opportunities for political point-scoring to which English governments have occasionally been quite as sensitive as nineteenth century popes.  The scientific study of archives as a historical source begins in the seventeenth century. In England it was attended by political controversy from the outset. In principle, the archives of the English government were closed to outsiders. Chief Justice Sir Edward Coke gave it as his opinion, in the Fourth Book of his Institutes, that “letters and writing concerning matters of state are not fit to be made vulgar.” In fact, the records of the main departments of state were so insecurely held in his time that it was in practice quite easy to get access to them. The Parliamentary opposition to the first two Stuart Kings included a number of competent antiquaries and historians, such as John Selden, Sir Robert Cotton and Sir Symonds d’Ewes, who saw in the public records a stick with which to beat the King. They scoured the records of the middle ages for material to support their case about the autonomy and privileges of Parliament. They went through the accounts of the medieval Exchequer in order to demonstrate that earlier Kings had been able to fund their governments without resort to taxation, unlike Charles I. They produced examples of the English Kings’ defence of England’s national interest to compare with what they saw as the craven conduct of contemporary ministers. Charles I’s decision in 1629 to seize the library of Sir Robert Cotton, which included a large number of originals and transcripts derived from the public records, shows that some of these attacks hit home.

The English civil war was followed by a period of relative openness, at any rate for those researchers who were willing to confront the problems of physical dispersal, rats, damp, and the almost complete absence of inventories. At the beginning of the eighteenth century, England became the first country in Europe to publish a large part of its diplomatic archives. The twenty folio volumes of Rymer’s Foedera, was originally commissioned by William III’s first minister Lord Halifax. It published in full many thousands of diplomatic instructions and memoranda, account books, administrative instructions and treaties relating to English foreign policy between 1100 and 1654. This was something which no European government had ever done before, or indeed afterwards until well into the nineteenth century. The Foedera is still, to this day, the basic tool of research into the foreign relations of England’s medieval Kings.

Ironically, the main occasion for the revival of censorship of the archives was the creation in the mid-nineteenth century of the Public Record Office. This event made it necessary for the first time to devise rules governing what could be made available to researchers, and what could not. For many years, each department was left to make its own rules. They were often overtly designed to protect the image of the state against criticism. At the time when the Public Record Office opened its doors to readers, in 1856, the Foreign Office insisted that nothing could be disclosed which was later than 1628. Among the documents which were restricted in this way were a large number which had actually been published by Rymer. In the following year, the closure date was changed to 1760, where it remained until after the First World War. When in 1891 the Master of the Rolls, who was then responsible for the Public Record Office, had the date moved forward to 1830, the Foreign Office had the decision reversed. There was, they said, “no sufficient guarantee against the admission of undesirable persons” to the reading rooms. As a result, the records of English foreign policy after 1760 could be consulted only with the permission of the Secretary of State, and then only on terms that the researcher submitted his notes to the Foreign Office for review. They were frequently returned with extensive sections blocked out in ink. The main result of this policy, as the Oxford historian H.A.L Fisher pointed out, was that British foreign policy in the century after 1760 was generally studied through the eyes of Britain’s historic rivals, notably France and Germany, whose diplomatic archives for this period were for the most part open. At the beginning of the twentieth century, even the domestic archives of the period of the revolutionary and Napoleonic wars were regarded as too sensitive for disclosure. The Prime Minister, Arthur Balfour, intervened personally in 1902 to veto a proposal to open all documents up to 1815, on the ground that even after the passage of a century the disclosure of documents relating to the government of Ireland was “full of contentious matter which bears on slumbering, though not yet deceased controversies.” Others objected that the opening of the archives of this period would undermine relations with France by revealing the scale on which Britain had funded the domestic opposition to its rulers during the revolutionary and Napoleonic wars. There is something rather touching about the confidence of these public servants that British undercover operations of an earlier age were still secret in 1902. In fact, much the best source of information about them is the files of the French chief of police Joseph Fouché. They had been open to readers in the French national archives for some time, and some of them were already in the process of being published by Alphonse Aulard and other French scholars.

It was not until between the wars that the rules were relaxed so as to make possible the serious study of modern English history by researchers in the Public Record Office, even by “undesirable persons”. However, the real landmark in this area was the Public Records Act of 1958. The Act reflected the recommendations of the Grigg Committee, which had reported in 1954. It removed the right of individual departments to make their own rules and introduced a standard closed period of fifty years, except in the case of documents whose disclosure would be damaging to national security or which contained confidential information about individuals still living. The exceptions were to be considered by the Lord Chancellor’s Advisory Council on the Public Records, a statutory body comprising officials, historians, archivists, and other experts, whose assessments were objective, well-informed and almost invariably followed. The fifty-year period was overtly designed to protect the decision-making process, not the historic reputation of the state. It had been proposed by the Grigg Committee because it was thought to represented the maximum duration of a political career.  Of course, like any closed period, the fifty-year rule was liable to suppress politically embarrassing information as well. But that was not the object of the exercise nor of course, in the long term, its effect. There has, since 1958, been very little scope for discretionary decisions designed to mould the conclusions of historians or restrict the use that unfriendly persons might make of the public records.

More recently, the whole concept of protecting the decision-making process has been challenged. The closure period was reduced to thirty years in 1968, and the last government announced in February of this year that it would legislate for a normal period of closure of twenty years. The Thirty Year Rule Review Panel, which had reported on the question in 2009, had recommended fifteen years, and some of the witnesses who gave evidence before it would have preferred a period as short as five. These proposals raise far more difficult questions than the kind of censorship which used to be the main preoccupation of government departments. The first point to be made is that the early disclosure of documents in the Public Record Office (as I persist in calling it) is only part of a much broader problem. It is just one of the ways, and not necessarily the most important, in which confidential exchanges within government may be come into the public domain. It is probably less significant than the two main avenues of unofficial disclosure, namely the memoirs of ministers and civil servants; and the growth of a culture of leaking, fed in part by a growing hostility to the whole notion of secrecy in the public service, and in part by a rejection of hierarchy, which leads individuals to feel that they should follow their own consciences in deciding what they have a right to say and the public to know. These things represent a major change in our political culture. During the Second World War, about 12,000 men and women served at one time or another at the Government Code and Cypher School at Bletchley Park, most of them young and recently recruited from the ordinary peacetime occupations to which they returned in 1945. Several hundred more were cleared to receive the intelligence that  Bletchley Park generated. There was no value in keeping the work of Bletchley Park secret for as long as was. Yet the culture of public affairs at the time meant that not one the many individuals involved disclosed the existence of wartime Ultra until 1974, when the officer responsible for controlling the distribution of Ultra before 1945 published his memoirs. This is a generational thing. I think that that degree of reticence would be inconceivable today.

Now, I have no position on the question whether the current belief in disclosure is good or bad for our standards of government and public administration. I doubt whether a single answer to that question is even possible. But I think that we need to be more honest with ourselves about the impact that it will have on the quality of the sources which will be available to future historians writing about the events of our own day. Of course, plenty will be written. But how reliable will it be?

I specialise in a period of history, namely the late middle ages, when no one had any reason to ask this question.  Although the use of documentary evidence for the writing of history was not unheard of in the middle ages (official documents are quite often quoted in chronicles), the idea of opening the archives of the state to outsiders for the purpose of research was simply unheard of. There are, however, some basic points about the nature of historical evidence that are broadly true in all periods. As a general rule the more self-conscious a historical record is, the less valuable it is as a source of information. Take a royal proclamation of the kind which represented the main means by which the Tudors communicated with their subjects en masse. It was intended to be read out at the sound of a bell in market squares across the country. It is a highly self-conscious source, deliberately contrived to make an impact on the public, perhaps as much by what it suppressed as by what it declared. As evidence of the government’s thinking, its value is limited by its public character and rhetorical purpose. The confidential memorandum which proposed it, or the record of the Privy Council at which it was discussed, are likely to tell us much more. The same point may be made by reference to the archives of diplomatic exchange. The English diplomat Sir Henry Wootton, who represented Elizabeth I and James I over many years at the courts of Germany and Italy, memorably defined an ambassador as an “honourable man sent forth to lie for his country abroad”. The ambassador’s formal instruction, which was a public document delivered to the court to which he was accredited as evidence of his authority, was a self-conscious statement, and as such almost useless as a historical source. The private instructions of a medieval or Renaissance diplomat, which told him what he was expected to achieve and how, is likely to be far more valuable to the historian. The deliberations of the council which prepared it will probably be more valuable still. Quite often, the best source for some fact is a document which incidentally records it in the course of fulfilling in an obscure corner of the administration some utterly humdrum function, such as an entry in the Exchequer accounts recording a payment to a diplomatic messenger, to a spy for an undercover mission, or a soldier for some unheroic enterprise. The reason for this is that of all government records, accounting documents are probably the least self-conscious. The clerk who writes them is likely to have little interest in the subject-matter and none at all in the impact which it may make on outsiders or later historians. I make these points in order to suggest that the prospect of publicity is a major source of bias in governmental documents. The problem about prematurely disclosing them is that the knowledge that this will happen adds a significant and unwelcome element of self-consciousness to their contents. It leads to selective and sometimes tendentious omissions. It encourages a level of level of generality which omits historically important but politically equivocal detail. It often causes the document to be written in a way which means much less to outsiders than to those who made it. All of this tends to undermine the integrity of the historical record.

The Radcliffe Committee, reporting in 1976 on the principles which should govern the publication of ministerial memoirs, identified three categories of information that ministers should not be permitted to disclose. The first two categories comprise information whose disclosure would contravene the requirements of national security or damage this country’s relationship with other governments. The third category, which is one germane to my theme this evening, is both more controversial and more difficult to define. This comprises information whose disclosure would be destructive of confidential relationships within government. Justifying this category in their report, Lord Radcliffe’s committee observed:

… the argument in its favour is quite simple and does not gain byelaboration…Those who are to act together in pursuance of a policy agreed in common do require and expect the observance of confidence as to what they say to each other; and unless they can be assured of the maintenance of that confidence they will not speak easily or frankly among themselves. Opinions, perhapsunpopular, perhaps embarrassing, will be muted or suppressed if they are known to be liable to future disclosure at the whim of some retired colleague. Business which should be discussed by the whole body will tend to be settled by two or three in a corner.

The principle that candour in the expressing and recording of opinions depends on confidentiality had never previously been articulated as clearly as it was by the Radcliffe Committee. But it was already, in 1976, a principle of some antiquity. To this day, every new Privy Councillor swears an oath, the essential features of which can be traced back to the thirteenth century, in which he swears to “keep secret all matters committed and revealed unto you or that shall be treated of secretly in council.” Such total reticence has never been easy to enforce. As Thomas Carlyle observed in his History of Frederick the Great, “men are very porous, weighty secrets oozing out of them like quicksilver though clay jars.” During the nineteenth century, English governments had few tools at their disposal for enforcing the confidentiality of the decision-making process, other than persuasion and social pressure. This sometimes worked, sometimes not. After the death of the Prime Minister George Canning in 1827, ministers went to considerable lengths, with very little success, to prevent his widow from commissioning a biography based on his personal papers, mainly because they were concerned that it would expose the bitter arguments over Catholic emancipation which had divided recent administrations. Half a century later, the posthumous publication of the diaries of Charles Greville, in which the diarist had patiently recorded the inner debates and scandals of the political class over the forty years in which he served Clerk to the Privy Council, was regarded as a profoundly shocking event.  Queen Victoria denounced the author’s “indiscretion, indelicacy, ingratitude, betrayal of confidence, and shameful disloyalty towards his sovereign”. Obscure pressures on Greville’s literary executor and publisher resulted in substantial parts of the diary being cut out of the published version, and when the manuscript was presented to the British Museum in 1895, the trustees directed that it should not be made available to readers. It was not published in full until the 1930s.

These challenges to the accepted convention were, however, relatively rare until after the First World War, when a flood of memoir-writers, diarists and manipulative leaking, was met by the first systematic attempts to enforce the confidentiality of the decision-making process. The advent of a Labour government in 1929 which was both bitterly divided and relatively inexperienced brought many of these issues to a head. After the controversial decision of the Cabinet in August 1931 to reduce unemployment benefit by 10 per cent, the Daily Herald published on 24 August a complete and accurate account of the discussion, with the names of the ministers who had voted against. This disclosure exposed the divisions of the government and made it impossible for ministers to maintain a united front against the instincts of the their own party in the House of Commons. The leak, which was intended to force the government back onto the path of ideological rectitude, in fact had exactly the opposite effect. The government resigned that afternoon. The conservative-dominated National Government endorsed the reductions. Whether the Daily Herald’s disclosure was constitutionally desirable is a difficult question. The fact that the general election which followed in November returned the National Government with the largest Parliamentary majority ever enjoyed by a British government may suggest that it served the cause of democratic choice. However, for a historian, what should matter is that the discussion of expenditure cuts around the Cabinet table would certainly not have been as open as it was if the leak had been anticipated. The decision would have been fixed in advance of the Cabinet meeting in the course of private discussions between much smaller groups of ministers. We would know a great deal less about it today.

That at any rate seems to have been the view of the Cabinet Secretary, Sir Maurice Hankey. He inaugurated after 1931 a number of rules about the retention of papers by ministers, access to sensitive policy documents by non-official persons and prosecutions in the case of the more egregious breaches. When, in 1934, Edgar Lansbury, the son of the then leader of the Labour Party, published a book about his father’s career which quoted extensively from cabinet papers of the Labour Government, he was charged under the Official Secrets Act and fined. Official papers were energetically retrieved by the Cabinet Office from serving and former ministers. Some, including Churchill and Lloyd George, were grand enough to resist. Most were not. When Arthur Balfour died in 1930, he left his private papers to his niece Blanche Dugdale to enable her to write the official biography. She subsequently gave them to Library of the British Museum. But the Cabinet Office prevailed upon the trustees of the Museum to close them, as a result of which they were not available to readers until 1968.

It is doubtful whether either the Cabinet Office or the trustees of the Museum had any legal right to act as they did. Nevertheless, Hankey’s principles survived more or less intact until the 1970s, when they began to break down under pressures very similar to those which had destroyed the previous, informal system before 1931. The publication of the first volume of the Crossman diaries in 1975 was a landmark in this process, not least because it provoked the first judicial decision on the issue. The Labour government of 1964-1970 contained an unusually large number of diarists. However, Crossman was special, because he deliberately set out to challenge the conventions concerning the secrecy surrounding government decision-making, of which he strongly disapproved. But it is fair to say that his diary includes a fair amount of material to support both sides of this particular argument. On 26 January 1967 there was a discussion in Cabinet about ministerial diaries, following a report in The Observer that Richard Crossman and Barbara Castle were both writing one. The discussion is recorded in different terms in the official minute in the Public Record Office and in the diaries of several of the participants. The problem discussed was not of course the mere keeping of diaries, but the possibility of premature publication. Both Crossman and Castle were reported to have signed contracts with publishers. George Brown, then Foreign Secretary, was concerned about the implications for the openness of discussion in Cabinet and appears to have had substantial support from other ministers. Harold Wilson spoke from a lengthy brief prepared by the Cabinet Office to the same effect. Tony Benn said that he too was a diarist, and what he objected to was that he had not been approached by a publisher with a contract. However, what mainly exercised the ministers around the table was the possibility that Crossman’s diary might be published before the next election. Some pointed out that even if publication was delayed until after the next election, if there was a very narrow Labour or Tory majority, followed by another election, and Crossman’s diaries came out between the two, it would do enormous damage to the fortunes of the Labour Party.

This incident, like the affair of Daily Herald in 1931, suggests a more general conclusion about the place of cabinet secrecy in our current constitution. The original purpose of cabinet secrecy in the eighteenth and early nineteenth centuries was to support the doctrine of collective ministerial responsibility. Secrecy enabled ministers and their advisers to discuss their differences freely without exposing them to the King, who might otherwise be enabled to pick off some of them against the others. Instead, he was faced with his ministers’ conclusions apparently endorsed by them en bloc. Clearly, the need to limit monarchical discretion is no longer a relevant factor. But that does not mean that the collective responsibility of ministers is redundant. Its modern function is to enable a government to discuss its differences internally while maintaining a common front in the face of its own party and the electorate. Whether or not we regard this as a good thing, it seems to me that for as long as parties remain the basis of our political system it is an inevitable thing. The only consequence of constructing our arrangements for disclosure on some different basis would be that the same discussions will occur elsewhere, out of the hearing of diarists or minute-takers. Richard Crossman was able to assure his colleagues that the terms of his contract with his publishers would ensure that the diaries were not published until well after an election. If he had told them that extracts might appear at any moment or just before an election campaign, it seems clear that Cabinet discussions in his presence would have been limited to matters that were unlikely to be controversial or to expose divisions in the party, while the real discussion would have occurred in small cabals elsewhere.

It is sometimes forgotten that although Lord Widgery refused to grant an injunction against the publication of the Crossman diaries, he actually accepted the principle of the collective responsibility of ministers and the confidentiality of governmental decision-making. But his judgment is mainly important for establishing that the confidential character of governmental decision-making was not absolutely enforceable, but only conditionally on the Court’s assessment of the public interest. The critical factor in Lord Widgery’s view was that the first volume of the diaries would appear ten years after the events which it covered. That in his view was long enough to serve the relevant public interest. I very much doubt whether today the Courts would even require ten years to have passed, unless the subject-matter fell within one of the first two of the Radcliffe categories, relating to national security or foreign relations, and perhaps not always then. The effective abolition of class claims to public interest immunity, i.e. claims to withhold disclosure of documents because of the principle of disclosing material of that kind rather that the sensitivity of its particular content, seems to point in the same direction.

In practice, in the last twenty years, Lord Radcliffe’s third category (documents tending to undermine relations of confidence within government) has been disregarded not just by the authors of ministerial and civil service memoirs but also by the Cabinet Office and the Foreign Office charged with enforcing it. The memoirs of ministers like Nigel Lawson, Lord Owen and Clare Short, and officials like Sir Christopher Meyer and Dame Stella Rimington, are cases in point. The reluctance of the Cabinet Office to insist on the third of the Radcliffe principles is understandable. The decisions of the Courts are at best equivocal. Moreover, injunction or no injunction, the confidential character of a text, once it has been written, is difficult to maintain in the age of international publishing and the internet. Lance Price’s 2005 memoirs of life in No 10 were substantially altered at the request of the Cabinet Secretary. But the Daily Mail, to whom he had sold the serialisation rights, subsequently obtained without his authority a copy of the unexpurgated version and published extracts side by side with final text.

There is a case, which has been made by for example Sir Christopher Meyer and Clare Short, that cultural changes have killed off confidential relationships within government anyway. There is a case, which was made by Richard Crossman, that even if these relationships subsist, they are less important than satisfying the legitimate interest of the public in knowing how government works. There is something in both of these points, although perhaps not as much as their authors believe. However, I believe that both points are misguided for a different reason, namely that they are self-defeating. They underestimate the desire of officials and ministers to be able to discuss issues confidentially, and the likelihood that they will take  avoiding action so that they can continue to do so.

In a revealing passage from his evidence to the Thirty Year Review Panel, Lord Wakeham remarked that when he was in government, between 1979 and 1994, he told his officials that they were never to put before him for his signature a letter which would be embarrassing if it appeared on the front page of The Guardian. Now there are of course two possible interpretations of this interesting instruction. One, which I do not for a moment believe, is that Lord Wakeham never intended, in his capacity as a minister of the Crown, to say anything which would be unpleasing to readers of The Guardian. The more likely interpretation is that whenever he had something to say which readers of The Guardian would not like, he intended to ensure that it was not recorded in writing.

The decline in the standard of government record-keeping over the past twenty-five years or so is a phenomenon which has often been remarked by those who are well-placed to judge. It has in recent years been associated with a tendency for critical decisions to be made by very small and informally constituted groups of ministers, what has been called “sofa government”. The tendency emerges very clearly from, for example, the 2004 report of the Butler Committee on the use of intelligence on weapons of mass destruction and the evidence given to the Chancery Division in the Railtrack litigation of 2005. I do not want to suggest that the fear of leaks or premature disclosure is the only factor at work here. But it seems obvious that it is a significant factor, and that the recording of government decision-making is the poorer for it. In preparation for this lecture, I consulted a number of senior civil servants who had recently retired about their practises. The sample is statistically insignificant, seven. It is has also been assembled on a wholly unscientific basis, namely that they were all people whom I happened to know well enough to ask the question and expect an honest answer. With one exception, every one of them admitted to having omitted significant information from internal documents, which in earlier times would have been included, and to having communicated them informally instead, so that they would not be recorded in writing. One of them remarked that in some departments it was quite common for politically sensitive matters to be omitted from documentary records and recorded only on marginal notes written on post-it stickers, which could be removed and binned after the right people had seen it. It is of course a great deal easier to identify these problems than to suggest ways of dealing with them. But I am bound to say that I regard the conclusions of the Thirty Year Review Panel as profoundly eccentric. They  accepted the third of the Radcliffe categories in principle. They also appear to have accepted the basic rationale of the thirty year rule, namely that documents recording the making of government policy should not be disclosed while those involved were liable to be still in government. But they considered that the period of closure should be reduced from thirty to fifteen years for two main reasons. One was that the Freedom of Information Act would result in the disclosure of much of the material anyway, without any delay at all. The other was that  ministerial careers were getting shorter. The Freedom of Information Act is a red herring. If it were relevant, it would suggest that there should be no period of closure at all at the Public Record Office. But it is not relevant, because it is subject to extensive exceptions, which include all three of Lord Radcliffe’s categories of sensitive information. In particular, Section 35 exempts information relating to the formulation or development of government policy and ministerial communications, including proceedings of the Cabinet. As for the suggestion that ministerial careers were getting shorter, that was based on a sample which was heavily weighted by recent and rather untypical experience. The long period of Tory government between 1979 and 1997, and the almost as long period of Labour government between 1997 and 2010, each had the effect of clearing out the old guard in the opposing party. We cannot assume that long periods of single party government are going to be the rule in future. We need not resort to extreme examples, like Winston Churchill, whose ministerial career spanned forty-seven years. Just look at the present government. Although the conservative party has skipped a generation in its choice of leadership, the present Lord Chancellor first became a minister thirty-one years ago and attended his first Cabinet meeting twenty-five years ago. The present Foreign Secretary first became a minister eighteen years ago and first sat in a Cabinet in 1995. At his present age of forty-nine he can reasonably expect his ministerial career to extend well beyond the twenty year period of closure which is currently proposed. The period of time during which civil servants can expect to influence policy has certainly not reduced. Indeed, it can be expected to increase now that it has become easier to serve beyond the traditional retiring age of sixty.

Of course none of this addresses the problem of leaks or ministerial memoirs. There is not much that can be done about leaks other than to pay more attention to the dissemination of information within the public service. As for ministerial and civil service memoirs, it seems to me that public servants should not be at liberty to decide for themselves to disclose information which they have acquired in their capacity as servants of the state and which the state withholds for considered reasons of public policy. As the House of Commons Select Committee on Public Administration has pointed out, if one accepts the Radcliffe principles, there are legally effective ways of giving effect to them.

Underlying all of these considerations, is a basic fact of human nature which it is unrealistic to ignore. The natural sensitivity of public figures to the impact of their discussions on outsiders, will inevitably influence what they will allow to appear in government documents, if these are liable to be released in the course of their careers. Perhaps the last word on this question can be left to Harold Wilson. During the debate in Cabinet in 1967, to which I have already referred, he said that he proposed to write his own memoirs three times. There would be a sober factual account, drawing on information in the public domain, which he would publish as soon as he left office. A second, somewhat more candid account would appear after he finally retired from politics. And a final version containing the real truth would be left with his papers to be published after his death. I ask you, which one would you rather read?