Nick Davies's Blog, page 10

From the archive, 12 October 1981: Championship dream ends with conker

It was a men-only affair when the world title was decided for the 17th time. And the top performers sank 18 pints

Melvin Skelton was philosophical in defeat. "I have been in training for six weeks every Saturday morning," he said. "But at the end of the day, it depends on your conker. The bloke that won hadn't done any training. I had a duff conker."

Melvin, an accountant, had just seen his hopes of becoming the 1981 World Conker Champion shattered into half a dozen pieces of mangled horse chestnut on the village green at Ashton, Northamptonshire. Where had he gone wrong?

"The weather conditions were against me," he said. "The wind kept blowing my conker about. And I had nerves. That's what really cost me the match – nerves. And that duff conker." His trainer, John Hillyard, nodded wisely.

"I have trained two runners-up and one winner," he recalled. "I was champion myself in 1970. I train them with Double Diamond. If they can drink 18 pints, they can play conkers. No women, of course. It's too dangerous for them."

John was one of the founding fathers of the championship. He was there in October 1964 on The Day The Fishing Trip Was Cancelled – as it is known in the popular history of Ashton – with Frank Elson, Joe Horne and Ron Marsh. Instead of fishing they had a conker contest. Ron won and a tradition was born.

Yesterday, the 17th official world championship drew nearly 4,000 spectators, raised more than £1,000 for the blind, attracted 64 contestants, needed the police and army cadets to control traffic, persuaded the local magistrates to grant two hours' extra drinking in the Chequered Skipper, and saw Ashton slump to defeat.

Melvin was their last chance. John and Ron had gone out in a hail of splinters early on. "Bloody awful," said Ron. "First round. Rubbish." Frank and Joe had been unable to perform. The local favourite, Jessie Leigh, had survived to the semi-final.

Melvin, who had competed three times before, struggled through to the final to meet Bill Cox, an outsider from Honiton, Devon. Just after noon they met in the middle of the ring on the village green. The crowd jeered and cheered. Melvin and Bill thrashed into each other. Bill kept missing. Melvin finally landed such a mighty blow that his own conker exploded. It was all over.

Bill accepted the trophy from King Conker, a gardener whose real name is Vic Owen. He wears 17 strings of conkers round his neck on ceremonial occasions – one for each year of the contest. Bill was magnanimous. "I've never played before," he said. "I just came up for the weekend with some friends."

These archive extracts, compiled by the Guardian's research and information department, appear online daily at gu.com/fromthearchive

Nick Davies
theguardian.com © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

MI5 feared GCHQ went 'too far' over phone and internet monitoring

Amid leaks from NSA whistleblower Edward Snowden, senior intelligence source reveals worries were voiced in 2008

Senior figures inside British intelligence have been alarmed by GCHQ's secret decision to tap into transatlantic cables in order to engage in the bulk interception of phone calls and internet traffic.

According to one source who has been directly involved in GCHQ operations, concerns were expressed when the project was being discussed internally in 2008: "We felt we were starting to overstep the mark with some of it. People from MI5 were complaining that they were going too far from a civil liberties perspective … We all had reservations about it, because we all thought: 'If this was used against us, we wouldn't stand a chance'."

The Guardian revealed on Friday that GCHQ has placed more than 200 probes on transatlantic cables and is processing 600m "telephone events" a day as well as up to 39m gigabytes of internet traffic. Using a programme codenamed Tempora, it can store and analyse voice recordings, the content of emails, entries on Facebook, the use of websites as well as the "metadata" which records who has contacted who. The programme is shared with GCHQ's American partner, the National Security Agency.

Interviews with the UK source and the NSA whistleblower Edward Snowden raise questions about whether the programme:

■ Exploits existing law which was passed by parliament without any anticipation that it would be used for this purpose.

■ For the first time allows GCHQ to process bulk internal UK traffic which is routed overseas via these cables.

■ Allows the NSA to engage in bulk intercepts of internal US traffic which would be forbidden in its own territory.

■ Functions with no effective oversight.

The key law is the Regulation of Investigatory Powers Act 2000, Ripa, which requires the home secretary or foreign secretary to sign warrants for the interception of the communications of defined targets. But the law also allows the foreign secretary to sign certificates that authorise GCHQ to trawl for broad categories of information on condition that one end of the communication is outside the UK.

According to the UK source: "Not so long ago, this was all about attaching crocodile clips to copper wires. And it was all about voice. Now, it's about the internet – massive scale – but still using the same law that was devised for crocodile clips. Ripa was primarily designed for voice, not for this level of interception. They are going round Ripa. The legislation doesn't exist for this. They are using old legislation and adapting it."

The source claimed that even the conventional warrant system has been distorted – whereas police used to ask for a warrant before intercepting a target's communications, they will now ask GCHQ to intercept the target's communications and then use that information to seek a warrant.

There is a particular concern that the programme allows GCHQ to break the boundary which stopped it engaging in the bulk interception of internal UK communications. The Ripa requirement that one end of a communication must be outside the UK was a significant restriction when it was applied to phone calls using satellites, but it is no longer effective in the world of fibre-optic cables. "The point is that this is an island," the source said. "Everything comes and goes – nearly everything – down fibre-optic cables. You make a mobile phone call, it goes to a mast and then down into a fibre-optic cable, under the ground and away. And even if the call is UK to UK, it's very likely – because of the way the system is structured – to go out of the UK and come back in through these fibre-optic channels."

Internet traffic is also liable to be routed internationally even if the message is exchanged between two people within the UK. "At one point, I was told that we were getting 85% of all UK domestic traffic – voice, internet, all of it – via these international cables."

Last year, the government was mired in difficulty when it tried to pass a communications bill that became known as the "snoopers' charter", and would have allowed the bulk interception and storage of UK voice calls and internet traffic. The source says this debate was treated with some scepticism inside the intelligence community – "We're sitting there, watching them debate the snoopers' charter, thinking: 'Well, GCHQ have been doing this for years'."

There are similar concerns about the role of the NSA. It could have chosen to attach probes to the North American end of the cables and documents shown to the Guardian by Edward Snowden suggest that key elements of the Tempora filtering process were designed by the NSA. Instead, the NSA agency has exported its computer programs and 250 of its analysts to operate the system from the UK.

Initial inquiries by the Guardian have failed to explain why this has happened, but US legislators are likely to want to check whether the NSA has sought to bypass legal or policy requirements which restrict its activity in the US. This will be particularly sensitive if it is confirmed that Tempora is also analysing internal US traffic.

The UK source challenges the official justification for the programme; that it is necessary for the fight against terrorism and serious crime: "This is not scoring very high against those targets, because they are wise to the monitoring of their communications. If the terrorists are wise to it, why are we increasing the capability?

"The answer is that you can't stop it. It is a self-fulfilling prophecy. The more we develop communications technology, the more they develop technology to intercept it. There was MS Chat – easy. Then Yahoo chat – did that, too. Then Facebook. Then Skype. Then Twitter. They keep catching up. It is good for us, but it is bad for us."

It is clear from internal paperwork that GCHQ has created systems to restrain the use of this powerful tool and to ensure that its use complies not only with Ripa but also with the 1998 Human Rights Act, which requires essentially that the use of the data must be proportional to the crime or threat investigated. Defenders insist that the mass of data is heavily filtered by the programme so that only that relating to legitimate targets is analysed.

However, there are doubts about the effectiveness of this. First, according to the UK source, "written definitions for targeting and filtering are very elastic. They are wide open to interpretation." The target areas defined by the Ripa certificates are secret.

Second, there is further room for interpretation when human analysts become involved in using the filtered intelligence to produce what are known as "contact chains". "Here is target A. But who is A talking to? Now we're into B and C and D." If analysts believe it is proportional, they can look at all the traffic – content and metadata – relating to all of the target's contact." GCHQ audits a sample of its analysts' work – believed to be 5% every six months – but even the statistical results of these audits are also secret.

Beyond the detail of the operation of the programme, there is a larger, long-term anxiety, clearly expressed by the UK source: "If there was the wrong political change, it could be very dangerous. All you need is to have the wrong government in place. It is capable of abuse because there is no independent scrutiny."

GCHQSurveillanceEspionageInternetData and computer securityData protectionNSAThe NSA filesEdward SnowdenNick Davies
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

How does GCHQ's internet surveillance work?

Learn more about the system for collecting content and metadata, and what GCHQ can do with it

What is an internet buffer?

In essence, an internet buffer is a little like Sky+, but on an almost unimaginably large scale. GCHQ, assisted by the NSA, intercepts and collects a large fraction of internet traffic coming into and out of the UK. This is then filtered to get rid of uninteresting content, and what remains is stored for a period of time – three days for content and 30 days for metadata.

The result is that GCHQ and NSA analysts have a vast pool of material to look back on if they are not watching a particular person in real time – just as you can use TV catch-up services to miss a programme you hadn't heard about.

How is it done?

GCHQ appears to have intercepts placed on most of the fibre-optic communications cables in and out of the country. This seems to involve some degree of co-operation – voluntary or otherwise – from companies operating either the cables or the stations at which they come into the country.

These agreements, and the exact identities of the companies that have signed up, are regarded as extremely sensitive, and classified as top secret. Staff are instructed to be very careful about sharing information that could reveal which companies are "special source" providers, for fear of "high-level political fallout". In one document, the companies are described as "intercept partners".

How does it operate?

The system seems to operate by allowing GCHQ to survey internet traffic flowing through different cables at regular intervals, and then automatically detecting which are most interesting, and harvesting the information from those.

The documents suggest GCHQ was able to survey about 1,500 of the 1,600 or so high-capacity cables in and out of the UK at any one time, and aspired to harvest information from 400 or so at once – a quarter of all traffic.

As of last year, the agency had gone halfway, attaching probes to 200 fibre-optic cables, each with a capacity of 10 gigabits per second. In theory, that gave GCHQ access to a flow of 21.6 petabytes in a day, equivalent to 192 times the British Library's entire book collection.

GCHQ documents say efforts are made to automatically filter out UK-to-UK communications, but it is unclear how this would be defined, or whether it would even be possible in many cases.

For example, an email sent using Gmail or Yahoo from one UK citizen to another would be very likely to travel through servers outside the UK. Distinguishing these from communications between people in the UK and outside would be a difficult task.

What does this let GCHQ do?

GCHQ and NSA analysts, who share direct access to the system, are repeatedly told they need a justification to look for information on targets in the system and can't simply go on fishing trips – under the Human Rights Act, searches must be necessary and proportionate. However, when they do search the data, they have lots of specialist tools that let them obtain a huge amount of information from it: details of email addresses, IP addresses, who people communicate with, and what search terms they use.

What's the difference between content and metadata?

The simple analogy for content and metadata is that content is a letter, and metadata is the envelope. However, internet metadata can reveal much more than that: where you are, what you are searching for, who you are messaging and more.

One of the documents seen by the Guardian sets out how GCHQ defines metadata in detail, noting that "we lean on legal and policy interpretations that are not always intuitive". It notes that in an email, the "to", "from" and "cc" fields are metadata, but the subject line is content. The document also sets out how, in some circumstances, even passwords can be regarded as metadata.

The distinction is a very important one to GCHQ with regard to the law, the document explains: "There are extremely stringent legal and policy constraints on what we can do with content, but we are much freer in how we can store metadata. Moreover, there is obviously a much higher volume of content than metadata.

"For these reasons, metadata feeds will usually be unselected – we pull everything we see; on the other hand, we generally only process content that we have a good reason to target."

GCHQThe NSA filesEspionageInternetEwen MacAskillJulian BorgerNick HopkinsNick DaviesJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

The legal loopholes that allow GCHQ to spy on the world

William Hague has hailed GCHQ's 'democratic accountability', but legislation drafted before a huge expansion of internet traffic appears to offer flexibility

William Hague was adamant when he addressed MPs on Monday last week. In an emergency statement (video) forced by the Guardian's disclosures about GCHQ involvement with the Prism programme, the foreign secretary insisted the agency operated within a "strong framework of democratic accountability and oversight".

The laws governing the intelligence agencies provide "the strongest systems of checks and balances for secret intelligence anywhere in the world", he said.

Leaked documents seen by the Guardian give the impression some high-ranking officials at GCHQ have a different view.

In confidential briefings, one of Cheltenham's senior legal advisers, whom the Guardian will not name, made a note to tell his guests: "We have a light oversight regime compared with the US".

The parliamentary intelligence and security committee, which scrutinises the work of the agencies, was sympathetic to the agencies' difficulties, he suggested.

"They have always been exceptionally good at understanding the need to keep our work secret," the legal adviser said.

Complaints against the agencies, undertaken by the interception commissioner, are conducted under "the veil of secrecy". And the investigatory powers tribunal, which assesses complaints against the agencies, has "so far always found in our favour".

The briefings offer important glimpses into the GCHQ's view of itself, the legal framework in which it works, and, it would seem, the necessity for reassuring the UK's most important intelligence partner, the United States, that sensitive information can be shared without raising anxiety in Washington.

None of the documents advocates law-breaking – quite the opposite. But critics will say they highlight the limitations of the three pieces of legislation that underpin the activities of GCHQ, MI5 and MI6 – which were repeatedly mentioned by Hague as pillars of the regulatory and oversight regime during his statement to the Commons.

The foreign secretary said GCHQ "complied fully" with the Regulation of Investigatory Powers Act (Ripa), the Human Rights Act (HRA) and the Intelligence Services Act (Isa).

Privacy campaigners argue the laws have one important thing in common: they were drafted in the last century, and nobody involved in writing them, or passing them, could possibly have envisaged the exponential growth of traffic from telecoms and internet service providers over the past decade.

Nor could they have imagined that GCHQ could have found a way of storing and analysing so much of that information as part of its overarching Mastering the Internet project.

The Tempora programme appears to have given Britain's spymasters that resource, with documents seen by the Guardian showing Britain can retain for up to 30 days an astronomical amount of unfiltered data garnered from cables carrying internet traffic.

This raises a number of questions about the way GCHQ officials and ministers have legitimised the programme.

The briefings, which are entitled UK Operational Legalities, stress that GCHQ "is an organisation with a highly responsible approach to compliance with the law".

GCHQ also has a well staffed legal team, known as OPP-LEG, to help staff navigate their way through the complexities of the law.

But there appears to be some nervousness about Tempora. In a paper written for National Security Agency (NSA) analysts entitled A Guide to Using Internet Buffers at GCHQ, the author notes: "[Tempora] represents an exciting opportunity to get direct access to enormous amounts of GCHQ's special source data.

"As large-scale buffering of metadata and content represent a new concept for GCHQ's exploitation of the internet, GCHQ's legal and policy officers are understandably taking a careful approach to their access and use."

So how did GCHQ secure the legal authority for setting up Tempora, and what safeguards are in place for sharing the intelligence with the Americans? According to the documents, the British government used Ripa to get taps on to the fibre-optic cables.

These cables carry internet traffic in and out of the country and contain details of millions of emails and web searches. The information from these cables went straight into the Tempora storage programme.

In one presentation, which appeared to be for US analysts from the NSA, GCHQ explained: "Direct access to large volumes of unselected SSE data [is] collected under a Ripa warrant."

The precise arrangement between the firms is unclear, as are the legal justifications put before ministers. Isa gives GCHQ some powers for the "passive collection" of data, including from computer networks.

But it appears GCHQ has relied on paragraph four of section 8 of Ripa to gain "external warrants" for its programmes.

They allow the agency to intercept external communications where, for instance, one of the people being targeted is outside Britain.

In most Ripa cases, a minister has to be told the name of an individual or company being targeted before a warrant is granted.

But section 8 permits GCHQ to perform more sweeping and indiscriminate trawls of external data if a minister issues a "certificate" along with the warrant.

According to the documents, the certificate authorises GCHQ to search for material under a number of themes, including: intelligence on the political intentions of foreign governments; military postures of foreign countries; terrorism, international drug trafficking and fraud.

The briefing document says such sweeping certificates, which have to be signed off by a minister, "cover the entire range of GCHQ's intelligence production".

"The certificate is issued with the warrant and signed by the secretary of state and sets out [the] class of work we can do under it … cannot list numbers or individuals as this would be an infinite list which we couldn't manage."

Lawyers at GCHQ speak of having 10 basic certificates, including a "global" one that covers the agency's support station at Bude in Cornwall, Menwith Hill in North Yorkshire, and Cyprus.

Other certificates have been used for "special source accesses" – a reference, perhaps, to the cables carrying web traffic. All certificates have to be renewed by the foreign secretary every six months.

A source with knowledge of intelligence confirmed: "Overall exercise of collection and analysis [is] done under a broad, overall legal authority which has to be renewed at intervals, and is signed off at a senior political level."

The source said the interception commissioner was able to "conclude that [the process] was not appropriate", and that the companies involved were not giving up the information voluntarily.

"We have overriding authority to compel [them] to do this," the source said. "There's an overarching condition of the licensing of the companies that they have to co-operate in this.

"Should they decline, we can compel them to do so. They have no choice. They can't talk about the warrant, they can't reveal the existence of it."

GCHQ says it can also seek a sensitive targeting authority (STA), which allows it snoop on any Briton "anywhere in the world" or any foreign national located in the UK.

It is unclear how the STA system works, and who has authority over it.

The intelligence agencies also have to take note of the HRA, which demands any interception is "necessary and proportionate".

But the documents show GCHQ believes these terms are open to interpretation – which "creates flexibility". When Tempora became fully functional in around 2011, GCHQ gave the NSA access to the programme on a three-month trial – and the NSA was keen to impress.

The US agency sent a briefing to some of its analysts urging them to show they could behave responsibly with the data. Under a heading – "The need to be successful!" – the author wrote: "As the first NSA users to receive operational access [to Tempora], we're depending on you to provide the business case required to justify expanded access. Most importantly we need to prove that NSA users can utilise the internet buffers in ways that are consistent with GCHQ's legal and policy rules.

"In addition, we need to prove that NSA's access … is necessary to prosecute our mission and will greatly enhance the production of the intelligence … success of this three-month trial will determine expanded NSA access to internet buffers in the future."

The NSA appears to have made a successful case. In May last year, an internal GCHQ memo said it had 300 analysts working on intelligence from Tempora, and the NSA had 250. The teams were supporting "the target discovery mission".

But the safeguards for the sharing of this information are unclear.

Though GCHQ says it only keeps the content of messages for three working days, and the metadata for up to 30 days, privacy campaigners here and in the US will want to know if the NSA is adhering to the same self-imposed rules. One concern for privacy campaigners is that GCHQ and the NSA could conduct intercepts for each other, and then offer to share the information – a manoeuvre that could bypass the domestic rules they have to abide by.

This was raised by MPs during last week's statement, with the former Labour home secretary David Blunkett calling for clarification on this potential loophole.

Last week, the Guardian sent a series of questions to the Foreign Office about this issue, but the department said it would not be drawn on it.

"It is a longstanding policy not to comment on intelligence matters; this includes our intelligence co-operation with the United States.

"The intelligence and security committee is looking into this, which is the proper channel for such matters."

GCHQSurveillanceEspionageInternetData and computer securityData protectionEwen MacAskillJulian BorgerNick HopkinsNick DaviesJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

Mastering the internet: how GCHQ set out to spy on the world wide web

Project Tempora – the evolution of a secret programme to capture vast amounts of web and phone data

The memo was finished at 9.32am on Tuesday 19 May 2009, and was written jointly by the director in charge of GCHQ's top-secret Mastering the Internet (MTI) project and a senior member of the agency's cyber-defence team.

The internal email, seen by the Guardian, was a "prioritisation and tasking initiative" to another senior member of staff about the problems facing GCHQ during a period when technology seemed to be racing ahead of the intelligence community.

The authors wanted new ideas – and fast.

"It is becoming increasingly difficult for GCHQ to acquire the rich sources of traffic needed to enable our support to partners within HMG [Her Majesty's government], the armed forces, and overseas," they wrote.

"The rapid development of different technologies, types of traffic, service providers and networks, and the growth in sheer volumes that accompany particularly the expansion and use of the internet, present an unprecedented challenge to the success of GCHQ's mission. Critically we are not currently able to prioritise and task the increasing range and scale of our accesses at the pace, or with the coherence demanded of the internet age: potentially available data is not accessed, potential benefit for HMG is not delivered."

The memo continued: "We would like you to lead a small team to fully define this shortfall in tasking capability [and] identify all the necessary changes needed to rectify it." The two chiefs said they wanted an initial report within a month, and every month thereafter, on what one of them described as "potential quick-win solutions not currently within existing programme plans".

Though this document only offers a snapshot, at the time it was written four years ago some senior officials at GCHQ were clearly anxious about the future, and casting around for ideas among senior colleagues about how to address a variety of problems.

According to the papers leaked by the US National Security Agency (NSA) whistleblower Edward Snowden, Cheltenham's overarching project to "master the internet" was under way, but one of its core programmes, Tempora, was still being tested and developed, and the agency's principal customers, the government, MI5 and MI6, remained hungry for more and better-quality information.

There was America's NSA to consider too. The Americans had been pushing GCHQ to provide better intelligence to support Nato's military effort in Afghanistan; a reflection, perhaps, of wider US frustration that information sharing between the US and the UK had become too lopsided over the past 20 years.

In the joint instruction from 2009, the director had twice mentioned the necessity to fulfil GCHQ's "mission", but the academics and commentators who follow Britain's intelligence agencies are unsure exactly what this means, and politicians rarely try to define it in any detail.

The "mission" has certainly changed and the agency, currently run by Sir Iain Lobban, may be under more pressure now than it has ever been.

The issues, and the enemies, have become more complex, and are quite different from the comparatively simple world into which Britain's secret services were born in 1909.

At the time, concern about German spies living in the UK led to the establishment of a Secret Service Bureau and, at the start of the first world war, two embryonic security organisations began to focus on "signals intelligence" (Sigint), which remains at heart of GCHQ's work.

The codebreakers of Bletchley Park became heroes of the second world war as they mastered the encryption systems used by the Nazis. And the priority during the cold war was Moscow.

During these periods GCHQ's focus was clear, and the priorities of the "mission" easier to establish.

There was no parliamentary scrutiny of its work so the agency, which moved from Milton Keynes to Cheltenham in the early 1950s, existed in a peculiar limbo.

That changed, and with it the boundaries of its work, with the 1994 Intelligence Services Act (Isa), which gave a legal underpinning to the agency for the first time. The act kept the powers and objectives of GCHQ broad and vague.

The agency was tasked with working "in the interests of national security, with particular reference to the defence and foreign policies of Her Majesty's government; in the interests of the economic wellbeing of the United Kingdom; and in support of the prevention and the detection of serious crime".

Reviewing the legislation at the time, the human rights lawyer John Wadham, then legal director of Liberty, highlighted the ambiguities of the expressions used, and warned that the lack of clarity would cause problems and concern.

"National security is used without further definition. It is true the courts themselves have found it impossible to decide what is or what is not in the interests of national security. The reality is that 'national security' can mean whatever the government of the day chooses it to mean." The same could be said for the clause referring to "economic wellbeing".

Arguably, GCHQ's responsibilities have broadened even further over the past decade: it has become the UK's lead agency for cyber-security – identifying the hackers, criminal gangs and state actors who are stealing ideas, information and blueprints from British firms.

Alarmed by the increase in these cyber-attacks, and faced with billions of pounds' worth of intellectual property being stolen every year, the government made the issue a tier-one priority in the 2010 strategic defence and security review. In a time of cuts across Whitehall, the coalition found an extra £650m for cyber-security initiatives, and more than half was given to GCHQ. It has left the agency with a vast array of responsibilities, which were set out in a pithy internal GCHQ memo dated October 2011: "[Our] targets boil down to diplomatic/military/commercial targets/terrorists/organised criminals and e-crime/cyber actors".

All this has taken place during an era in which it has become harder, the intelligence community claims, for analysts to access the information they believe they need. The exponential growth in the number of mobile phone users during the noughties, and the rise of a new breed of independent-minded internet service providers, conspired to make their work more difficult, particularly as many of the new firms were based abroad, outside the jurisdiction of British law.

Struggling to cope with increased demands, a more complex environment, and working within laws that critics say are hopelessly outdated, GCHQ starting casting around for new, innovative ideas. Tempora was one of them.

Though the documents are not explicit, it seems the Mastering the Internet programme began life in early 2007 and, a year later, work began on an experimental research project, run out of GCHQ's outpost at Bude in Cornwall.

Its aim was to establish the practical uses of an "internet buffer", the first of which was referred to as CPC, or Cheltenham Processing Centre.

By March 2010, analysts from the NSA had been allowed some preliminary access to the project, which, at the time, appears to have been codenamed TINT, and was being referred to in official documents as a "joint GCHQ/NSA research initiative".

TINT, the documents explain, "uniquely allows retrospective analysis for attribution" – a storage system of sorts, which allowed analysts to capture traffic on the internet and then review it.

The papers seen by the Guardian make clear that at some point – it is not clear when – GCHQ began to plug into the cables that carry internet traffic into and out of the country, and garner material in a process repeatedly referred to as SSE. This is thought to mean special source exploitation.

The capability, which was authorised by legal warrants, gave GCHQ access to a vast amount of raw information, and the TINT programme a potential way of being able to store it.

A year after the plaintive email asking for new ideas, GCHQ reported significant progress on a number of fronts.

One document described how there were 2 billion users of the internet worldwide, how Facebook had more than 400 million regular users and how there had been a 600% growth in mobile internet traffic the year before. "But we are starting to 'master' the internet," the author claimed. "And our current capability is quite impressive."

The report said the UK now had the "biggest internet access in Five Eyes" – the group of intelligence organisations from the US, UK, Canada, New Zealand and Australia. "We are in the golden age," the report added.

There were caveats. The paper warned that American internet service providers were moving to Malaysia and India, and the NSA was "buying up real estate in these places".

"We won't see this traffic crossing the UK. Oh dear," the author said. He suggested Britain should do the same and play the "US at [their] own game … and buy facilities overseas".

GCHQ's mid-year 2010-11 review revealed another startling fact about Mastering the Internet.

"MTI delivered the next big step in the access, processing and storage journey, hitting a new high of more than 39bn events in a 24-hour period, dramatically increasing our capability to produce unique intelligence from our targets' use of the internet and made major contributions to recent operations."

This appears to suggest GCHQ had managed to record 39bn separate pieces of information during a single day. The report noted there had been "encouraging innovation across all of GCHQ".

The NSA remarked on the success of GCHQ in a "Joint Collaboration Activity" report in February 2011. In a startling admission, it said Cheltenham now "produces larger amounts of metadata collection than the NSA", metadata being the bare details of calls made and messages sent rather than the content within them.

The close working relationship between the two agencies was underlined later in the document, with a suggestion that this was a necessity to process such a vast amount of raw information.

"GCHQ analysts effectively exploit NSA metadata for intelligence production, target development/discovery purposes," the report explained.

"NSA analysts effectively exploit GCHQ metadata for intelligence production, target development/discovery purposes. GCHQ and NSA avoid processing the same data twice and proactively seek to converge technical solutions and processing architectures."

The documents appear to suggest the two agencies had come to rely on each other; with Tempora's "buffering capability", and Britain's access to the cables that carry internet traffic in and out of the country, GCHQ has been able to collect and store a huge amount of information.

The NSA, however, had provided GCHQ with the tools necessary to sift through the data and get value from it.

By May last year, the volume of information available to them grew again, with GCHQ reporting that it now had "internet buffering" capability running from its headquarters in Cheltenham, its station in Bude, and a location abroad, which the Guardian will not identify. The programme was now capable of collecting, a memo explained with excited understatement, "a lot of data!"

Referring to Tempora's "deep dive capability", it explained: "It builds upon the success of the TINT experiment and will provide a vital unique capability.

"This gives over 300 GCHQ and 250 NSA analysts access to huge amounts of data to support the target discovery mission. The MTI programme would like to say a big thanks to everyone who has made this possible … a true collaborative effort!"

Tempora, the document said, had shown that "every area of ops can get real benefit from this capability, especially for target discovery and target development".

But while the ingenuity of the Tempora programme is not in doubt, its existence may trouble anyone who sends and receives an email, or makes an internet phone call, or posts a message on a social media site, and expects the communication to remain private.

Campaigners and human rights lawyers will doubtless want to know how Britain's laws have been applied to allow this vast collection of data. They will ask questions about the oversight of the programme by ministers, MPs and the intelligence interception commissioner, none of whom have spoken in public about it.

GCHQThe NSA filesEspionageSurveillanceInternetData and computer securityData protectionEwen MacAskillJulian BorgerNick HopkinsNick DaviesJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

GCHQ taps fibre-optic cables for secret access to world's communications

Exclusive: British spy agency collects and stores vast quantities of global email messages, Facebook posts, internet histories and calls, and shares them with NSA, latest documents from Edward Snowden reveal

Britain's spy agency GCHQ has secretly gained access to the network of cables which carry the world's phone calls and internet traffic and has started to process vast streams of sensitive personal information which it is sharing with its American partner, the National Security Agency (NSA).

The sheer scale of the agency's ambition is reflected in the titles of its two principal components: Mastering the Internet and Global Telecoms Exploitation, aimed at scooping up as much online and telephone traffic as possible. This is all being carried out without any form of public acknowledgement or debate.

One key innovation has been GCHQ's ability to tap into and store huge volumes of data drawn from fibre-optic cables for up to 30 days so that it can be sifted and analysed. That operation, codenamed Tempora, has been running for some 18 months.

GCHQ and the NSA are consequently able to access and process vast quantities of communications between entirely innocent people, as well as targeted suspects.

This includes recordings of phone calls, the content of email messages, entries on Facebook and the history of any internet user's access to websites – all of which is deemed legal, even though the warrant system was supposed to limit interception to a specified range of targets.

The existence of the programme has been disclosed in documents shown to the Guardian by the NSA whistleblower Edward Snowden as part of his attempt to expose what he has called "the largest programme of suspicionless surveillance in human history".

"It's not just a US problem. The UK has a huge dog in this fight," Snowden told the Guardian. "They [GCHQ] are worse than the US."

However, on Friday a source with knowledge of intelligence argued that the data was collected legally under a system of safeguards, and had provided material that had led to significant breakthroughs in detecting and preventing serious crime.

Britain's technical capacity to tap into the cables that carry the world's communications – referred to in the documents as special source exploitation – has made GCHQ an intelligence superpower.

By 2010, two years after the project was first trialled, it was able to boast it had the "biggest internet access" of any member of the Five Eyes electronic eavesdropping alliance, comprising the US, UK, Canada, Australia and New Zealand.

UK officials could also claim GCHQ "produces larger amounts of metadata than NSA". (Metadata describes basic information on who has been contacting whom, without detailing the content.)

By May last year 300 analysts from GCHQ, and 250 from the NSA, had been assigned to sift through the flood of data.

The Americans were given guidelines for its use, but were told in legal briefings by GCHQ lawyers: "We have a light oversight regime compared with the US".

When it came to judging the necessity and proportionality of what they were allowed to look for, would-be American users were told it was "your call".

The Guardian understands that a total of 850,000 NSA employees and US private contractors with top secret clearance had access to GCHQ databases.

The documents reveal that by last year GCHQ was handling 600m "telephone events" each day, had tapped more than 200 fibre-optic cables and was able to process data from at least 46 of them at a time.

Each of the cables carries data at a rate of 10 gigabits per second, so the tapped cables had the capacity, in theory, to deliver more than 21 petabytes a day – equivalent to sending all the information in all the books in the British Library 192 times every 24 hours.

And the scale of the programme is constantly increasing as more cables are tapped and GCHQ data storage facilities in the UK and abroad are expanded with the aim of processing terabits (thousands of gigabits) of data at a time.

For the 2 billion users of the world wide web, Tempora represents a window on to their everyday lives, sucking up every form of communication from the fibre-optic cables that ring the world.

The NSA has meanwhile opened a second window, in the form of the Prism operation, revealed earlier this month by the Guardian, from which it secured access to the internal systems of global companies that service the internet.

The GCHQ mass tapping operation has been built up over five years by attaching intercept probes to transatlantic fibre-optic cables where they land on British shores carrying data to western Europe from telephone exchanges and internet servers in north America.

This was done under secret agreements with commercial companies, described in one document as "intercept partners".

The papers seen by the Guardian suggest some companies have been paid for the cost of their co-operation and GCHQ went to great lengths to keep their names secret. They were assigned "sensitive relationship teams" and staff were urged in one internal guidance paper to disguise the origin of "special source" material in their reports for fear that the role of the companies as intercept partners would cause "high-level political fallout".

The source with knowledge of intelligence said on Friday the companies were obliged to co-operate in this operation. They are forbidden from revealing the existence of warrants compelling them to allow GCHQ access to the cables.

"There's an overarching condition of the licensing of the companies that they have to co-operate in this. Should they decline, we can compel them to do so. They have no choice."

The source said that although GCHQ was collecting a "vast haystack of data" what they were looking for was "needles".

"Essentially, we have a process that allows us to select a small number of needles in a haystack. We are not looking at every piece of straw. There are certain triggers that allow you to discard or not examine a lot of data so you are just looking at needles. If you had the impression we are reading millions of emails, we are not. There is no intention in this whole programme to use it for looking at UK domestic traffic – British people talking to each other," the source said.

He explained that when such "needles" were found a log was made and the interception commissioner could see that log.

"The criteria are security, terror, organised crime. And economic well-being. There's an auditing process to go back through the logs and see if it was justified or not. The vast majority of the data is discarded without being looked at … we simply don't have the resources."

However, the legitimacy of the operation is in doubt. According to GCHQ's legal advice, it was given the go-ahead by applying old law to new technology. The 2000 Regulation of Investigatory Powers Act (Ripa) requires the tapping of defined targets to be authorised by a warrant signed by the home secretary or foreign secretary.

However, an obscure clause allows the foreign secretary to sign a certificate for the interception of broad categories of material, as long as one end of the monitored communications is abroad. But the nature of modern fibre-optic communications means that a proportion of internal UK traffic is relayed abroad and then returns through the cables.

Parliament passed the Ripa law to allow GCHQ to trawl for information, but it did so 13 years ago with no inkling of the scale on which GCHQ would attempt to exploit the certificates, enabling it to gather and process data regardless of whether it belongs to identified targets.

The categories of material have included fraud, drug trafficking and terrorism, but the criteria at any one time are secret and are not subject to any public debate. GCHQ's compliance with the certificates is audited by the agency itself, but the results of those audits are also secret.

An indication of how broad the dragnet can be was laid bare in advice from GCHQ's lawyers, who said it would be impossible to list the total number of people targeted because "this would be an infinite list which we couldn't manage".

There is an investigatory powers tribunal to look into complaints that the data gathered by GCHQ has been improperly used, but the agency reassured NSA analysts in the early days of the programme, in 2009: "So far they have always found in our favour".

Historically, the spy agencies have intercepted international communications by focusing on microwave towers and satellites. The NSA's intercept station at Menwith Hill in North Yorkshire played a leading role in this. One internal document quotes the head of the NSA, Lieutenant General Keith Alexander, on a visit to Menwith Hill in June 2008, asking: "Why can't we collect all the signals all the time? Sounds like a good summer project for Menwith."

By then, however, satellite interception accounted for only a small part of the network traffic. Most of it now travels on fibre-optic cables, and the UK's position on the western edge of Europe gave it natural access to cables emerging from the Atlantic.

The data collected provides a powerful tool in the hands of the security agencies, enabling them to sift for evidence of serious crime. According to the source, it has allowed them to discover new techniques used by terrorists to avoid security checks and to identify terrorists planning atrocities. It has also been used against child exploitation networks and in the field of cyberdefence.

It was claimed on Friday that it directly led to the arrest and imprisonment of a cell in the Midlands who were planning co-ordinated attacks; to the arrest of five Luton-based individuals preparing acts of terror, and to the arrest of three London-based people planning attacks prior to the Olympics.

As the probes began to generate data, GCHQ set up a three-year trial at the GCHQ station in Bude, Cornwall. By the summer of 2011, GCHQ had probes attached to more than 200 internet links, each carrying data at 10 gigabits a second. "This is a massive amount of data!" as one internal slideshow put it. That summer, it brought NSA analysts into the Bude trials. In the autumn of 2011, it launched Tempora as a mainstream programme, shared with the Americans.

The intercept probes on the transatlantic cables gave GCHQ access to its special source exploitation. Tempora allowed the agency to set up internet buffers so it could not simply watch the data live but also store it – for three days in the case of content and 30 days for metadata.

"Internet buffers represent an exciting opportunity to get direct access to enormous amounts of GCHQ's special source data," one document explained.

The processing centres apply a series of sophisticated computer programmes in order to filter the material through what is known as MVR – massive volume reduction. The first filter immediately rejects high-volume, low-value traffic, such as peer-to-peer downloads, which reduces the volume by about 30%. Others pull out packets of information relating to "selectors" – search terms including subjects, phone numbers and email addresses of interest. Some 40,000 of these were chosen by GCHQ and 31,000 by the NSA. Most of the information extracted is "content", such as recordings of phone calls or the substance of email messages. The rest is metadata.

The GCHQ documents that the Guardian has seen illustrate a constant effort to build up storage capacity at the stations at Cheltenham, Bude and at one overseas location, as well a search for ways to maintain the agency's comparative advantage as the world's leading communications companies increasingly route their cables through Asia to cut costs. Meanwhile, technical work is ongoing to expand GCHQ's capacity to ingest data from new super cables carrying data at 100 gigabits a second. As one training slide told new users: "You are in an enviable position – have fun and make the most of it."

GCHQThe NSA filesEspionageInternetPrivacyData and computer securityData protectionEwen MacAskillJulian BorgerNick HopkinsNick DaviesJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

The Guardian Audio Edition: 18 June 2013

Audio versions of a selection of a selection of articles from the Guardian newspaper and website

Reading this on a mobile? Click here to listen

In this week's edition:

• Exclusive: phones were monitored and fake internet cafes set up to gather information from allies at the G20 summit in London in 2009. By Ewen MacAskill, Nick Davies, Nick Hopkins, Julian Borger and James Ball. Click here to read article

• Forget Field Marshal Twitter. What scares rulers like Turkey's Recep Tayyip Erdoğan are the street's wild squadrons, says Simon Jenkins. Click here to read article

• Channel 4's Jon Snow, the only UK journalist covering election from Tehran, found people desperate for an end to their isolation. Click here to read article

• In this age of information, is it possible to make yourself unGoogleable? And does it earn you added credibility, as fashion designer Phoebe Philo and bands such as !!! suggest? By Stuart Jeffries. Click here to read article

• Superheroes: what started out as cheap, populist entertainment in comic book form has turned into an endless parade of sexist, semi-fascist bores. By Joe Queenan. Click here to read article

In this week's audiobook review, we look back over almost 20 years of bad behaviour with the latest outing for Francesca Simon's Horrid Henry, and look ahead with the launch of a new series from Eoin Colfer, WARP.

The Guardian Audio Edition is supported by Audible.co.uk. To listen to the audiobooks reviewed in this week's edition go to audible.co.uk/guardian audio.

Francesca SimonEoin ColferEwen MacAskillNick DaviesNick HopkinsJulian BorgerJames BallJon SnowSimon JenkinsStuart JeffriesJoe Queenan
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

GCHQ intercepted foreign politicians' communications at G20 summits

Exclusive: phones were monitored and fake internet cafes set up to gather information from allies in London in 2009

Foreign politicians and officials who took part in two G20 summit meetings in London in 2009 had their computers monitored and their phone calls intercepted on the instructions of their British government hosts, according to documents seen by the Guardian. Some delegates were tricked into using internet cafes which had been set up by British intelligence agencies to read their email traffic.

The revelation comes as Britain prepares to host another summit on Monday – for the G8 nations, all of whom attended the 2009 meetings which were the object of the systematic spying. It is likely to lead to some tension among visiting delegates who will want the prime minister to explain whether they were targets in 2009 and whether the exercise is to be repeated this week.

The disclosure raises new questions about the boundaries of surveillance by GCHQ and its American sister organisation, the National Security Agency, whose access to phone records and internet data has been defended as necessary in the fight against terrorism and serious crime. The G20 spying appears to have been organised for the more mundane purpose of securing an advantage in meetings. Named targets include long-standing allies such as South Africa and Turkey.

There have often been rumours of this kind of espionage at international conferences, but it is highly unusual for hard evidence to confirm it and spell out the detail. The evidence is contained in documents – classified as top secret – which were uncovered by the NSA whistleblower Edward Snowden and seen by the Guardian. They reveal that during G20 meetings in April and September 2009 GCHQ used what one document calls "ground-breaking intelligence capabilities" to intercept the communications of visiting delegations.

This included:

• Setting up internet cafes where they used an email interception programme and key-logging software to spy on delegates' use of computers;

• Penetrating the security on delegates' BlackBerrys to monitor their email messages and phone calls;

• Supplying 45 analysts with a live round-the-clock summary of who was phoning who at the summit;

• Targeting the Turkish finance minister and possibly 15 others in his party;

• Receiving reports from an NSA attempt to eavesdrop on the Russian leader, Dmitry Medvedev, as his phone calls passed through satellite links to Moscow.

The documents suggest that the operation was sanctioned in principle at a senior level in the government of the then prime minister, Gordon Brown, and that intelligence, including briefings for visiting delegates, was passed to British ministers.

A briefing paper dated 20 January 2009 records advice given by GCHQ officials to their director, Sir Iain Lobban, who was planning to meet the then foreign secretary, David Miliband. The officials summarised Brown's aims for the meeting of G20 heads of state due to begin on 2 April, which was attempting to deal with the economic aftermath of the 2008 banking crisis. The briefing paper added: "The GCHQ intent is to ensure that intelligence relevant to HMG's desired outcomes for its presidency of the G20 reaches customers at the right time and in a form which allows them to make full use of it." Two documents explicitly refer to the intelligence product being passed to "ministers".

According to the material seen by the Guardian, GCHQ generated this product by attacking both the computers and the telephones of delegates.

One document refers to a tactic which was "used a lot in recent UK conference, eg G20". The tactic, which is identified by an internal codeword which the Guardian is not revealing, is defined in an internal glossary as "active collection against an email account that acquires mail messages without removing them from the remote server". A PowerPoint slide explains that this means "reading people's email before/as they do".

The same document also refers to GCHQ, MI6 and others setting up internet cafes which "were able to extract key logging info, providing creds for delegates, meaning we have sustained intelligence options against them even after conference has finished". This appears to be a reference to acquiring delegates' online login details.

Another document summarises a sustained campaign to penetrate South African computers, recording that they gained access to the network of their foreign ministry, "investigated phone lines used by High Commission in London" and "retrieved documents including briefings for South African delegates to G20 and G8 meetings". (South Africa is a member of the G20 group and has observer status at G8 meetings.)

A detailed report records the efforts of the NSA's intercept specialists at Menwith Hill in North Yorkshire to target and decode encrypted phone calls from London to Moscow which were made by the Russian president, Dmitry Medvedev, and other Russian delegates.

Other documents record apparently successful efforts to penetrate the security of BlackBerry smartphones: "New converged events capabilities against BlackBerry provided advance copies of G20 briefings to ministers … Diplomatic targets from all nations have an MO of using smartphones. Exploited this use at the G20 meetings last year."

The operation appears to have run for at least six months. One document records that in March 2009 – the month before the heads of state meeting – GCHQ was working on an official requirement to "deliver a live dynamically updating graph of telephony call records for target G20 delegates … and continuing until G20 (2 April)."

Another document records that when G20 finance ministers met in London in September, GCHQ again took advantage of the occasion to spy on delegates, identifying the Turkish finance minister, Mehmet Simsek, as a target and listing 15 other junior ministers and officials in his delegation as "possible targets". As with the other G20 spying, there is no suggestion that Simsek and his party were involved in any kind of criminal offence. The document explicitly records a political objective – "to establish Turkey's position on agreements from the April London summit" and their "willingness (or not) to co-operate with the rest of the G20 nations".

The September meeting of finance ministers was also the subject of a new technique to provide a live report on any telephone call made by delegates and to display all of the activity on a graphic which was projected on to the 15-sq-metre video wall of GCHQ's operations centre as well as on to the screens of 45 specialist analysts who were monitoring the delegates.

"For the first time, analysts had a live picture of who was talking to who that updated constantly and automatically," according to an internal review.

A second review implies that the analysts' findings were being relayed rapidly to British representatives in the G20 meetings, a negotiating advantage of which their allies and opposite numbers may not have been aware: "In a live situation such as this, intelligence received may be used to influence events on the ground taking place just minutes or hours later. This means that it is not sufficient to mine call records afterwards – real-time tip-off is essential."

In the week after the September meeting, a group of analysts sent an internal message to the GCHQ section which had organised this live monitoring: "Thank you very much for getting the application ready for the G20 finance meeting last weekend … The call records activity pilot was very successful and was well received as a current indicator of delegate activity …

"It proved useful to note which nation delegation was active during the moments before, during and after the summit. All in all, a very successful weekend with the delegation telephony plot."

GCHQSurveillanceEspionageG20Gordon BrownEwen MacAskillNick DaviesNick HopkinsJulian BorgerJames Ball
theguardian.com © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

G20 summit: NSA targeted Russian president Medvedev in London

Leaked documents reveal Russian president was spied on during visit, as questions are raised over use of US base in Britain

American spies based in the UK intercepted the top-secret communications of the then Russian president, Dmitry Medvedev, during his visit to Britain for the G20 summit in London, leaked documents reveal.

The details of the intercept were set out in a briefing prepared by the National Security Agency (NSA), America's biggest surveillance and eavesdropping organisation, and shared with high-ranking officials from Britain, Australia, Canada and New Zealand.

The document, leaked by the NSA whistleblower Edward Snowden and seen by the Guardian, shows the agency believed it might have discovered "a change in the way Russian leadership signals have been normally transmitted".

The disclosure underlines the importance of the US spy hub at RAF Menwith Hill in Harrogate, North Yorkshire, where hundreds of NSA analysts are based, working alongside liaison officers from GCHQ.

The document was drafted in August 2009, four months after the visit by Medvedev, who joined other world leaders in London, including the US president, Barack Obama, for the event hosted by the British prime minister, Gordon Brown.

Medvedev arrived in London on Wednesday 1 April and the NSA intercepted communications from his delegation the same day, according to the NSA paper, entitled: "Russian Leadership Communications in support of President Dmitry Medvedev at the G20 summit in London – Intercept at Menwith Hill station."

The document starts with two pictures of Medvedev smiling for the world's media alongside Brown and Obama in bilateral discussions before the main summit.

The report says: "This is an analysis of signal activity in support of President Dmitry Medvedev's visit to London. The report details a change in the way Russian leadership signals have been normally transmitted. The signal activity was found to be emanating from the Russian embassy in London and the communications are believed to be in support of the Russian president."

The NSA interception of the Russian leadership at G20 came hours after Obama and Medvedev had met for the first time. Relations between the two leaders had been smoothed in the runup to the summit with a series of phone calls and letters, with both men wanting to establish a trusting relationship to discuss the ongoing banking crisis and nuclear disarmament.

In the aftermath of their discussions on 1 April, the two men issued a joint communique saying they intended to "move further along the path of reducing and limiting strategic offensive arms in accordance with the treaty on the non-proliferation of nuclear weapons".

A White House official who briefed journalists described the meeting as "a very successful first meeting focused on real issues". The official said it had been important for the men to be open about the issues on which they agreed and disagreed. Obama had stressed the need to be candid, the official noted.

While it has been widely known the two countries spy on each other, it is rare for either to be caught in the act; the latest disclosures will also be deeply embarrassing for the White House as Obama prepares to meet Vladimir Putin, who succeeded Medvedev as president, in the margins of the G8 summit this week.

The two countries have long complained about the extent of each other's espionage activities, and tit-for-tat expulsions of diplomats are common. A year after Obama met Medvedev, the US claimed it had broken a highly sophisticated spy ring that carried out "deep cover" assignments in the US.

Ten alleged Russian spies living in America were arrested.

Putin was withering of the FBI-led operation: "I see that your police have let themselves go and put some people in jail, but I guess that is their job. I hope the positive trend that we have seen develop in our bilateral relations recently will not be harmed by these events." Last month, the Russians arrested an American in Moscow who they alleged was a CIA agent.

The new revelations underline the significance of RAF Menwith Hill and raise questions about its relationship to the British intelligence agencies, and who is responsible for overseeing it. The 560-acre site was leased to the Americans in 1954 and the NSA has had a large presence there since 1966.

It has often been described as the biggest surveillance and interception facility in the world, and has 33 distinct white "radomes" that house satellite dishes. A US base in all but name, it has British intelligence analysts seconded to work alongside NSA colleagues, though it is unclear how the two agencies obtain and share intelligence – and under whose legal authority they are working under.

Dmitry MedvedevSurveillanceRussiaUS foreign policyUnited StatesG20Ewen MacAskillNick DaviesNick HopkinsJulian BorgerJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds

UK intelligence agencies planned to spy on Commonwealth summit delegates

Top-secret document, prepared by GCHQ, contained proposals to target Commonwealth allies at heads of government summit

UK intelligence agencies planned to spy on delegates to the Commonwealth heads of government meeting in 2009, including being asked to obtain information to give UK ministers an advantage in talks with their Commonwealth counterparts, according to a top-secret document seen by the Guardian.

The meeting, which takes place every two years, was held in Trinidad in 2009. The UK delegation was headed by the Queen, with Prince Philip also in attendance, along with Gordon Brown, the then prime minister, David Miliband, then foreign secretary, and Douglas Alexander, then international development secretary.

A page from an internal top-secret intranet of GCHQ, shared with the NSA, discovered by the 29-year-old whistleblower Edward Snowden and seen by the Guardian, shows a list of "key intelligence requirements" set out for the summit.

Alongside notes to check for threats against the security of the UK delegation during the visit, the document lists "Intelligence to inform UK senior's [sic] Bi-lats", "Initelligence [sic] on South Africa's views on Zimbabwe prior to Brown/Zuma meeting" and "climate change reporting".

The revelation that UK intelligence agencies made plans to target ministers and officials from Commonwealth countries, as well as the targeting of G20 officials disclosed elsewhere, is likely to raise tensions among the Commonwealth nations, who may seek clarity over whether their officials were bugged, and if so to what extent.

The note, which was prepared in advance of the meeting, also sets out a schedule for different UK agencies to set up their activities in Trinidad. MI6 were tasked to set up several days before the event, with GCHQ's operation beginning with the arrival of delegates. The Guardian is not publishing the original document as it contains logistical details and some limited references to personnel.

The 2009 Commonwealth meeting, which was also attended by Nicolas Sarkozy, then president of France, appears to have been the first time MI6 – formally known as SIS, or the Secret Intelligence Service – had been asked to gather intelligence from a Commonwealth heads of government gathering.

"SIS have no past history of targeting this meeting," the document notes in an explanation of why operations might be limited in their scope.

As it was prepared in advance of the Commonwealth meeting, the memo does not confirm to what extent surveillance was carried out, or even whether planned operations actually took place.

However, it does stress to agency staff that "we will be measured on our ability to deliver".

The memo also shows that the agencies were preparing to brief senior ministers, and the prime minister, during the conference.

The memo noted that Lady Kinnock was available for briefings from 25 to 29 November, David Miliband could be briefed from 26 to 29 November, and Gordon Brown on 29 November only.

There is no indication as to whether the briefings actually took place, or whether the ministers were aware of the security services' plans for the summit.

Commonwealth summitGCHQSurveillanceG20Ewen MacAskillNick DaviesNick HopkinsJulian BorgerJames Ball
guardian.co.uk © 2013 Guardian News and Media Limited or its affiliated companies. All rights reserved. | Use of this content is subject to our Terms & Conditions | More Feeds