Taxpayer-funded grants to NZ tech firm used to build surveillance equipment for UK government

Taxpayer-funded grants to a private New Zealand tech firm were used to build mass surveillance equipment for Britain's largest intelligence agency, the Government Communications Headquarters (GCHQ).

Nicky Hager Source: 1 NEWS

These revelations flow from a cache of leaked documents passed on to The Intercept news site and now reported on in co-operation with 1 NEWS.

Auckland-based Endace Limited received $11.1 million in two research and development grants from the New Zealand Government in 2010. 

With a logo featuring an eye, Endace’s motto is “Power to see all”.  The company builds network monitoring equipment and says that its commercial technology is used by customers worldwide to protect their organisations and customer information from cyber threats and breaches.

The leaked documents show Endace used some of the Government grants to build powerful Internet “data capture” systems  for GCHQ in 2010-12, exactly as the British agency expanded its surveillance of European, trans-Atlantic, African and Asian undersea cables and Internet trunk lines.

The government funding was used, in part, for the development of a system called Medusa.  Medusa utilised high-capacity surveillance technology capable of tapping into huge flows of emails, instant messages, social media interactions, Web browsing histories and other data as they are being transmitted across the Internet.

Millions of people's communications, likely including those of many New Zealanders, will have been intercepted using the equipment supplied by the Auckland firm.

The documents show Endace has also sold equipment to controversial Israeli and Moroccan agencies, each with a long record of human rights abuses. Its wider client list has included the US, Australian, Canadian, Spanish, Danish and Indian governments; and some of the world's largest Internet and phone companies enabling them to comply with laws requiring that their networks be “interception capable” for police and intelligence agencies.

Friendly Government Agency

The documents show that Endace reprioritised its research and engineering schedule in 2010-2012 to produce new interception equipment “developed primarily at the behest of the lead customer FGA”.

FGA is the company's internal codename for GCHQ. It stands for Friendly Government Agency.

However in the leaked company documents the staff slip between talking about “FGA” and “GCHQ” for the same equipment orders, leaving no doubt that their lead customer for the surveillance systems is the British spies.

Britain Begins Mass Internet Monitoring

The dramatic expansion of British mass surveillance was documented in previously published GCHQ documents leaked by US whistleblower Edward Snowden. GCHQ staff wrote in 2010 that “25 percent of all Internet traffic” was carried by long-distance cables that cross UK territory. The agency was aiming to create what it described as “the world's biggest” surveillance system. GCHQ's undersea cable interception was underway in 2009, with the monitored traffic measured in tens of Gigabits per second (10Gbps).

The British spies were, by 2009, intercepting and rerouting 87 lots of 10Gbps “to GCHQ processing systems” and they aimed to increase this to 415 x 10Gbps by March 2011. Their longer term goal was to “grow our Internet access to 800 10Gs (sic).”

The scale of GCHQ's plans were massive. A single 10Gbps capacity Internet cable can process the equivalent in data of about 1 million average-sized emails every minute; 800 10Gbps cables could process 13 million emails every second, or 780 million a minute.

But GCHQ needed new technology to fulfil its goals – and that's where Endace had a part to play. As the British agency was working to beef up its surveillance capabilities, Endace was providing it with a host of specialised data “capture servers”.

Details about the sales have not been made public before; they are contained in company documents obtained by The Intercept.

In March 2011, an Endace memo about  GCHQ “requirements” said that “FGA have an initial order for 20 systems for delivery in March [2011].” Each system was built for “FGA only” and had two data capture cards, each intercepting 10Gbps of Internet traffic.

Endace added in the document that it was anticipating “an order of 30-40 additional systems” soon after; and indeed the following month a purchase order for 27 more capture servers arrived. It specified delivery to Loading Bay, Hubble Road, Cheltenham, home of  GCHQ, and states “a potential for 300-500 systems over the next two to three years is being discussed”.

The company documents are confirmed by internal GCHQ records, obtained by Edward Snowden and published by The Intercept, that show Ninja systems and Endace “DAG” capture devices being used as part of British undersea cable and Internet interception. GCHQ documents from 2010 and 2011 repeatedly mention the Endace products while discussing the capture and processing of “Internet-derived” and mobile phone data to extract people's Gmail, Hotmail, WhatsApp and Facebook information.


Later in 2011, Endace was developing a new product for GCHQ called Medusa: interception equipment that could now capture Internet traffic at 100 Gigabits per second. It represented a massive increase in surveillance capability.

Medusa was for “customer UK Government” and was included in the Endace sales systems in September 2011 “so FGA can order the prototype”.

Endace staff produced “Medusa Weekly Status Reports” about their progress towards “delivery of 100Gb rx data monitoring device”. They were “keeping FGA updated on the fortnightly review meetings”.

The 18 November 2011 Medusa status report said “FGA are very pleased with the prototypes we delivered last week.”

Endace then moved straight into “Stage Two” to enhance the surveillance capabilities of Medusa. One of the refinements GCHQ wanted was called “Separate MAC insertion by IP type”. This suggests the British agency may have wanted the ability to target individuals by searching Internet traffic for the built-in hardware address (MAC) of their computers, routers, or phones. The Endace staff wrote that this was “a committed feature for GCHQ”.

Medusa funded using NZ public funds

The Medusa status reports reveal that Endace was using New Zealand government research funding to develop the new equipment for GCHQ. They state that the Medusa system was “being built under the FoRST-funded service for FGA”.

FoRST was the Foundation of Research Science and Technology, the body that handed out New Zealand government research grants. This role has since been taken over by Callaghan Innovation.  Endace received a product development grant of $4.4 million in July 2010 and a further $6.7 million that December.

The public announcement for the July grant said the funding was for “50% of the cost of a series of substantial product developments over the next two years”, but did not say what the products were nor who they were for. A considerable part of this was used on projects for GCHQ.

Callaghan Innovation interim chief executive, Hemi Rolleston, said Endace’s application provided details and technical specifications of the “priority products” they were developing, including Medusa.

“There was no information in the application linking a specific product to a specific client, although Endace advised they had longstanding relationships with many financial services and government agencies in New Zealand, the US and Europe.  This is not at all unusual as by definition, a product undergoing R&D is not yet complete, so it’s unusual to have already made a confirmed sale.”

Entrepreneur of the Year

By December 2011 Endace's share price had risen by 50 percent in 12 months and company chair Ian Graham was named Entrepreneur of the Year at the New Zealand Engineering Excellence awards.  We contacted Endace’s founder, Professor Ian Graham, but he did not want to comment for this article.

Meanwhile, Endace was continuing to focus its resources on developing the new Medusa surveillance technology for GCHQ. A February 2012 status report noted that the  system was still being built “within the FoRST budget”. Many staff had been assigned to “deliver[ing] all of the functionality required by GCHQ,” the report said.

On 17 September 2012, emails between Endace staff discussed an order of “80 units” for “FGA” of the high-capacity data capture cards used to power systems like Medusa. This order on its own would have provided equivalent capacity to all the undersea cable interception GCHQ was doing in 2009.

“There is some urgency of this order,” one Endace employee wrote. A company financial report that month confirmed a “large GCHQ order”.

A few months later the Snowden mass surveillance revelations began appearing in the world news. A Guardian newspaper headline read “GCHQ taps fibre-optic cables for secret access to world's communications”.

The CEO of the New Zealand Manufacturers and Exporters Association, John Walley, tweeted the Guardian story and commented: “why do you think endace was so successful”. The story behind Endace was evidently known to some people in business circles.

How the interception works

Interception of fibre-optic cables – the cables that transport Internet traffic around the world – is achieved by use of an “optical splitter”, a device about the size of a garden hose fitting that reflects a fraction of the light passing along the cable off into the surveillance equipment.

Endace founder Ian Graham explained in a 2004 news story that he and colleagues had developed equipment that allowed customers to “see a copy of all the internet traffic passing that point.” He said: “We put a time stamp on it and feed it to software which gets out the information that the user needs.”

They named the New Zealand-developed equipment “DAG cards”, standing for Data Acquisition and Generation.

When the company won a 2009 exporter of the year award, the company's “world-beating product” was said to monitor Internet traffic “looking for spyware and other web and email abuse”. An official government publication in 2012 called “Building Innovation” discussed the grants to Endace and said its products were for “monitoring, measuring and protecting critical infrastructure”.

Worldwide Clients

Endace's leaked client lists show three categories of customers: governments, telecommunications companies and finance companies. Endace has sold surveillance tools to at least the first two of these. It also sells network monitoring equipment which is used by companies to check and maintain their own data networks. In a statement Stuart Wilson, Endace’s current CEO, states that its commercial technology is used by customers worldwide “who rely on network recording to protect their critical infrastructure and data from cyber criminals, terrorists and state-sponsored cyber security threats by allowing accurate and definitive investigation of security events and breaches”.

The government clients appear to be mostly intelligence agencies. A 2008 Endace client list included: “UK FGA”; the Canadian and Australian defence departments (where their electronic spy agencies are located); a mysterious US company called Rep-Tron Systems Limited, located in Baltimore a few blocks from the National Security Agency; and “DSGT Rabat Morocco” - an apparent reference to Morocco's domestic security agency, the General Directorate for Territorial Surveillance.

Sirine Rached, North Africa researcher with Amnesty International says any sales to Morocco would be of particular concern. “In Morocco, digital surveillance is intimately linked with repression of peaceful dissent – people who are peacefully protesting or criticizing the authorities face intimidation, arrest, unfair trials and sometimes imprisonment," Rached said. "We fear that the more that these surveillance tools are sold [to Moroccan agencies], the more we will see human rights abuses, especially in relation to freedom of expression and information.” 

Other customer lists include the Israeli Ministry of Defence (home of its Unit 8200 electronic spy agency), the Government of India, the Spanish Ministry of Defence and Denmark's foreign intelligence service, called the Danish Defence Intelligence Service. Endace products offer these government clients the ability to “monitor, intercept and capture 100% of traffic on networks”.

Additional clients were the US Army and the US Navy's “information dominance” command called SPAWAR.

The Endace customer lists also include many of the world's biggest telcos and Internet companies, including AT&T, AOL, Verizon, Cogent Communications, Telstra, Swisscom, Deutsche Telekom, Telena Italy, Vastech South Africa and France Telecom. Some of these may use the Endace equipment for checking their networks. But another use and feature is providing access for law enforcement and intelligence agencies to intercept the messages and data of the phone and Internet users.

All telcos and Internet companies in the US, Europe, New Zealand and elsewhere are required by law to install “intercept capable” equipment on their networks. When police or spy agencies want private data about a customer (with or without a warrant depending on the country) it can be extracted easily using these “Lawful Intercept” systems (or LI accesses).

A company “product strategy” document from 2010 said that Endace had “seen early success” providing a “LI product” to the major US telco and Internet company Sprint Corporation. Sprint did not respond to a request for comment.

Endace's status reports also noted that once the Medusa 100Gbps surveillance gear was completed to GCHQ's satisfaction, the company began developing an “AT&T Prototype”. AT&T declined to comment when approached about this article, but this is one of many examples in the Endace documents which refer to Endace supplying surveillance equipment to the giant US telco. The Snowden documents contain numerous examples of AT&T cooperating closely with intelligence agencies.

Endace also had some large finance sector companies, including Morgan Stanley, Reuters and Bank of America.  It is not clear whether any of these operations had an intelligence component.

Marketing surveillance

Endace representatives promoted their LI systems at a huge 2007 security trade show.  Advertising brochures from then that were leaked and published by Wikileaks in 2013 describe the products and talk up the need for greater state surveillance.

“The need is high and the need is now,” one Endace brochure said. “Unfortunately, we now face threats to public security in many countries.... Well equipped and well informed Law Enforcement Agencies are crucial.”

Another Endace brochure on Lawful Intercept equipment said telecommunications networks carry many types of information: voice, video, email, and instant messaging. “These networks provide rich intelligence of law enforcement, IF they can be accessed securely and with high precision.” Telcos and Internet companies needed to know what was happening on their networks to serve national security, the brochure said.

A diagram showed Endace Ninja equipment being used for lawful intercept operations, with taps in the “Carrier Network” leading off to a terminal for use by “Law Enforcement Agency”.

A third brochure gave a “LI compliance” case study for a telco “transporting traffic from more than 50 million users.” It said “Endace monitoring probes are configured to record and securely deliver targeted traffic streams.” The cost was “US$ millions” and it said “this project is currently being rolled out”.

Beginnings in Hamilton

The company was born out of research at Waikato University in Hamilton in the 1990s. The first DAG network monitoring cards were developed (for network checking rather than intelligence uses) starting in 1994, led by the dean of the School of Computing and Mathematical Sciences, Professor Ian Graham.

The technology was commercialised into a private company in 2001, co-founded by Graham and the university's management professor Neil Richardson. The company staff included various of Graham's former university researchers and students. Richardson and Graham were directors and two of the largest shareholders.

The company had formed shortly after the September 11 attacks, Graham told the New Zealand Herald in 2004, and “immediately got a whole lot of interest from people who wanted to do network monitoring”.

In 2012, after the major GCHQ sales, the company was sold for NZ$154 million to the US Emulex Corporation.  Late last year Endace was sold again to New Zealand company Echidna Limited in a management buyout.

Endace got one more New Zealand government grant in 2013 but in June 2014 it reported a loss of NZ$35 million for the year. It cut two-thirds of its workforce and still reported a US$97.4 million loss for the June 2015 financial year. The Callaghan Innovation, which manages public research grants, required Endace to return a sizeable portion of the most recent research and development grants provided.

In a written statement, Endace’s current chief executive and two-thirds owner Stuart Wilson said: “Callaghan & NZTE have been instrumental in helping Endace become a global leader in our field helping fund research and development of our network data capture and recording technology which generates significant export revenue for New Zealand, and builds important technical capability for our country.  Last year we were happy to give NZ$1.9m back to the NZ people, satisfying our commitments to Callaghan.” 

He declined to answer our questions about Medusa but in previous Endace media releases has said that demand for Endace's technology was accelerating globally with some large contracts in the government and private sectors.