How Obama took on six major areas of concern about NSA surveillance
Which recommendations -- from his review panel, privacy groups and others -- the president embraced, delegated or ignored
James Ball in New York
17 January 2014
In an address from the Justice Department in Washington, president Barack Obama set out his administration's substantive response  to seven months of surveillance revelations, based on documents disclosed by 30-year-old former intelligence contractor Edward Snowden.
A surveillance reform panel established by the president last year set out 46 recommendations  for change in how the NSA and other US intelligence operates. Obama accepted a small number of these, kicked others over to review panels or Congress, and ignored others entirely.
Below, we set out what action Obama is taking on six major areas of concern, what his reform panel advocated, and whether either is enough to address criticisms from privacy advocates and others.
Domestic phone records collection
The continued collection and storage of all US phone metadata -- who calls who, when, and for how long -- was the first of the Snowden revelations,  and perhaps the one with the most sustained pressure, as it involves deliberate domestic US surveillance.
As a result, it's not a surprise that some of the most substantive steps announced by the president on Friday concerned phone metadata collection, performed under Section 215 of the Patriot Act.
Obama announced his intention to "transition" to a new system which would no longer require the US government to directly keep the phone database itself. Instead, the NSA will be required either to take the records from phone companies themselves, or from an as-yet-unspecified third party (with the latter option more likely, for ease of rapid searching).
This complies almost exactly with what the surveillance reform panel recommended. They said the scheme should transition "as soon as reasonably possible to a system in which such metadata is held instead either by private providers or by a private third party", though they did also call for more legal safeguards on when the data could be searched.
There's also a catch in the timing: Obama rightly acknowledged such a transition is complex, and there are no firm options in place as to how it will be done. The attorney general and intelligence agencies have been instructed to present the president with "options" by 28 March. Beyond that, there's no timetable or roadmap.
While that course of action was endorsed by Obama's review panel, it's not one that privacy advocates are particularly fond of: phone records will still be collected, stored, and available for search by the intelligence agencies. The question of where the data actually sits is, to opponents of the program, far less significant than the fact it is being stored at all.
The president did announce one piece of immediate action, though. Previously, when intelligence agencies have wanted to search the contact networks of suspected terrorists, they have jumped three "hops" from the targets (friends of friends of friends).
Now, the president has restricted the agency to two hops, rather than three. Back-of-the-envelope Guardian calculations  suggest this could reduce the number of people caught in the dragnet of a typical search from around five million to nearer 31,000.
Foreign intelligence surveillance court
This court, often referred to as the Fisa court, is the secret panel of judges which oversees many of the NSA's bulk collection programs.
The court's meetings are closed, its rulings are classified, and it has no representation for surveillance targets (or a general privacy advocate). As a result, the court has been criticised as a mere rubber stamp rather than a real check on the NSA's power.
Obama's review panel suggested a series of reforms to the court, including changing how its appointment procedure was handled, introducing "the position of public interest advocate to represent privacy and civil liberties interests", making "greater technological expertise available to the judges", and increasing transparency of decisions.
The actual measures announced on Friday fall short of that particular goal. Obama has announced an annual review of which Fisa court decisions should be declassified and made public, but has put the director of national intelligence in charge of overseeing that procedure.
Additionally, Obama called for the "establishment of a panel of advocates from outside government" to advise the court of "significant" cases, but has called on Congress to implement this panel, and to decide how its role would work -- assuming Congress establishes it at all.
The US has numerous bulk-collection programs directed primarily at foreign targets, which can also pull in the data of US individuals. Many of these are enabled by Section 702 of the Fisa Amendment Act of 2008 -- the provision of US law which makes it permissible to collect intelligence without an individual warrant if at least one end of a communication is a non-US person.
This is the law which enables much of the NSA's "Upstream" collection from cable taps, as well as the Prism program, collecting data from US tech companies.
Obama's review board suggested a broad aim of moving away from bulk collection in favor of targeted surveillance, alongside greater transparency in the number of FAA702 and other orders issued, and suggested narrowing the circumstances in which such surveillance orders were issued.
Specifically, the panel's report said agencies should "examine the feasibility of creating software that would allow the National Security Agency and other intelligence agencies more easily to conduct targeted information acquisition rather than bulk-data collection".
The board also advocated restricting a series of procedures which created a backdoor (or loophole), revealed by the Guardian in August,  allowing for this data collected without a warrant to be searched for US person information.
Though specifics were thin on the ground, this latter recommendation appears to be the one embraced most thoroughly by the president, who announced "additional restrictions on government's ability to retain, search, and use in criminal cases, communications between Americans and foreign citizens incidentally collected under Section 702".
Obama also announced a review of how long information collected in bulk would be retained, including that of foreigners, and an annual review of the feasibility of ending some bulk collection in favor of more tightly-targeted efforts -- though neither step comes with deadlines or goals.
Proposed reforms to National Security Letters, Section 215 and FAA 702 -- including tightening their scope and aims, and increasing transparency on how many were issued -- were left largely unaddressed by the president, though he did express a willingness to discuss such steps with Congress, if they so wished.
Foreign citizens' privacy rights
The revelation of US spying on foreign citizens -- not to mention allied world leaders  -- has been a major diplomatic headache for the US throughout 2013.
While Obama addressed such subjects at length in his address and policy directive, much of what was said was effectively a restatement of longstanding laws and policies: the US only spies for national security reasons, only takes industrial secrecy of other nations when there is a pressing national security lead, and doesn't spy on foreign citizens for prurient reasons.
And allied world leaders, he said, "deserve to know that if I want to learn what they think about an issue, I will pick up the phone and call them, rather than turning to surveillance".
For those world leaders came an assurance, albeit a vague one: the US will only spy on the leaders of "close friends and allies" when there is "a compelling national security" reason.
For regular citizens came assurances of additional safeguards, including a new and shorter -- but unspecified -- timetable for the deletion of mass-collected foreign intelligence. Other, also unspecified, safeguards will be developed by the director of national intelligence and the attorney general, the president said.
Whether such steps will assuage the concerns (and legal challenges) of allies in Europe and South America is an open question, but the steps do fall substantially short of what Obama's review panel recommended, which included the suggestion of extending the protections of the US's 1974 Privacy Act to non-Americans, and a list of criteria to be used before targeting foreign leaders -- including asking if there was "reason to believe that the foreign leader may be being duplicitous", considering if "other collection means or collection targets that could reliably reveal the needed information" and weighing "the negative effects if the leader became aware of the US collection".
Encryption and security
The revelation that the NSA -- which is responsible for US cyber-defence -- had been working with the US tech industry to undermine encryption  and security standards, on which internet security relies, was one of the most significant disclosures, and one which sparked several recommendations from Obama's review panel.
The review panel made several very specific recommendations, including that the NSA should "fully support and not undermine efforts to create encryption standards", and "not in any way subvert, undermine, weaken, or make vulnerable generally available commercial software".
The panel also advised the NSA should change its policies with regards to newly discovered software vulnerabilities -- known as zero-days. These new vulnerabilities provide opportunity to break into systems far more easily, as no software patch or antivirus exists to prevent the attack.
Obama's panel advocated the NSA making use of such vulnerabilities only in exceptional circumstances, for a short time, and making its general policy to notify vendors and others of the vulnerability so it can be addressed and secured more rapidly.
However, despite several mentions of cyber-threats, neither Obama's address nor his directive made any mention of either the NSA's efforts against encryption, or his panel's recommendations on the topic -- leading to a continuation of the unusual situation in which the agency responsible for the US's cyber-defence is able to continue undermining its own efforts.
Structural reform of the NSA
Obama's review panel made several recommendations about how to change the top of the NSA to help restore trust, and make the agency's foreign intelligence mission clearer.
These included making the director of the NSA a Senate-confirmable position, making civilians eligible for the job -- even stating the president should "give serious consideration to making the next director of the National Security Agency a civilian" -- and separating out the agency's intelligence and cyber-defence roles, to make both clearer, especially given controversy over encryption standards.
The panel also recommended appointing privacy advisers at a senior level within the agency.
Of the recommendations, it is only the latter which Obama addressed, announcing such an appointment. Of any of the other proposed reforms of the most senior intelligence roles in the country, after a year of unprecedented national and international scrutiny and outrage, there was no mention.