(redacted), NYT: NSA/CSS: OIG: Report on the Special Study of NSA's Purge of Pen Register and Trap and Trace Bulk Metadata (PDF)
19 November 2015, NYT: NSA Declassifies Inspector General Reports About Defunct Bulk E-mail Metadata Program
14 August 2014, NYT: Reagan-Era Order on Surveillance Violates Rights, Says Departing Aide
19 November 2013, NYT: Latest Release of Documents on N.S.A. Includes 2004 Ruling on Email Surveillance
29 September 2013, NYT: N.S.A. Gathers Data on Social Connections of U.S. Citizens
NOV. 19, 2015
File Says N.S.A. Found Way to Replace Email Program
By CHARLIE SAVAGE
WASHINGTON -- When the National Security Agency's bulk collection of records about Americans' emails came to light in 2013, the government conceded the program's existence but said it had shut down the effort in December 2011 for "operational and resource reasons."
While that particular secret program stopped, newly disclosed documents  show that the N.S.A. had found a way to create a functional equivalent. The shift has permitted the agency to continue analyzing social links revealed by Americans' email patterns, but without collecting the data in bulk from American telecommunications companies -- and with less oversight by the Foreign Intelligence Surveillance Court.
The disclosure comes as a sister program that collects Americans' phone records in bulk is set to end this month. Under a law enacted in June, known as the U.S.A. Freedom Act, the program will be replaced with a system in which the N.S.A. can still gain access to the data to hunt for associates of terrorism suspects, but the bulk logs will stay in the hands of phone companies.
The newly disclosed information about the email records program is contained in a report by the N.S.A.'s inspector general that was obtained by The New York Times through a lawsuit under the Freedom of Information Act.  One passage lists four reasons that the N.S.A. decided to end the email program and purge previously collected data. Three were redacted, but the fourth was uncensored. It said that "other authorities can satisfy certain foreign intelligence requirements" that the bulk email records program "had been designed to meet."
The report explained that there were two other legal ways to get such data. One was the collection of bulk data that had been gathered in other countries, where the N.S.A.'s activities are largely not subject to regulation by the Foreign Intelligence Surveillance Act  and oversight by the intelligence court. Because of the way the Internet operates, domestic data is often found on fiber optic cables abroad.
The N.S.A. had long barred analysts from using Americans' data that had been swept up abroad, but in November 2010 it changed that rule,  documents leaked by Edward J. Snowden have shown. The inspector general report cited that change to the N.S.A.'s internal procedures.
The other replacement source for the data was collection under the FISA Amendments Act of 2008, which permits warrantless surveillance on domestic soil that targets specific noncitizens abroad, including their new or stored emails to or from Americans.
"Thus," the report said, these two sources "assist in the identification of terrorists communicating with individuals in the United States, which addresses one of the original reasons for establishing" the bulk email records program.
Timothy Edgar, a privacy official in the Office of the Director of National Intelligence in both the George W. Bush and Obama administrations who now teaches at Brown University, said the explanation filled an important gap in the still-emerging history of post-Sept. 11, 2001, surveillance.
"The document makes it clear that N.S.A. is able to get all the Internet metadata it needs through foreign collection," he said. "The change it made to its procedures in 2010 allowed it to exploit metadata involving Americans. Once that change was made, it was no longer worth the effort to collect Internet metadata inside the United States, in part because doing so requires N.S.A. to deal with" restrictions by the intelligence court.
Observers have previously suggested that the N.S.A.'s November 2010 rules change on the use of Americans' data gathered abroad might be connected to the December 2011 end of the bulk email records program. Marcy Wheeler of the national security blog Emptywheel, for example, has argued  that this was probably  what happened.
And officials, who spoke on the condition of anonymity to discuss sensitive collection programs, have said the rules change and the FISA Amendments Act helped make the email records program less valuable relative to its expense and trouble. The newly disclosed documents amount to official confirmation.
The N.S.A. and the Office of the Director of National Intelligence did not respond to a request for comment.
After the Sept. 11 attacks, Mr. Bush secretly authorized the N.S.A. to conduct surveillance and data-collection activities without obeying the Foreign Intelligence Surveillance Act, in a program called Stellarwind.
The email records component caused many internal headaches. In 2004, the Justice Department questioned its legality, contributing to a confrontation in the hospital room of Attorney General John Ashcroft and the threat of a mass resignation.
Mr. Bush then halted the program until the intelligence court began issuing secret orders  authorizing it.
The court limited the categories of data that the N.S.A. was permitted to collect and restricted how it could gain access to the data. After violations of those limits were revealed in 2009, the N.S.A. suspended the program until mid-2010, only to end it the next year.