16 December 2013

Spies Reign Obama

http://www.newyorker.com/reporting/2013/12/16/131216fa_fact_lizza?currentPage=all

The Political Scene

State of Deception

Why won’t the President rein in the intelligence community?

by Ryan Lizza December 16, 2013

A senator campaigning for reform says that the intelligence leadership drives “how decisions get made at the White House.”

On March 12, 2013, James R. Clapper appeared before the Senate Select Committee on Intelligence to discuss the threats facing America. Clapper, who is seventy-two, is a retired Air Force general and Barack Obama’s director of National Intelligence, in charge of overseeing the National Security Agency, the Central Intelligence Agency, and fourteen other U.S. spy agencies. Clapper is bald, with a gray goatee and rimless spectacles, and his affect is intimidatingly bureaucratic. The fifteen-member Intelligence Committee was created in the nineteen-seventies, after a series of investigations revealed that the N.S.A. and the C.I.A. had, for years, been illegally spying on Americans. The panel’s mission is to conduct “vigilant legislative oversight” of the intelligence community, but more often it treats senior intelligence officials like matinée idols. As the senators took turns at the microphone, greeting Clapper with anodyne statements and inquiries, he obligingly led them on a tour of the dangers posed by homegrown extremists, far-flung terrorist groups, and emerging nuclear powers.

“This hearing is really a unique opportunity to inform the American public to the extent we can about the threats we face as a nation, and worldwide,” Dianne Feinstein, a California Democrat and the committee’s chairman, said at one point. She asked committee members to “refrain from asking questions here that have classified answers.” Saxby Chambliss, a Georgia Republican, asked about the lessons of the terrorist attack in Benghazi. Marco Rubio, a Florida Republican, asked about the dangers of Egypt’s Muslim Brotherhood.

Toward the end of the hearing, Feinstein turned to Senator Ron Wyden, of Oregon, also a Democrat, who had a final question. The two senators have been friends. Feinstein held a baby shower for Wyden and his wife, Nancy Bass, before the birth of twins, in 2007. But, since then, their increasingly divergent views on intelligence policy have strained the relationship. “This is an issue where we just have a difference of opinion,” Wyden told me. Feinstein often uses the committee to bolster the tools that spy agencies say they need to protect the country, and Wyden has been increasingly concerned about privacy rights. For almost a decade, he has been trying to force intelligence officials like Clapper to be more forthcoming about spy programs that gather information about Americans who have no connection to terrorism.

Wyden had an uneasy kind of vindication in June, three months after Clapper’s appearance, when Edward Snowden, a former contractor at the N.S.A., leaked pages and pages of classified N.S.A. documents. They showed that, for the past twelve years, the agency has been running programs that secretly collect detailed information about the phone and Internet usage of Americans. The programs have been plagued by compliance issues, and the legal arguments justifying the surveillance regime have been kept from view. Wyden has long been aware of the programs and of the agency’s appalling compliance record, and has tried everything short of disclosing classified information to warn the public. At the March panel, he looked down at Clapper as if he were about to eat a long-delayed meal.

Wyden estimates that he gets about fifteen minutes a year to ask questions of top intelligence officials at open hearings. With the help of his intelligence staffer, John Dickas, a thirty-five-year-old from Beaverton, Oregon, whom Wyden calls “the hero of the intelligence-reform movement,” Wyden often spends weeks preparing his questions. He and Dickas look for opportunities to interrogate officials on the gaps between what they say in public and what they say in classified briefings. At a technology conference in Nevada the previous summer, General Keith Alexander, the director of the N.S.A., had said that “the story that we have millions or hundreds of millions of dossiers on people is absolutely false.” Wyden told me recently, “It sure didn’t sound like the world I heard about in private.” For months, he tried to get a clarification from the N.S.A. about exactly what Alexander had meant. Now he had the opportunity to ask Clapper in public. As a courtesy, he had sent him the question the day before.

Wyden leaned forward and read Alexander’s comment. Then he asked, “What I wanted to see is if you could give me a yes or no answer to the question ‘Does the N.S.A. collect any type of data at all on millions or hundreds of millions of Americans?’ ”

Clapper slouched in his chair. He touched the fingertips of his right hand to his forehead and made a fist with his left hand.

“No, sir,” he said. He gave a quick shake of his head and looked down at the table.

“It does not?” Wyden asked, with exaggerated surprise.

“Not wittingly,” Clapper replied. He started scratching his forehead and looked away from Wyden. “There are cases where they could inadvertently perhaps collect, but not wittingly.”

Wyden told me, “The answer was obviously misleading, false.” Feinstein said, “I was startled by the answer.” In Washington, Snowden’s subsequent leaks created the most intense debate about the tradeoffs between national security and individual liberty since the attacks of September 11th. The debate will likely continue. According to Feinstein, Snowden took “millions of pages” of documents. Only a small fraction have become public. Under directions that the White House issued in June, Clapper declassified hundreds of pages of additional N.S.A. documents about the domestic-surveillance programs, and these have only begun to be examined by the press. They present a portrait of an intelligence agency that has struggled but often failed to comply with court-imposed rules established to monitor its most sensitive activities. The N.S.A. is generally authorized to collect any foreign intelligence it wants—including conversations from the cell phone of Germany’s Chancellor, Angela Merkel—but domestic surveillance is governed by strict laws. Since 2001, the N.S.A. has run four surveillance programs that, in an effort to detect terrorist plots, have swept up the contents of the phone and Internet communications of hundreds of thousands of Americans, and collected the telephone and Internet metadata of many more Americans. (Metadata is data about data. For telephone records, it can include numbers dialled, the date, time, and length of calls, and the unique identification of a cell phone. Internet metadata can include e-mail and I.P. addresses, along with location information, Web sites visited, and many other electronic traces left when a person goes online.)

Soon after the March hearing, Dickas called a senior member of Clapper’s staff and requested that Clapper acknowledge that his statement had been wrong. Through his staff member, Clapper declined. In July, however, after Snowden’s leaks, Clapper finally wrote to the committee and offered a formal retraction: “My response was clearly erroneous, for which I apologize.” Wyden told me, “There is not a shred of evidence that the statement ever would’ve been corrected absent the Snowden disclosures.”

Wyden is now working on a bill that would ban the mass collection of phone records and reform the court that oversees the N.S.A.’s domestic surveillance. Feinstein, who has resisted most of Wyden’s efforts at disclosure over the years, has put forward her own legislation, which would authorize the N.S.A. to continue bulk collection. Wyden dismisses her bill as “cosmetic stuff that just puts the old wine in a new bottle.” Feinstein counters that it “puts some very stringent parameters on” the program. She adds, “Senator Wyden also calls it a ‘surveillance program.’ It’s not a surveillance program—it is a data-collection program.”

Feinstein and Clapper insist that Wyden’s latest proposals would deprive the N.S.A. of crucial tools that it uses to disrupt terrorist plots. President Obama has been mostly silent on the issue. In August, he appointed a five-person panel to review intelligence policy, and the group is scheduled to issue recommendations by the end of the year. His decisions about what changes to endorse could determine whether his Presidency is remembered for rolling back one of the most controversial national-security policies of the Bush years or codifying it.

Wyden, who said that he has had “several spirited discussions” with Obama, is not optimistic. “It really seems like General Clapper, the intelligence leadership, and the lawyers drive this in terms of how decisions get made at the White House,” he told me. It is evident from the Snowden leaks that Obama inherited a regime of dragnet surveillance that often operated outside the law and raised serious constitutional questions. Instead of shutting down or scaling back the programs, Obama has worked to bring them into narrow compliance with rules—set forth by a court that operates in secret—that often contradict the views on surveillance that he strongly expressed when he was a senator and a Presidential candidate.

“These are profoundly different visions,” Wyden said, referring to his disagreements with Obama, Feinstein, and senior intelligence officials. “I start with the proposition that security and liberty are not mutually exclusive.” He noted that General Alexander had an “exceptionally expansive vision” of what the N.S.A. should collect. I asked Wyden for his opinion of the members of the review panel, most of whom are officials with ties to the intelligence establishment. He smiled and raised his eyebrows. An aide said, “Hope springs eternal.”

I—IS IT LEGAL?

In 1961, when John F. Kennedy took office, he inherited a scheme from his predecessor, Dwight Eisenhower, to invade Cuba with a small band of exiles and overthrow Fidel Castro. The plot, devised by the C.I.A. and carried out in April of that year, was a disaster: the invading forces, shepherded by C.I.A. operatives, were killed or captured, and Castro’s stature increased.

The failed plot is richly documented in a 1979 book, “Bay of Pigs: The Untold Story,” written by Senator Wyden’s father, Peter. At the time of its release, the book, which won an Overseas Press Club award, was the most comprehensive account of the Bay of Pigs fiasco. (During a six-hour interview with Peter Wyden, Castro marvelled that the author “knows more about it than we do.”) One recent morning, when Ron Wyden and I were sitting in his office discussing the N.S.A., he leaped out of his chair and walked across the room to a small bookshelf. “I want to show you something,” he said, and handed me a tattered copy of his father’s book. It describes how the C.I.A.’s arrogance and obsessive secrecy, combined with Kennedy’s naïveté, led a young President to embrace a wildly flawed policy, resulting in an incident that the author likens to “Waterloo staged by the Marx Brothers.” In Ron Wyden’s view, the book explains a great deal about the modern intelligence community and his approach to its oversight.

Wyden, a former college-basketball player, is a gangly six feet four and speaks in an incongruous high-pitched voice. He grew up in Palo Alto, California, and graduated from Stanford, where his mother was a librarian. He went to law school at the University of Oregon and, in 1972, worked as a volunteer on the campaign of Senator Wayne Morse. Morse, an Oregon Democrat, had been one of two senators to vote against the Gulf of Tonkin Resolution, eight years earlier, and became an outspoken opponent of the Vietnam War. The position had cost him the ’68 race; the Republican Bob Packwood won. “Perhaps more than any other political figure I’ve either been around or studied, Morse embodied a sense of independence,” Wyden said. “I thought, This is what public service is supposed to be about.”

Wyden was Morse’s expert on issues important to seniors in Oregon, and he later set up the Oregon chapter of the Gray Panthers, an organization that fought for seniors’ rights. One of the earliest national newspaper stories about Wyden, which ran in the Times on January 7, 1979, described a victory that elderly Oregonians won in the state legislature, where a Wyden-backed plan to allow non-dentists to fit and sell dentures was approved. “I think the measure really shows that senior citizens have bulging political biceps,” Wyden told the Times.

The next year, at thirty-one, Wyden won a U.S. House seat in a Portland district. Although he focussed on domestic issues, he entered politics just as major changes were taking place in the intelligence agencies. In the nineteen-seventies, a Senate committee chaired by Frank Church revealed widespread abuses at the N.S.A., the C.I.A., and other agencies, including active programs to spy on Americans. An N.S.A. program called Project SHAMROCK, which started shortly after the Second World War, had persuaded three major American telegraph companies to hand over most of their traffic. By the time the program was shut down, in 1975, the N.S.A. had collected information on some seventy-five thousand citizens. For many years, the information was shared with the C.I.A., which was running its own illegal domestic-intelligence program, Operation CHAOS.

The Church committee recommended not only sweeping reform of the laws governing the intelligence community but also a new system of oversight. Senator Walter Mondale, a member of the committee, said he worried about “another day and another President, another perceived risk and someone breathing hot down the neck of the military leader then in charge of the N.S.A.” Under those circumstances, he feared, the N.S.A. “could be used by President ‘A’ in the future to spy upon the American people.” He urged Congress to “very carefully define the law.” In 1978, Congress passed the Foreign Intelligence Surveillance Act, or FISA, which forbade the intelligence agencies to spy on anyone in the U.S. unless they had probable cause to believe that the person was a “foreign power or the agent of a foreign power.” The law set up the Foreign Intelligence Surveillance Court, and, in 1976, Congress created the Senate Select Committee on Intelligence. The N.S.A. and other spy agencies are instructed to keep the committee, as well as a similar one in the House, “fully and currently informed.”

In 1995, Packwood resigned, after numerous women accused him of sexual harassment and assault, and Wyden won a special election, in 1996, to replace him. In early 2001, he landed a spot on the Intelligence Committee. His father had told him about how the intelligence community had stonewalled his requests for basic information for his book. Wyden soon encountered that opacity himself, he told me, especially after September 11th: “That really changed the debate.”

On October 13, 2001, fifty computer servers arrived at the N.S.A.’s headquarters, in Fort Meade, Maryland. The vender concealed the identity of the N.S.A. by selling the servers to other customers and then delivering the shipments to the spy agency under police escort. According to a 2009 working draft of a report by the N.S.A.’s inspector general, which Snowden provided to Glenn Greenwald, of the Guardian, their arrival marked the start of four of the most controversial surveillance programs in the agency’s history—programs that, for the most part, are ongoing. At the time, the operation was code-named STARBURST.

In the days after 9/11, General Michael Hayden, the director of the N.S.A., was under intense pressure to intercept communications between Al Qaeda leaders abroad and potential terrorists inside the U.S. According to the inspector general’s report, George Tenet, the director of the C.I.A., told Hayden that Vice-President Dick Cheney wanted to know “if N.S.A. could be doing more.” Hayden noted the limitations of the FISA law, which prevented the N.S.A. from indiscriminately collecting electronic communications of Americans. The agency was legally vacuuming up just about any foreign communications it wanted. But when it targeted one side of a call or an e-mail that involved someone in the U.S. the spy agency had to seek permission from the FISA court to conduct surveillance. Tenet later called Hayden back: Cheney wanted to know what else the N.S.A. might be able to do if Hayden was given authority that was not currently in the law.

Hayden resurrected a plan from the Clinton years. In the late fall of 1999, a large body of intelligence suggested that Osama bin Laden was planning multiple attacks around New Year’s Eve. The Clinton Administration was desperate to discover links between Al Qaeda operatives and potential terrorists in the U.S., and N.S.A. engineers had an idea that they called “contact chaining.” The N.S.A. had collected a trove of telephone metadata. According to the N.S.A. report, “Analysts would chain through masked U.S. telephone numbers to discover foreign connections to those numbers.”

Officials apparently believed that, because the U.S. numbers were hidden, even from the analysts, the idea might pass legal scrutiny. But the Justice Department thought otherwise, and in December of 1999 it advised the N.S.A. that the plan was tantamount to electronic surveillance under FISA: it was illegal for the N.S.A. to rummage through the phone records of Americans without a probable cause. Nonetheless, the concept of bulk collection and analysis of metadata was born. During several meetings at the White House in the fall of 2001, Hayden told Cheney that the FISA law was outdated. To collect the content of communications (what someone says in a phone call or writes in an e-mail) or the metadata of phone and Internet communications if one or both parties to the communication were in the U.S., he needed approval from the fisa court. Obtaining court orders usually took four to six weeks, and even emergency orders, which were sometimes granted, took a day or more. Hayden and Cheney discussed ways the N.S.A. could collect content and metadata without a court order.

The Vice-President’s lawyer, David Addington, drafted language authorizing the N.S.A. to collect four streams of data without the FISA court’s permission: the content of Internet and phone communications, and Internet and phone metadata. The White House secretly argued that Bush was allowed to circumvent the FISA law governing domestic surveillance thanks to the extraordinary power granted by Congress’s resolution, on September 14th, declaring war against Al Qaeda. On October 4th, Bush signed the surveillance authorization. It became known inside the government as the P.S.P., the President’s Surveillance Program. Tenet authorized an initial twenty-five million dollars to fund it. Hayden stored the document in his office safe.

Over the weekend of October 6, 2001, the three major telephone companies—A. T. & T., Verizon, and BellSouth, which for decades have had classified relationships with the N.S.A.—began providing wiretap recordings of N.S.A. targets. The content of e-mails followed shortly afterward. By November, a couple of weeks after the secret computer servers were delivered, phone and Internet metadata from the three phone companies began flowing to the N.S.A. servers over classified lines or on compact disks. Twenty N.S.A. employees, working around the clock in a new Metadata Analysis Center, at the agency’s headquarters, conducted the kind of sophisticated contact chaining of terrorist networks that the Clinton Justice Department had disallowed. On October 31st, the cover term for the program was changed to STELLARWIND.

Nearly everyone involved wondered whether the program was legal. Hayden didn’t ask his own general counsel, Robert Deitz, for his opinion until after Bush signed the order. (Deitz told Hayden he believed that it was legal.) John Yoo, a Justice Department lawyer, wrote a legal opinion, the full text of which has never been disclosed, arguing that the plan was legal. When Deitz tried to obtain the text, Addington refused his request but read him some excerpts over the phone. Hayden never asked for the official legal opinion and never saw it, according to the inspector general’s report. In May, 2002, the N.S.A. briefed Judge Colleen Kollar-Kotelly, the incoming chief of the Foreign Intelligence Surveillance Court, about the program. She was shown a short memo from the Department of Justice defending its legality, but wasn’t allowed to keep a copy. The N.S.A.’s inspector general later said he found it “strange that N.S.A. was told to execute a secret program that everyone knew presented legal questions, without being told the underpinning legal theory.”

Meanwhile, Wyden, on the Intelligence Committee, found himself involved in the first debate about the U.S.A. Patriot Act, a law that the Bush White House pushed through Congress in October, 2001, and which included major changes to FISA. Tucked into the bill, in Section 215, was something called the “business records” provision. It allowed the government to seize “any tangible thing” from a company as long as officials proved to the FISA court that the item was “sought for an investigation to protect against international terrorism.”

Of the many new powers that Congress granted law enforcement through the Patriot Act—roving wiretaps, delayed-notice search warrants—this was not the most controversial provision at the time. It was often innocuously described as the “library records” provision, conjuring the notion that the government should know if someone is checking out bomb-making books. Some members of Congress were satisfied with the wording because Representative Jim Sensenbrenner, a Republican who was the chairman of the Judiciary Committee, and who wrote the Patriot Act, had defeated an effort by the Bush White House to make the provision even more expansive. Wyden voted for the legislation, which included the most substantial modifications of FISA since 1978, when it was enacted, but he helped attach “sunsets” to many provisions, including Section 215, that hadn’t been thoroughly examined: in five years, Congress would have to vote again, to reauthorize them. As Wyden later wrote, “The idea was that these provisions would be more thoughtfully debated at a later, less panicked time.” The Patriot Act passed overwhelmingly. (Russ Feingold, of Wisconsin, was the only senator to oppose it.)

Three months later, the Defense Department started a new program with the Orwellian name Total Information Awareness. T.I.A. was based inside the Pentagon’s Information Awareness Office, which was headed by Admiral John Poindexter. In the nineteen-eighties, Poindexter had been convicted, and then acquitted, of perjury for his role in the Iran-Contra scandal. He wanted to create a system that could mine a seemingly infinite number of government and private-sector databases in order to detect suspicious activity and preëmpt attacks. The T.I.A. system was intended to collect information about the faces, fingerprints, irises, and even the gait of suspicious people. In 2002 and 2003, Wyden attacked the program as a major affront to privacy rights and urged that it be shut down.

In the summer of 2003, while Congress debated a crucial vote on the future of the plan, Wyden instructed an intern to sift through the Pentagon’s documents about T.I.A. The intern discovered that one of the program’s ideas was to create a futures market in which anonymous users could place bets on events such as assassinations and terrorist attacks, and get paid on the basis of whether the events occurred. Wyden called Byron Dorgan, a Democratic senator from North Dakota, who was also working to kill the program. “Byron, we’ve got what we need to win this,” he told him. “You and I should make this public.” Twenty-four hours after they exposed the futures-market idea at a press conference, Total Information Awareness was dead. Poindexter soon resigned.

It was Wyden’s first real victory on the Intelligence Committee. “If you spend enough time digging into these documents and doing the work, it can pay off,” Wyden told me. “The one advantage that I have, being on the Intelligence Committee, is a chance to get access to information. But you really have to fight for it.”

In the first season of “Homeland,” the Showtime drama about the C.I.A. and terrorism, the protagonist, an agent named Carrie Mathison, conducts warrantless surveillance on an American whom she suspects is a terrorist. Saul Berenson, her boss at the C.I.A., realizes that it’s problematic, so he persuades a judge on the FISA court to give the operation the court’s legal imprimatur. Like many of the show’s plot twists, the episode seemed implausible. But it is a pale shadow of what happened with the Bush-era surveillance programs. Between 2001 and 2007, according to the inspector general’s report, before the four STELLARWIND programs had all gained a legal legitimacy, the N.S.A. wiretapped more than twenty-six hundred American telephones and four hundred American e-mail accounts, and collected phone and Internet metadata from hundreds of millions more.

During that time, an expanding circle of people in Washington, including members of Congress, lawyers at the Justice Department, reporters, and, eventually, the public, gradually became aware of the Bush programs. Jay Rockefeller, then the top Democrat on the Intelligence Committee, was one of the first officials to express dissent. On July 17, 2003, Rockefeller came back shaken from a White House meeting with Cheney, who had briefed him on the N.S.A. programs. While Congress was shutting down the Total Information Awareness program, the four phone- and Internet-spying programs under STELLARWIND had been up and running for about two years. Rockefeller drafted a handwritten letter to Cheney. “Clearly, the activities we discussed raise profound oversight issues,” he wrote. “As you know, I am neither a technician nor an attorney. Given the security restrictions associated with this information, and my inability to consult staff or counsel on my own, I feel unable to fully evaluate, much less endorse these activities. As I reflected on the meeting today, and the future we face, John Poindexter’s TIA project sprung to mind, exacerbating my concern regarding the direction the Administration is moving with regard to security, technology, and surveillance.”

Some Administration officials were concerned, too. In early March of 2004, Deputy Attorney General James Comey, who was serving as the acting Attorney General while John Ashcroft was in the hospital, determined that three of the four STELLARWIND programs were legal, but that the program involving the bulk collection of Internet metadata was not. Cheney summoned Comey to the White House and tried to change his mind, telling him that his decision would put thousands of lives at risk. Comey wouldn’t budge. Bush then sent two top White House aides to the hospital to visit Ashcroft, who was in the intensive-care unit after surgery. Ashcroft refused to overrule Comey, and the White House decided that Alberto Gonzales, Bush’s counsel, would sign a new authorization instead. Addington called Hayden the following day to make sure that he would accept the document despite the opposition of the Justice Department. “Will you do it?” he asked, according to the N.S.A. report. Hayden told me that he agreed, because he “had multiple previous such orders from D.O.J.” and “strong congressional support,” and also had in mind “the deaths of nearly two hundred Spaniards that morning in an Al Qaeda terrorist attack in Madrid.”

Lawyers many tiers below the Attorney General slowly became aware that the N.S.A. was working on something that people referred to simply as “the program.” Not long after Comey’s refusal, one Justice lawyer, Thomas M. Tamm, picked up a pay phone in a Metro station and called the Times. He told the newspaper everything he knew about STELLARWIND. As the paper began investigating Tamm’s allegations, the N.S.A. decided that the STELLARWIND programs needed a legal justification that carried more weight than a letter from the President. Like the C.I.A.’s Saul Berenson in “Homeland,” the agency asked the FISA court to make the programs legal. (As of March 26th, the Internet-metadata program had been suspended.) According to the N.S.A. report, lawyers at the N.S.A. and the Justice Department “immediately began efforts to re-create this authority.”

Over the summer, on two consecutive Saturdays, Hayden met with Judge Kollar-Kotelly, of the FISA court, to press for new authority to run the Internet-metadata program. On July 14, 2004, she gave her assent. She cited a contentious 1979 Supreme Court case, Smith v. Maryland, which held that police could place a type of monitor called a “pen register” on a suspect’s phone without a warrant. But the order didn’t target a single device; it allowed the N.S.A. to collect the metadata of all U.S. devices communicating with devices outside the U.S. According to the N.S.A. report, “The order essentially gave N.S.A. the same authority to collect bulk Internet metadata that it had under the P.S.P.,” Bush’s original, warrantless plan. (Later, Judge Kollar-Kotelly reportedly expressed misgivings about the N.S.A.’s misuse of the program, even shutting it down at one point, when she learned that the N.S.A. might have been overstepping its authority.)

On December 16, 2005, the Times broke the news about some aspects of the President’s four-pronged surveillance program. After the story appeared, Bush addressed the country to defend the P.S.P., calling it the “Terrorist Surveillance Program.” He claimed that it had been “thoroughly reviewed by the Justice Department and N.S.A.’s top legal officials,” and that N.S.A. analysts “receive extensive training to insure they perform their duties consistent with the letter and intent of the authorization.” Wyden didn’t know whether to be more shocked by the details of the N.S.A. program or by the way he learned about it. “I read about it in the New York Times,” he told me.

The Times had uncovered many details about the two programs that collected the content of e-mails and phone calls, and won a Pulitzer for its investigation, but the two metadata programs run by the N.S.A. were still largely unknown, even to most members of the Senate Intelligence Committee. Some details of the metadata programs soon appeared in the Times, in USA Today, and in a story by Seymour Hersh in this magazine. But the Bush Administration never officially confirmed the existence of the programs, which remained secret until this year.

II—OBAMA SIGNS ON

Even without a full picture of the programs, two senators who were not on the Intelligence Committee became intense critics of N.S.A. domestic surveillance: Barack Obama and Joe Biden. In May, 2006, after the USA Today article appeared, Biden said it was frightening to learn that the government was collecting telephone records. “I don’t have to listen to your phone calls to know what you’re doing,” he told CBS News. “If I know every single phone call you made, I’m able to determine every single person you talked to. I can get a pattern about your life that is very, very intrusive.”

Obama’s objections to domestic surveillance stretched back even further. In 2003, as a Senate candidate, he called the Patriot Act “shoddy and dangerous.” And at the 2004 Democratic Convention, in the speech that effectively launched his eventual campaign for President, he took aim at the “library records” provision of the law. “We worship an awesome God in the blue states, and we don’t like federal agents poking around our libraries in the red states,” he declared. In 2005, when he arrived in Washington, Obama became one of Wyden’s new allies in his attempts to reform the law. The Patriot Act was up for reauthorization, and, at Wyden’s urging, the Senate was trying to scale back the “library records” section. One of the first bills that Obama co-sponsored, the Security and Freedom Enhancement Act, would have required that the government present “specific and articulable facts” if it wanted a court order for records, a much higher standard than the existing one.

Obama and several other senators, including John Kerry, now the Secretary of State, and Chuck Hagel, the current Secretary of Defense, laid out their legal case against the provision in a letter to colleagues on December 14, 2005. The government could “obtain library, medical and gun records and other sensitive personal information under Section 215 of the Patriot Act on a mere showing that those records are relevant to an authorized intelligence investigation,” they wrote. It allowed “government fishing expeditions targeting innocent Americans. We believe the government should be required to convince a judge that the records they are seeking have some connection to a suspected terrorist or spy.” The following day, on the Senate floor, Obama said that the provision “seriously jeopardizes the rights of all Americans and the ideals America stands for.”

The Bush White House fought Obama’s changes, but offered a few minor concessions. Most notably, a business that received a demand for records could challenge in court a nondisclosure agreement that accompanied the demand. That was enough to placate some Democrats, including Obama. Wyden objected that the change did nothing to address Obama’s concerns, but the reauthorization of the Patriot Act passed the Senate on March 1, 2006. Wyden, eight other Democrats, and one Independent voted against it; Obama and Biden voted for it. Bush signed the law on March 9th.

Wyden later learned that, while he and Obama were fighting to curtail Section 215, the N.S.A.’s lawyers were secretly arguing before the FISA court that the provision should allow the N.S.A. to legally collect the phone records of all Americans. The lawyers, encouraged by their success in retroactively legalizing the Internet-metadata program, believed that they could persuade the FISA court to force phone companies to regularly hand over their entire databases. At the FISA court, there are no lawyers challenging the government’s arguments; all the N.S.A. needed to do was convince a single judge. Had Obama’s language been adopted, the N.S.A.’s case would have collapsed.

Just after noon on May 24, 2006, the FISA court issued a secret opinion ratifying the N.S.A.’s audacious proposal. It became known as the Business Records Order. That bland language concealed the fact that the court’s opinion dramatically reinterpreted the scope of the “library records” provision. The FISA court essentially gave the N.S.A. authority to place a pen register on everyone’s phone. Anytime an American citizen makes a call, it is logged into an N.S.A. database. The court required some new oversight by the Justice Department and new rules for accessing the database, but it was a nearly complete victory for the agency. The change was unknown to most members of Congress, including Obama and Wyden, who had just finished debating the Patriot Act. “What do I know?” Wyden would tell people who asked him about sensitive national-security issues. “I’m only on the Intelligence Committee.”

At the time, the public and Congress were understandably focussed on Bush’s warrantless wiretapping, and only a few officials understood the full details of the phone-metadata program. Wyden began asking questions. In June, 2006, after some stonewalling, the Bush Administration began providing summary briefings to the committee about the program. Wyden wasn’t allowed to bring any staff, and the N.S.A. didn’t respond to many of his follow-up questions. It wasn’t until the next January, after the Democrats took over Congress and were able to change the rules so that Wyden could bring Dickas to the briefings, that he fully understood what the agency was doing with the Business Records Order. He was stunned. “Look at the gap between what people think the law is and how it’s been secretly interpreted,” he said. “Holy Toledo!”

The National Counterterrorism Center is in an X-shaped building, known as Liberty Crossing, that is disguised as a suburban office park. It sits on a hill a few miles from C.I.A. headquarters, in northern Virginia. The center was created in 2003, at the recommendation of the 9/11 Commission, which concluded that the attacks might have been prevented if the F.B.I. and the C.I.A. had done a better job of sharing intelligence. At the base of the flagpole at the N.C.T.C.’s main entrance is a concrete jigsaw puzzle that represents the organization’s central mission: fitting together the seemingly random pieces of intelligence that flow into Liberty Crossing from the N.S.A., the C.I.A., the F.B.I., and other agencies.

The director of the N.C.T.C. since 2011 has been Matthew G. Olsen, a former federal prosecutor. He is a young-looking fifty-one, despite his hair, which has thinned and become grayer since he took his current job. Down the hall from his office is a door marked “Weapons, Tactics, and Targets Group,” which is part of the N.C.T.C.’s Directorate of Intelligence. The N.C.T.C. helps prepare the target lists, sometimes called kill lists, of terrorists who must be approved by Obama as legitimate threats in order to be the object of C.I.A. drone strikes. In a recent dissertation about the N.C.T.C., a former C.I.A. analyst, Bridget Rose Nolan, quoted a colleague who described the process as: “You track ’em, we whack ’em.” The day after I visited, in mid-November, a drone over Pakistan that sought to strike a terrorist compound fired three missiles that Pakistani officials claimed hit a madrassa and killed six people.

Olsen is one of the few high-level national-security officials to have dealt with the legal issues of the N.S.A.’s programs in both the Bush and the Obama Administrations, and he offers a fair reflection of how the current President and his top advisers approach them. In September, 2006, Olsen moved to the Justice Department’s new National Security Division, which was charged with overseeing the increasingly complex FISA cases concerning the N.S.A. He led a hundred lawyers in what was then called the Office of Intelligence Policy Review, which did all the preparatory work for the FISA court. Olsen started four months after the court secretly legalized the phone-metadata program. “I didn’t know any of it before I took the job,” he told me. “Only a handful of people in the entire government knew anything about it.”

Two weeks into the job, Olsen received his first assignment from lawyers at the N.S.A. The N.S.A. had been lobbying the FISA court to approve its four domestic-surveillance programs. The two metadata programs had been O.K.’ed; now Olsen and his colleagues had to persuade the FISA judge to make the phone and e-mail wiretapping programs legal. He did not see the job as especially controversial.

“It was a huge policy debate, one of the biggest ones post-9/11, and we’re still having it,” he said. “But at the time I felt like a lawyer who’d been handed a problem at a very tactical level: How do we figure this out? What are the legal rules we’re applying? What are the facts? How do we work with the N.S.A.?” He added, “I thought the goal was actually quite laudable. I was pleased to have the opportunity to work on an important thing, and I thought, Yes, if we could figure out a way to put this on a more firm legal footing, whether through judicial authority or legislative authority, that would be quite an important achievement, and it would be better for the country.”

The legal case for phone and Internet wiretapping was harder to make than the arguments concerning metadata. The Supreme Court had ruled in 1979 that metadata was not covered by the Fourth Amendment, but the content of phone calls and e-mails certainly was. Since 9/11, the N.S.A. had largely ignored the law requiring it to get a warrant for each domestic target whose content it collected. The FISA court was not impressed with Olsen’s attempt to justify legalizing the program. It issued new rules that vastly reduced the amount of collection from foreign phone and Internet sources. Olsen and his team tried different legal theories, but the court balked. Eventually, he and his colleagues decided that Bush would have to go to Congress instead and ask for legislation to amend the FISA law.

In 2008, Olsen helped lobby Congress to approve a new system that would curtail the fiSA court’s role and allow the N.S.A. to intercept enormous numbers of communications to and from the U.S. The FISA court had only to review and certify the over-all system that the N.S.A. would use; it no longer had to approve each target. Congress passed the FISA Amendments Act of 2008 on July 9th. All four Bush programs now had legal cover.

In the Senate Intelligence Committee, only Wyden and Feingold voted against the new FISA law. They were troubled by the central provision—Section 702—which created the new system governing N.S.A. surveillance of phone and Internet content. “I am one of the few members of this body who has been fully briefed on the warrantless-wiretapping program,” Feingold said at the time, in a speech on the Senate floor. “I can promise that if more information is declassified about the program in the future, as is likely to happen . . . members of this body will regret that we passed this legislation.” Wyden was reassured when Obama was elected President. Although Obama had voted for the new law, he promised at the time of the vote that, if he became President, his Attorney General would immediately “conduct a comprehensive review of all our surveillance programs.”

In February of 2009, days after Obama was sworn in, Olsen and Benjamin Powell, a Bush holdover and the general counsel for the Office of the Director of National Intelligence, went to the White House to brief the new President and Eric Holder, the new Attorney General, on the N.S.A.’s programs. There was no way to know how Obama would react. During the campaign, Holder, who was serving as a top legal adviser to Obama, had said that Bush’s original surveillance program operated in “direct defiance of federal law.” Obama had sponsored the legislation curbing the authority of the business-records provision, which was now crucial to the N.S.A. Greg Craig, Obama’s White House counsel, was also at the meeting. Because Obama had not been a member of the Intelligence Committee, much of the information was new to him. Powell, who led the briefing, and Olsen also had some news: the FISA court had just ruled that the phone-records program had so many compliance issues that the court was threatening to shut it down. The court was waiting for a response from the new Administration about how to proceed.

Olsen had recently discovered that for the previous two and a half years, the period when the phone-metadata program was supposed to have followed strict new procedures laid out by the FISA court, the N.S.A. had been operating it in violation of those procedures—and had misled the court about it. The N.S.A. was supposed to search its archive of metadata only after it had determined that there was a “reasonable, articulable suspicion”—RAS—to believe that the phone number or other search term was related to terrorism.

RAS was the thin wall between a legal program with some oversight and one with the potential for domestic spying and tremendous privacy violations. It was what prevented an analyst from querying the database for his girlfriend’s personal information or for a Tea Party activist’s network of contacts or for a journalist’s sources. Since 2006, in numerous filings before the FISA court, the N.S.A. had falsely sworn that every search term was RAS-approved. The agency had built a list of some eighteen thousand phone numbers and other search terms that it continuously checked against the metadata as it flowed into the N.S.A.’s servers. Of these, it turned out, fewer than two thousand had legal legitimacy. Thousands of the unauthorized search terms were associated with Americans. On January 15th, Olsen had informed the FISA court of the problem.

Reggie Walton, the FISA judge overseeing the program at that time, wrote, in an opinion on January 28th, that he was “exceptionally concerned” that the N.S.A. had been operating the program in “flagrant violation” of the court’s orders and “directly contrary” to the N.S.A.’s own “sworn attestations.” Walton was considering rescinding the N.S.A.’s authority to run the program, and was contemplating bringing contempt charges against officials who misled the court or perhaps referring the matter to “appropriate investigative offices.” He gave Olsen three weeks to explain why the court shouldn’t just shut down the program. The controversy was known at the court as the “ ‘big business’ records matter.”

At the White House, Olsen and Powell told Obama of the problems. “I want my lawyers to look into this,” Obama said. He pointed at Holder and Craig. Olsen believed that the N.S.A. simply had difficulty translating the court’s legal language into technical procedures; it could all be fixed. Wyden believed that the court never should have allowed the N.S.A. to collect the data in the first place. In his view, the court’s unusually harsh opinion gave Obama an opportunity to terminate the program.

“That was a very, very significant moment in the debate,” Wyden told me. “Everybody who had been raising questions had been told, ‘The FISA court’s on top of this! Everything that’s being done, the FISA court has given the O.K. to!’ And then we learned that the N.S.A. was routinely violating the court orders that authorized bulk collection. In early 2009, it was clear that the N.S.A.’s claims about bulk-collection programs and how carefully those programs were managed simply were not accurate.”

On February 17th, about two weeks after the White House briefing, Olsen, in a secret court filing, made the new Administration’s first official statement about Bush’s phone-metadata program: “The government respectfully submits that the Court should not rescind or modify the authority.” He cited a sworn statement from Keith Alexander, who had replaced Hayden as the director of the N.S.A. in 2005, and who insisted that the program was essential. “Using contact chaining,” Olsen wrote, “N.S.A. may be able to discover previously unknown telephone identifiers used by a known terrorist operative . . . to identify hubs or common contacts between targets of interest who were previously thought to be unconnected, and potentially to discover individuals willing to become US Government assets.”

Judge Walton replied that he was still troubled by the N.S.A.’s “material misrepresentations” to the court, and that Alexander’s explanation for how they happened “strains credulity.” He noted that the FISA court’s orders “have been so frequently and systemically violated that it can fairly be said that” the N.S.A. program “has never functioned effectively” and that “thousands of violations” occurred. The judge placed new restrictions on the program and ordered the agency to conduct a full audit, but he agreed to keep it running. Olsen, and Obama, had saved Bush’s surveillance program.

It was the first in a series of decisions by Obama to institutionalize some of the most controversial national-security policies of the Bush Administration. Faced with a long list of policies to roll back—torture, the wars in Afghanistan and Iraq, the use of the prison at Guantánamo Bay to hold suspected terrorists—reining in the N.S.A.’s surveillance programs might have seemed like a low priority. As core members of Al Qaeda were killed, the danger shifted to terrorists who were less organized and more difficult to detect, making the use of the N.S.A.’s powerful surveillance tools even more seductive. “That’s why the N.S.A. tools remain crucial,” Olsen told me. “Because the threat is evolving and becoming more diverse.”

Feinstein said, “It is very difficult to permeate the vast number of terrorist groups that now loosely associate themselves with Al Qaeda or Al Nusra or any other group. It is very difficult, because of language and culture and dialect, to really use human intelligence. This really leaves us with electronic intelligence.”

The N.S.A.’s assurances that the programs were necessary seemed to have been taken at face value. The new President viewed the compliance problems as a narrow issue of law; it was the sole responsibility of the FISA court, not the White House, to oversee the programs. “Far too often, the position that policy makers have taken has been that if the intelligence agencies want to do it then the only big question is ‘Is it legal?’ ” Wyden said. “And if government lawyers or the fisa court secretly decides that the answer is yes, then the intelligence agencies are allowed to go ahead and do it. And there never seems to be a policy debate about whether the intelligence agencies should be allowed to do literally anything they can get the fisa court to secretly agree to.”

Any doubts about the new Administration’s position were removed when Obama turned down a second chance to stop the N.S.A. from collecting domestic phone records. The business-records provision of the Patriot Act was up for renewal, and Congress wanted to know the Administration’s position.

It was one thing to have the Justice Department defend the program in court. But now Obama had to decide whether he would publicly embrace a section of the Patriot Act that he had criticized in his most famous speech and that he had tried to rewrite as a senator. He would have to do so knowing that the main government program authorized by the business-records provision was beset by problems. On September 14th, Obama publicly revealed that he wanted the provision renewed without any changes. “At the time of the U.S.A. Patriot Act, there was concern that the F.B.I. would exploit the broad scope of the business-records authority to collect sensitive personal information on constitutionally protected activities, such as the use of public libraries,” a Justice Department official wrote in a letter to Congress, alluding to one of Obama’s former concerns. “This simply has not occurred.” The letter, which was unclassified, did not explain the details of the metadata program or the spiralling compliance issues uncovered by the court.

Wyden’s early hope, that Obama represented a new approach to surveillance law, had been misguided. “I realized I had a lot more to do to show the White House that this constant deferring to the leadership of the intelligence agencies on fundamental policy issues was not going to get the job done,” he said.