28 January 2013

ATT Project Greenstar Secretly Spied Millions of Calls

Greenstar prefigures current ATT's once-secret participation in intercepting vast telecommunications data for the National Security Agency. More: https://www.eff.org/nsa-spying/faq

EXPLODING THE PHONE

The Untold Story of the Teenagers and Outlaws Who Hacked Ma Bell

PHIL LAPSLEY

Grove Press New York

[pp. 92-97]

If there were no billing records for fraudulent calls, there was no way to know how many fraudulent calls there were or how long they lasted. And that meant AT&T was gazing into the abyss. Say the phone company catches some college students with electronic boxes. Fantastic! But elation is soon replaced by worry. Is that all of them? Or is that just the tip of the iceberg? Are there another ten college students doing it? A hundred? Are there a thousand fraudulent calls a year or are there a million?

Engineers hate stuff like this.

Bell Labs, filled to the brim with engineers, proposed a crash program to build an electronic toll fraud surveillance system and deploy it throughout the network. It would keep a watchful eye over the traffic flowing from coast to coast, ever vigilant for suspicious calls -- not every call, mind you, but a random sampling of a subset of them, enough to gather statistics. For the first time Bell Labs -- and AT&T's senior management -- would have useful data about the extent of the electronic toll fraud problem. Then they'd be in a position to make billion-dollar decisions.

The project was approved; indeed, AT&T gave Bell Labs a blank check and told them to get right to work. Tippy-top secret, the program had the coolest of code names: Project Greenstar. Within Bell Labs Greenstar documents were stamped with a star outlined in green ink to highlight their importance and sensitivity. Perhaps as a joke, the project lead was given a military dress uniform hat with a green general's star on it, an artifact that was passed on from one team lead to the next over the years.

Greenstar development began in 1962 and the first operational unit was installed at the end of 1964. Bill Caming, AT&T's corporate attorney for privacy and fraud matters, became intimately familiar with the program. "We devised six experimental units which we placed at representative cities," Caming said. "Two were placed in Los Angeles because of not only activity in that area, but also different signaling arrangements, and one was placed in Miami, two were originally placed in New York, one shortly thereafter moving to Newark, NJ, and one was placed in Detroit, and then about January 1967 moved to St. Louis."

Ken Hopper, a longtime Bell Labs engineer involved in network security and fraud detection, recalls that the Greenstar units were big, bulky machines. "I heard the name 'yellow submarine' applied to one of them," he says. They lived in locked rooms or behind fenced-in enclosures in telephone company switching buildings. A single Greenstar unit would be connected to a hundred outgoing long-distance trunk lines and could simultaneously monitor five of them for fraud. The particular long-distance trunk lines being monitored were selected at random as calls went out over them. At its core, Greenstar looked for the presence of 2,600 Hz on a trunk line when it shouldn't be there. It could detect both black box and blue box fraud, since both cases were flagged by unusual 2,600 Hz signaling.

As Caming described it, "there were in each of these locations a hundred trunks selected out of a large number, and the [ ... ] logic equipment would select a call. There were five temporary scanners which would pick up a call and look at it with this logic equipment and determine whether or not it had the proper [ ... ] supervisory signals, whether, for example, there was return answer supervision. When we have a call, we have a supervisory signal that goes to and activates the billing equipment which usually we call return answer supervision. That starts the billing process and legitimizes the call, and if you find voice conversation without any return answer signal, and that is what it was looking for, it is an indication, a strong indication, of a possible black box that the caller called in; and if, for example, you heard the tell-tale blue box tone [ ... ] this was a very strong indication of illegality because that tone has no normal presence upon our network at that point."

When Greenstar detected something unusual, it took an audacious next step: it recorded the telephone call. With no warrant and with no warning to the people on the line, suspicious calls were silently preserved on spinning multitrack reel-to-reel magnetic tapes. If Greenstar judged it had found a black box call it recorded for sixty to ninety seconds; if it stumbled upon a blue box it recorded the entire telephone call. Separate tracks recorded the voice, supervisory signals, and time stamps.

When the tapes filled up they were removed by two plant supervisors. "They were the only two who had access from the local [telephone] company," Caming says. Then they were sent via registered mail to New York City. There, at the Greenstar analysis bureau, specially trained operators -- "long-term chief operators who had great loyalty to the system [who] were screened for being people of great trust," Ken Hopper says -- would listen to the tapes, their ears alert for indications of fraud. The operators would determine whether a particular call was illegal or was merely the result of an equipment malfunction or "talk off" -- somebody whose voice just happened to hit 2,600 Hz and had caused a false alarm. When these operators were finished listening, the tapes would be bulk erased and sent back for reuse.

"The greatest caution was exercised," Bill Caming recalls. "I was very concerned about it. The equipment itself was fenced in within the central office so that no one could get to it surreptitiously and extract anything of what we were doing. We took every pain to preserve the sanctity of the recordings."

Project Greenstar went on for more than five and a half years. Between the end of 1964 and May 1970, Greenstar randomly monitored some 33 million U.S. long-distance phone calls, a number that was at once staggeringly large and yet still an infinitesimally tiny fraction of the total number of long-distance calls placed during those years. Of these 33 million calls, between 1.5 and 1.8 million were recorded and shipped to New York to be listened to by human ears. "We had to have statistics," said Caming. Statistics they got: they found "at least 25,000 cases of known illegality" and projected that in 1966 they had "on the order of 350,000 [fraudulent] calls nationwide."

"Boy, did it perk up some ears at 195 Broadway," says Hopper. It wasn't even that 350,000 fraudulent calls was that big a number. Rather, it was the fact that there was really nothing that could be . done about it, at least not at once. "It was immediately recognized that if such fraud could be committed with impunity, losses of staggering proportions would ensue," Caming said. ''At that time we recognized -- and we can say this more confidently in public in retrospect -- that we had no immediate defense. This was a breakthrough almost equivalent to the advent of gunpowder, where the hordes of Genghis Khan faced problems of a new sort, or the advent of the cannon."

The initial plan with Greenstar was simple: Wait. Watch. Listen. Gather statistics. Tell no one. Most important, don't do anything that would give it away. "There was no prosecution during those first couple of years," Hopper says. "It was so the bad guys would not be aware of the fact that they're being measured." It was only later, Hopper says. that AT&T decided to switch from measurement to prosecution. Even then! Hopper said, "The presence of Greenstar would not be divulged and that evidence gathered to support toll fraud prosecutions would be gathered by other means." Instead, Hopper relates, Greenstar would be used to alert Bell security agents to possible fraud. The security agents would then use other means, such as taps and recordings, to get the evidence needed to convict. "Greenstar bird-dogging it would not be brought out," says Hopper. "It was just simply a toll fraud investigation brought about by unusual signaling and you would not talk about the fact that there was a Greenstar device. That was the ground rule as I understood it. Any court testimony that I ever gave, I never talked about any of that." As another telephone company official put it, "If it ever were necessary to reveal the existence of this equipment in order to prosecute a toll fraud case, [AT&T] would simply decline to prosecute."

Bill Caming became AT&T's attorney for privacy and fraud matters in September 1965. Greenstar had been in operation for about a year when he was briefed on it. His reaction was immediate: "Change the name. I don't even know what it is, but it just sounds illegal. Change the name." More innocent-sounding code names like "Dewdrop" and "Ducky" were apparently unavailable, so AT&T and Bell Labs opted for something utilitarian and unlikely to attract attention: Greenstar was rechristened "Toll Test Unit."

As the new legal guy at AT&T headquarters, Caming faced questions that were both important and sensitive. Forget how it sounded, was Greenstar actually illegal? And if it was, what should be done about it? Before joining AT&T Caming had been a prosecutor at the Nuremberg war crimes trials after World War II. He was highly regarded, considered by many to be a model of legal rectitude. Was there any way he could see that the AT&T program was legit?

There was. He later stated under oath that there was "no question" Greenstar was in fact legal under laws of the day -- a surprising conclusion for what at first blush appears to be an astonishing overreach on the part of the telephone company. There were two parts to Caming's reasoning. The first had to do with the odd wording of the wiretap laws of the early 1960s; using this wording Caming was able to thread a line of legal logic through the eye of a very specific needle to conclude that the program was legal under the law prior to 1968. The second part had to do with his position at American Telephone and Telegraph. In 1968, when Congress was considering new wiretapping legislation, Caming was in a position to help lawmakers draft the new law. He made very sure that the new wiretap act didn't conflict with AT&T's surveillance program.

Caming even informed the attorneys at the Justice Department's Criminal Division about Greenstar in 1966 and 1967, in connection with some prosecutions. "Now, that does not say that they cleared it or gave me their imprimatur," he allowed. But then, he added, "we did not feel we needed it."

Years later, the Congressional Research Service agreed with Caming regarding the legality of the program -- to a degree. While not going so far as to say there was "no question" that Greenstar was legal, it was concluded that "It is not certain that the telephone company violated any federal laws by the random monitoring of telephone conversations during the period from 1964 to 1970. This uncertainty exists because the Congressional intent [in the law] is not clear, and case law has not clearly explained the permissible scope of monitoring by the company."

This whole mess formed a challenging business conundrum for AT&T executives, the sort of thing that would make for a good business school case study. Put yourself in their shoes. You have made an incredibly expensive investment in a product -- the telephone network -- that turns out to have some gaping security holes in it. You have, as Bill Caming said, no immediate defense against the problem. You finally have some statistics about how bad the problem is. It's bad, but it's not terrible, unless it spreads, in which case it's catastrophic. Replacing the network will take years and cost a billion dollars or so. The Justice Department isn't sure there are any federal laws on the books that actually apply. And every time you prosecute the fraudsters under state laws, not only do you look bad in the newspapers -- witness the Milwaukee Journal's 1963 front-page headline "Lonely Boy Devises Way of Placing Free Long Distance Calls" -- but the resulting publicity makes the problem worse.

AT&T played the best game it could with a bad hand. For now it would quietly monitor the network, keeping a weather eye on the problem. When the company found college kids playing with the network, investigators would give them a stern talking -- to and confiscate their colored boxes. Execs would start thinking about a slow, long-term upgrade to the network to eliminate the underlying problem. And if opportunity knocked and they could help out the feds with an organized crime prosecution -- and in the process set a clear precedent for the applicability of the federal Fraud by Wire law -- well, that would be lovely.

That opportunity came knocking in 1965. As it turned out used a sledgehammer.

[pp. 115-16]

On May 5, 1969, the Supreme Court declined to hear their case. More than three years after the FBI took a sledgehammer to Ken Hanna's door, the issue was finally settled. If you were making illegal calls you had no right to privacy. The phone company could tap your line and turn the recordings over to law enforcement.

For the phone company, the victory was about much more than convicting Hanna or Dubis. AT&T now had a case that had gone all the way to the Supreme Court, one that proved, definitively, that 18 USC 1343 -- the Fraud by Wire law that the Justice Department had believed wasn't relevant -- did apply to blue boxes. Thanks to Hanna's failed appeal, the matter was now settled. AT&T finally had an arrow in its quiver to use against the fraudsters.

Throughout all of this legal drama one mystery remains: how had the telephone company found out about Hanna's or Dubis's blue box calls in the first place?

In the Hanna case, Miami telephone company security agent Jerry Doyle received a telephone call from the Internal Audit and Security Group at AT&T headquarters in New York asking him to investigate Hanna's telephone line for a possible blue box. How did investigators in New York know that somebody in Miami was making illegal calls? Hanna's attorneys asked Doyle this very question but Doyle said he didn't know.

There was a one-word answer that nobody was giving: Greenstar. Hanna had been caught up in AT&T's toll fraud surveillance network. Imagine what would have happened if this had come out during Hanna's trial. After all, the Hanna case took almost four years to resolve and went to the Supreme Court based on tape recordings of each of his illegal calls. Think of the legal circus that would have ensued if Hanna's defense attorneys had learned that the telephone company had been randomly monitoring millions of telephone calls nationwide and recording hundreds of thousands of them.

This added considerably to the stress of prosecuting Greenstar cases. AT&T attorney Caming recalls, "That was the problem in the Hanna case! Fortunately, defense counsel never probed too far as to what our original sources of information were." With blue box prosecutions, he adds, "We were always on pins and needles as to what might spill over into the public press."

Fortunately for AT&T in the Hanna and Bubis cases their luck held. And although Caming wasn't a gambler or a bookmaker, he knew a thing or two about luck. In particular, he knew it didn't last forever.

[p. 144]

At that point, the phone company billing records show something anomalous: here's a call to a number, 555-1212, that should never look like it answered and yet it does. The phone company doesn't like anomalies in its network, not so much because they think somebody might be messing with them, but just because anomalies probably mean that something is broken somewhere and needs repair.

"I knew that was an irregularity," Acker says. "My fear was, you know, if this registers on your tape" -- Acker knew the phone company in those days used paper tape for billing records -- "they'll be able to tell that [the call] answered, and they know it's not supposed to." Acker's fears were right on the money. The phone company was indeed using computer-generated reports of supervision irregularities to spot blue boxes. Along with Greenstar, these reports were a primary tool the Bell System used to detect such fraud and, due to Greenstar's secrecy, were among the most effective for prosecution.

Acker's surprise caller was a security agent from his telephone company, New York Telephone. The agent had already talked to Acker's friend John, likely because of 555-1212 supervision anomalies. But the reason the agent wanted to talk to Acker was more concrete. John had ratted out Acker to the security agent.

"He spilled his guts," Acker says. "That was just an inconceivable no-no to me. That pretty much trashed our friendship. Forever and ever." Forty years later you can still hear the intensity in Acker's voice. "When you get in trouble, you don't squeal on anybody."

[p. 182]

Charlie Schulz and Ken Hopper, members of the technical staff of the Telephone Crime Lab at Bell Laboratories.

Hopper's path to the Telephone Crime Lab was a circuitous one. In 1971 he was a distinguished-looking forty-five-year-old electrical engineer, a bit on the heavy side, with blue eyes, short brown hair, and glasses. Hopper had joined the Bell System some twenty-five years earlier, shortly after the end of World War II. Within a few years he had found himself at Bell Laboratories' Special Systems Group working on government electronics projects. The stereotype of government work is that it's boring, but Hopper was a lightning rod for geek adventure: wherever he went to do technical things physical danger never seemed far behind. There was the time he had to shoot a polar bear that had broken into his cabin while he was stationed up in the Arctic working on the then secret Distant Early Warning Line, the 1950s-era radar system that would provide advance warning of a Soviet bomber attack. Or the time he almost died in a cornfield in Iowa while building a giant radio antenna for a 55-kilowatt transmitter to "heat up the ionosphere" for another secret project. Then there's the stuff he still can't really talk about in detail, involving submarines and special tape recorders and undersea wiretaps of Soviet communications cables.

The Special Systems Group was a natural to help AT&T with the Greenstar toll-fraud surveillance network in the 1960s, Hopper says, and that work led to involvement with other telephone security matters. But the Telephone Crime Lab also owes its existence to the FBI. Hopper recalls, "In the mid-1960s the FBI laboratory came to our upper management and said they were getting electronic-involved crimes. They had no people in their laboratory that could examine evidence in these cases, especially related to communication systems, and they asked for Bell Labs' assistance. Upper management of Bell Labs agreed that this was in the public interest and that we would do that. The work was assigned to my organization, Charlie Schulz being the supervisor. We had just a few people, never more than two or three, working on this stuff.

[pp. 304-05]

The Ashley-Gravitt affair was much in the newspapers that fall and attracted the attention of Louis Rose, an investigative reporter at the St. Louis Post-Dispatch, Missouri's preeminent newspaper. Rose had written a series of articles examining the apparently cozy relationship between Southwestern Bell and the Missouri Public Service Commission, its regulator in that state. "I had been looking at all the expenditures and all of the salaries and donations by Southwestern Bell," Rose recalls. James Ashley, he says, "found a convenient thing in me, because I was already looking up these ties."

In January 1975 the Texas scandal spread to North Carolina when a former Southern Bell vice president -- another who had been forced out of the telephone company, as it happened -- admitted during an interview that he had run a $12,000-a-year political kickback fund for the Bell System. The telephone company soon found itself being investigated by an assortment of agencies: the Securities and Exchange Commission, the Department of Justice, the Federal Wiretap Commission, the FCC, and the Texas attorney general.

The next shoe to drop in the scandal was, in a way, predictable, so predictable, in fact, that Bill Caming, AT&T's patrician attorney for privacy and fraud matters, had predicted it ten years earlier. Caming couldn't say exactly when it would happen, or exactly how it would happen, but he was sure it would happen. Ever since I965, when he had first learned about AT&T's Greenstar toll-fraud surveillance system, with its tape recordings of millions of long-distance calls and its racks of monitoring equipment kept behind locked cages in telephone company central offices, Caming had maintained it was a matter of when -- and not if -- the news of Greenstar would eventually leak.

The "when" turned out to be February 2, 1975. The "how" was a front-page headline in the St. Louis Post-Dispatch: "Bell Secretly Monitored Millions of Toll Calls." The article, by Louis Rose, quoted an anonymous source within the phone company and was chock-full of details: a list of the cities where Greenstar had been installed, the specifics of its operation, the stunning news that the phone company had monitored 30 million calls and tape-recorded some 1.5 million of them. Someone -- someone high up, it seemed -- had spilled the beans. By the next day the story had been picked up by the newswires and the New York Times.

Caming didn't need a crystal ball to predict what happened next: a phone call from the chair of the House Subcommittee on Courts, Civil Liberties, and the Administration of Justice. "He said. 'I think we're going to have to have one of your guys come down and explain all this to us," Caming knew, as he had known for ten years now, that he would be the guy.

Less than three weeks later Caming found himself before the U.S. Congress. swearing to tell the truth, the whole truth, and nothing but the truth. Seated with Caming were Earl Conners, chief of security for Chesapeake and Potomac Telephone Company, and John Mack, a Bell Labs engineer who was intimately familiar with the technical details of Greenstar. True to his reputation for loquaciousness (or maybe it was his legal training) Caming made sure his colleagues never got to speak more than two dozen words over the course of the three-hour hearing. Caming explained AT&T's motivations for launching the surveillance system, how it operated, and, most important, why it was legal -- indeed, not just legal, but in fact the only option AT&T had to combat blue box and black box fraud at the time. Never once did he refer to it as "Greenstar," the name that ten years earlier he said "just sounds illegal." Perhaps it was Caming's legal reasoning, perhaps it was his appearance -- competent, prepared, confident, yet self-effacing -- or perhaps it was 195 Broadway's deft handling of the press on the matter, but AT&T managed to weather the Greenstar storm without much damage. Despite some alarming headlines there was little fallout and no criminal investigation. The Greenstar matter quickly faded away.

[pp. 358-59]

Notes

95 "decline to prosecute": Rose, "Bell Secretly Monitored Millions of Toll Calls."

96 "Change the name": During my interviews with Bill Caming I often used the term Greenstar in our discussions. Ever the AT&T attorney, he would periodically correct me: "No, that's not its name. That was an internal code name that we stopped using." Sometime later I visited the AT&T Archives in Warren, New Jersey, which maintains a computerized index of old Bell System files. I typed in "Greenstar" and watched the display light up like a Christmas tree as it found relevant documents. When I mentioned this to Caming a few days later, he gave a rueful laugh and responded, "Well, I guess you can't keep a good name down."

96 two parts to Caming's reasoning: Before 1968, the federal wiretapping law was Section 605 of Title 18 of the United States Code. It was a strangely written law. As discussed in the next chapter, section 605 did not make wiretapping ("interception") itself illegal. Rather, to commit a crime under 605 you had to both intercept a communication and then disclose the contents of the communication to someone else. Clearly when Greenstar recorded a call and a human listened to it, there was an interception, but because the trained operator listening to the tapes never discussed the contents of the communication (just the signaling of the call itself), there was no disclosure, and thus, AT&T asserted, no crime. In 1968 the Omnibus Crime Control and Safe Streets Act became the new law that governed wiretapping -- but that law had specific carve outs for random monitoring and interception of communications by telephone company personnel attempting to protect the assets of the telephone company.

96 "imprimatur": Caming, "Surveillance," pp. 243-44.

96 Congressional Research Service: Ibid., p. 234.

97 "Lonely Boy": "Lonely Boy Devises Way of Placing Free Long Distance Calls."