19 May 2003. Thanks to all below who responded. Responses from other readers welcomed, especially from Russia. Send to jya@pipeline.com
John Young posted to Intelligence Forum (http://www.intelforum.org) on 6 May 2003: Thomas Powers wrote in 1983 that Russian code papers purchased from the Finns by William Donovan of OSS played a role in decoding Russian diplomatic traffic under the VENONA program. That while the papers were returned to the Russians by order of the Roosevelt administration the Signal Intelligence Service made copies before the return. Powers writes of the code papers: "... as I found from my own inquiries, that this material was copied before it was turned over to Gromyko, that it provided the means of decoding Russian diplomatic traffic collected by the NSA's predecessor, the Signal Intelligence Service ... this traffic is ... referred to in code-breaking and counterintelligence circles as the VENONA material." See Powers' "Intelligence Wars," pp. 15-16. This contrasts with the VENONA decoding account by Cecil James Phillips, National Security Agency, in the Preface to "Venona: Soviet Espionage and the American Response, 1939-1957:" http://www.cia.gov/csi/books/venona/preface.htm Phillips writes: "Arlington Hall's Venona breakthrough in 1943-46 was a purely analytic accomplishment, achieved without the benefit of either Soviet codebooks or plain-text copies of original messages. The 1944-46 messages--which yielded the early translations and the bulk of all translations--were recovered over a period of years by Arlington Hall cryptanalysts and decoded from a "codebook" that crypto- linguist Meredith Gardner reconstructed by using classic codebreaking techniques. " Coincidentally, a chronology of VENONA states: "1944 1 May The KGB, apparently on short notice, changes the indicator system for its cables, leaving the one-time pad page numbers en clair. November SSA's Cecil Phillips discovers the new KGB indicator, which is then used to detect "key" duplicated in Trade messages. December OSS purchases Soviet code and cipher material from Finnish sources; the Roosevelt administration orders the material returned to the Soviet Embassy in Washington." http://www.cia.gov/csi/books/venona/chron.htm ----- Is Powers correct that the codebooks helped with VENONA or has his claim been disproven since 1983? If the Russian material helped with VENONA why would Phillips state otherwise? Are there other decoding secrets Phillips has reason to hide? John Young
These responses from Intelligence Forum ----- Date: Mon, 12 May 2003 06:29:51 -0400 (EDT) From: John Chambers <jchambers@cas.org> Subject: Re: Query on VENONA Probably a bit of both. The pads recovered by the Finns were damaged (partly burned as I recall). One-time pads are a private key system: there is one pair for each communicating pair and have all the key management problems associated with private keys. The pads recovered in Finland may or may not have been amongst those that were distributed twice. If they weren't, they would at least give some idea of the true randomness of the pads. If they were, it is still necessary to break messages and recover the values of the one time pad to prove that the pad in question had been used. Once that original, and purely cryptographic, break had been made, the pad could be used to read other messages that were encrypted with it. These would all have been from a single source. Alec Chambers ----- Date: Mon, 12 May 2003 12:35:18 -0400 From: Westintel Research <Westintel@compuserve.com> Subject: Query on VENONA John Young queries the contribution made to the VENONA program by the four STELLA POLARIS codebook recovered by the Finns and then sold to OSS. this material was unavailable to Meredich Gardner when he reconstructed the NVVD codebook. The Swedish collection, combined with the archive recovered by TICOM-6 from Burgsheidungen enabled Samuel Chew to achieve his 1953 breakthrough. The question implies that the VENONA texts from 1940-1949 were all in the same cipher system, but they were not. Equally, different techniques were applied by the British and American cryptographers in 1944, 1953 and finally 1964. This is documented in VENONA: The Greatest Secret of the Cold War (HarperCollins, 1999) by your humble servant, assisted by Ceil Phillips and the Finns. Nigel West ----- Date: Tue, 13 May 2003 15:07:20 +0100 To: intelforum@his.com From: Ralph Erskine <rerskine@clara.co.uk> At 20:25 06/05/03 -0700, John Young wrote: [Snip JY message.] So far as I am aware, Thomas Powers had no personal knowledge of Venona. Cecil Phillips' statement in the Preface to "Venona" refers to "Arlington Hall's Venona breakthrough" in _1943-46_ being "achieved without the benefit of ... Soviet codebooks". There is no good reason to doubt this. A Soviet codebook did help much later, but only "after the main analytic breakthroughs were made (through 1952). It was not until 1953 that a partially burned codebook (recovered ... in 1945) was discovered to be related to the Venona cryptographic systems after another cryptanalytic breakthrough" (NSA Introductory History of VENONA and Guide to the Translations", 8). [See following.] Powers (or his informant) has confused two different periods. Cecil Phillips worked on Venona as a very young cryptanalyst. He did have many "decoding secrets" to keep, but this was not one of them. Ralph Erskine
http://www.nsa.gov/docs/venona/monographs/monograph-1.html Introductory History of VENONA and Guide to the Translations [Excerpt] VENONA Myths and Misunderstandings In spite of what has been written in a number of books and articles, Arlington Hall made the VENONA breakthroughs purely through sweat-of-the-brow analysis. There was no cryptanalytic assistance for Lieutenant Richard Hallock, Cecil Phillips, or Meredith Gardner and their colleagues from lost, discovered, or battlefield-recovered Soviet codebooks during the years in which the main analytic breakthroughs were made (through 1952). It was not until 1953 that a photocopy of a partially burned codebook (recovered by U.S. Military Intelligence in 1945) was discovered to be related to the VENONA cryptographic systems after another cryptanalytic breakthrough. The successful decryption of the VENONA messages was a triumph of analysis by a small group of intelligent and dedicated women and men working long hours in their cramped offices at Arlington Hall.
These responses from mail list H-DIPLO, forwarded by BA: ----- From: "H-DIPLO [Rausch]" <h-d1plo@socrates.Berkeley.EDU> Date: Mon, 19 May 2003 15:08:44 -0700 From: John Earl Haynes The Soviet codebook/material captured by the Finns and Venona is a complicated and not entirely clear matter. The Soviet cipher system that was the target of the American Venona project was a two-part system: a more conventional encoding system and a second "one-time pad" encoding. A Soviet message would first be encoded using a conventional codebook where words and phrases (or single letters if something was not in the codebook and needed to be spelled out) were converted into a numeric cipher. But what made the Soviet system theoretically invulnerable was second part, the one-time pad part, where the numeric values generated the conventional encoding were then changed using random number additives from a one-time pad. This generated a unique cipher that would never be repeated and could only be read by someone having the identical page from the one-time pad book. Literally, this cipher was used only one time, thus its "one-time pad" label. Unless one could break through the one-time pad part of the system, even having a complete copy of the Soviet code book would have been useless. The great success of the Venona project was the discovery by U.S. Army Signals Intelligence that the Soviets had made a procedural technical error in the one-time pad part of the system and had distributed identical pad pages, making them not "one-time" but "two-time," and rendering a portion of the messages vulnerable to having all or a portion of the additive removed. For those messages where the random-number additive could be stripped away, the remaining coded message was then vulnerable to conventional code-breaking analysis, and here any portion of the Soviet codebook, no matter how badly damaged, would have assisted. Did the material captured by the Finns assist in the second part of the Venona task? What has confused this issue is that there appear to have been two sets of Soviet cipher material captured by the Finns and obtained by the United States. One set was obtained by the OSS in late 1944 from Finnish intelligence officers after Finland left the war. In December 1944 Secretary of State Stettinius suggested, and President Roosevelt agreed, that material be returned to the Soviets as an act of good will to the USSR. This incident has been know for decades. NSA did not provide a detailed account of the Venona project until 1995. Prior to that time among the few specifics that circulated was that a damaged Soviet codebook obtained from the Finns had assisted the project. Several historians, Thomas Powers in 1983 referred to in Young’s post, and also me and my colleague Harvey Klehr (see our 1995 _The Secret World of American Communism_), assumed that the OSS must have kept a copy of the material returned to the Soviets and that this material later assisted the Venona project. However, Klehr and I told a different story in our 1999 _Venona: Decoding Soviet Espionage in America_. By that time NSA (see Robert Louis Benson and Michael Warner’s _Venona: Soviet Espionage and the American Response 1939-1957_, 1996) and Venona project veterans such as Cecil Phillips had provided much new detail. First, they said that they had no knowledge of and no record of every obtaining a copy of a Finnish-supplied Soviet codebook from the OSS. (And no one has found in OSS records any indication that a copy was kept or that it was turned over to Army Signals Intelligence.) Second Phillips, Benson and Warner relate that Finnish troops had captured the code book, partially burned, when they overran the Soviet consulate at Petsamo, Finland, in June 1941. The Finns then gave their allies the Germans a copy of the codebook and other captured Soviet cipher material. Then in May 1945 a U.S. Army intelligence team headed by Lieutenant Colonel Paul Neff located a copy at a German signals intelligence archive in Saxony. The archive was in territory assigned to the Soviet zone of occupation, and Neff’s team removed the copy only a day or so prior to Soviet occupation of the area. This material then made its way to Army Signals Intelligence. To further complicate the story, Phillips and NSA explained that while the material captured in Saxony had assisted the project, the assistance needed to be put in perspective. The cipher material captured by the Finns in 1941 reflected the Soviet code book in use at the time. The Soviets changed their code book in 1942. Consequently, while the Finnish material assisted in decoding some of the early Venona messages, the overwhelming bulk of Venona messages that were successfully read were from the period of the later Soviet code book, and here the captured material was irrelevant and the success was due to conventional cryptologic analysis. And, of course, all of it hinged on the breaking through the one-time pad system, again a matter where the Finnish material was irrelevant. John Earl Haynes ----- From: "H-DIPLO [Rausch]" <h-d1plo@socrates.Berkeley.EDU> Date: Mon, 19 May 2003 15:10:31 -0700 From: Uri Bar-Noi This affair with the Soviet one-time page pad doesn't make much sense. Could anyone explain, please, why after retrieving the stolen one-time page pad, the NKVD continued to use the same cipher, estimating that the Americans could intercept and read its traffic without any difficulty? Normally, the Soviets should have introduced a new one-time page pad instead of the old and compromised one. Furthermore, one should ask why the Americans felt obliged to alarm the Soviets and let them know that they were intercepting their diplomatic messages. Uri Bar-Noi Diplomatic Historian Open University of Israel ----- From: "H-DIPLO [Rausch]" <h-d1plo@socrates.Berkeley.EDU> Date: Mon, 19 May 2003 17:18:46 -0700 From: Hayden B. Peake There was no stolen one-time-pad page involved. There was a partial Soviet codebook retrieved by the Finns and sold or given to OSS, but it did not play a role in the initial decryption of the VENONA traffic, that was done manually as described by BENSON and WARNER in their book VENONA. The partial codebook did play a role in 1953 in reconstructing a Soviet code book, and thus some of the messages, but that was long after the principal traffic was decrypted. Hayden Peake