Revealing Secrets - Introduction

This book, "Revealing Secrets: An unofficial history of Australian signals intelligence and the advent of cyber," explores the history and significance of signals intelligence (SIGINT) in Australia. It covers the antecedents of SIGINT and its connection to cyber activities. The book aims to shed light on the often secretive nature of SIGINT and its role in Australia's national intelligence. It traces the development of SIGINT from the establishment of the Defence Signals Bureau in 1947 to the present day. The authors highlight the importance of understanding SIGINT in order to have a sufficient understanding of its purpose and activities. The book also explores the impact of cryptology on revolutions and wars throughout history and its role in military affairs. It reveals little-known stories of the contributions made by SIGINT in various conflicts, such as the First and Second World Wars. The authors emphasize the need for a comprehensive understanding of SIGINT history to maintain th Revealing Secrets An unofficial history of Australian signals intelligence and the advent of cyber Authored by John Blaxland and Clare Bergen Narrated by Simon Wahlstrom Introduction Why does Australia have a National Signals Intelligence Agency? How important are the ties with the United States, the United Kingdom, Canada and New Zealand to this arrangement? What does it do and why is it controversial? What has SIGINT to do with cyber? This book addresses these questions explaining the antecedents of SIGINT and cyber for a contemporary audience. In this book we seek to present a broad panorama giving a sense of the scale and scope of SIGINT and its role in the advent of cyber. There are already comprehensive histories of the National SIGINT Agencies of the United Kingdom and the United States. Parts of Australia's SIGINT history has been covered by authors including Desmond Bull and David Horner. Others, including John Fahey, Craig Colley and David Dufty have sought to address particular historical aspects in the first half of the 20th century, notably during the Second World War. This book, however, is the first to offer a consolidated history of Australian SIGINT and cyber in Australia, covering the period all the way from Federation through to the present. This account covers the antecedents and early days of Australian SIGINT up to and including the Second World War. The structure of this book needs to be understood in terms of what happened to the Australian Signals Directorate official history project. Initially commissioned by the former Director-General Mike Burgess in 2019, this project was cut off in mid-stride by his successor. The post-war chapters covering the years since 1945, therefore, are derived from publicly available material, mindful that many of the records that would enable a fuller account remain inaccessible to the public. These later chapters are a preview, or taster, pointing to where further research is needed. We hope these final chapters will encourage greater openness in future. It is fair to say that the secret of success in the espionage business lies in keeping one's success secret. This definition of success has meant, however, that the uninitiated are unaware of SIGINT's pivotal role in Australia's national intelligence. This book aims to set the record straight as far as possible, exposing much that was most secret for decades after the event. The establishment of the Defence Signals Bureau, or DSB, in 1947 is widely seen as the beginning of Australia's national SIGINT capability. It was the culmination, however, of a series of earlier events. DSB was a cover name for the Melbourne Signal Intelligence Centre, to be used where it is necessary to conceal the nature of its functions. Today it is known as the Australian Signals Directorate, or ASD. The DSB, or ASD as it became, was the central, but not sole, component of the national SIGINT enterprise, which included SIGINT assets in the Free Arms Services. This consolidated historical account began as an official history of the ASD. In the process of writing, we became conscious that this was only a part of Australia's SIGINT history, which does not make sense on its own. So, what is SIGINT? SIGINT concerns the revealing of other secrets and the protection of one's own. SIGINT in this work refers to all aspects of information obtained by intercepting analogue or digital messages, without the sender's permission or knowledge, transmitted via radio waves using the electromagnetic spectrum, or via line, including cable telegraphy and optical fibre. This process involves code breaking, direction finding, analysis, translation, interpretation, and then dissemination to customers other than the originally intended recipient of such messages. SIGINT also involves the protection of the sources and methods used to acquire and exploit those intercepted messages. SIGINT-related activity has been considered part of a field known as electronic warfare, or EW. SIGINT, in fact, is probably best understood as part of the suite of EW capabilities that armed forces had developed in the 20th century, including electronic countermeasures, or ECM, electronic counter-countermeasures, known as ECCM, and electronic support measures, or ESM. Both ECM and ECCM have also been defined as components of electronic protection, EP. Much of what we understand to be SIGINT, the collection and processing of foreign electronic emissions, falls under the category of ESM. Certain protective security aspects of SIGINT could be categorised as ECM. For the purposes of this work, however, we will principally use the term SIGINT. Many former practitioners kept their oath of secrecy until the end of their lives, never telling others we also served. If Australia's national SIGINT agency was, and still is, too little known and understood, the frontline work of SIGINT forces in the free services is even less so. Why is the story important? The traitor unmasked at the end of John le Carré's novel, Tinker Tailor Soldier Spy, declares that secret services were the only real measure of a nation's political health, the only real expression of its subconscious. Alex Younger, former head of MI6, has said that you can tell a lot about the soul of a country from its intelligence services. SIGINT, the most secret branch of intelligence, has influenced Australia's history and people, largely without most of us knowing it or fully understanding it, but it has left an imprint on us all. Traditionally intelligence, and SIGINT in particular, has been shrouded in secrecy. In recent years, in the interest of transparency and accountability, leaders and heads of intelligence agencies in democratic countries, notably the United Kingdom and, to a certain extent, in Australia and elsewhere, have lifted the veil judiciously to reveal the purposes of some of these agencies and some of their work. In the case of the United Kingdom, Christopher Andrew on MI5, Keith Jeffery on MI6, and John Ferris on GCHQ. In Australia, the work headed by David Horner in the three-volume history of ASIO. This is to their credit, but to reveal only part of an enterprise can be misleading. Former Prime Minister Malcolm Turnbull's revelation of the Australian SIGNALS Directorate's offensive capabilities in 2016 provide fears in some quarters of mass surveillance. Revealing only part of what agencies do now can backfire, but at the same time there are compelling reasons for secrecy, including the protection of sources and methods. How is it possible then to maintain the necessary secrecy for SIGINT while ensuring that the Australian people, who this branch of intelligence serves, have a sufficient understanding of its purpose and what it does? There is no perfect answer, but this book will argue that history which takes SIGINT into account is an essential first step. The beginning of Australian SIGINT was not the establishment of the Defence Signals Bureau in 1947, nor the formation of Central Bureau, a combined Australian Army, Royal Australian Air Force and US Army organisation in 1942. It was not the effort in the 1920s, championed by the Royal Australian Navy's Directorate of Signals and Communications, Commander S. C. Cresswell, to establish an Australian naval SIGINT capability, or the work of a brilliant young Australian codebreaker called Eric Knave, or even the Australian Commonwealth Naval Board's establishment of its own stations using Marconi wireless sets in the early 1900s. This book first introduces the reader to cryptology in the ancient world. In the millennium leading up to the Industrial Revolution, the effect of cryptological developments on the revolutions and wars of the 18th and 19th centuries, the discovery of electricity and the invention of the telegraph, it sets the scene for the revolution in technological and military affairs which laid the foundation for Australian SIGINT today. It seeks to shed light where history has been misunderstood, or only half understood, because the cryptographic dimension was unknown at the time, or deliberately left out. The Babington plot, for instance, which led to the execution of Mary Crina Scott's in bold ciphers, the precursor of SIGINT. George Scogle, an anonymous codebreaker, contributed to the defeat of Napoleon and the Battle of Salamanca in 1812. The book traverses the world wars, explaining how the role of cryptology is only now beginning to be recognised. Having not ignored SIGINT, Lord Jellicoe might have defeated the German fleet at the Battle of Jutland in 1916. If not for the decryption of the telegraph, in which Germany's foreign minister Zimmermann asked the Mexicans to keep American troops busy on their border, promising them in New Mexico and Texas in return, the United States might not have entered the First World War. Few Australians would know the story of the ANZAC squadron's unskilled wireless operators who taught themselves to intercept Turkish wireless communications in Mesopotamia in the First World War. Fewer still would know that, according to Nigel de Grey, the British SIGINT official, gun historian, in the art of traffic analysis, the often crucial tactical level of SIGINT revealed, for example, the location of enemy forces. The Australian army held the lead. In the Pacific War, early in his remarkable career, Eric Knave was praised by the British Admiralty for discovering the strategic basis for Japan's warfighting plans. The Coast Watchers was a 700-strong organisation established in the 1920s. They were not involved in SIGINT per se, but their work provided a plausible alternative explanation, or cover, for SIGINT. This way, sanitised material from SIGINT reports could be used without fear of compromising the work of the codebreakers and, if necessary, explained to be uninitiated. Sanitisation involved removing indications of the SIGINT source so as to avoid raising enemy suspicions that their codes had been compromised. Obscuring a report's telltale indicators of electromagnetic or electronic interception made it less sensitive and therefore able to be shared with those who needed it for operational contingency planning. Coast Watcher reports were the perfect plausible alternative explanation. A notable member of Australia's Coast Watchers was Ruby Boyce-Jones, who transmitted coded intelligence messages to an island in the South Pacific, braving death threats from the Japanese. Admiral Halsey would later declare that Coast Watcher intelligence saved Guadalcanal, which in turn had saved the Pacific. This was high praise. But while the Battle of Midway is famous, the Australian SIGINT contribution to the Allied victory is still virtually unknown and, in the words of an officer of the Fleet Radio Unit Melbourne, thanks to our information, submarines knew when and where to prey on Japanese ships. Frumel, Central Bureau, D-Special Section and Section 22 are all odd-sounding, deliberately nondescript names, but they, along with the Service Intercept Stations and the Coast Watchers, played a role that was vital and generally unrecognised. The author of Britain's official history of British intelligence in the Second World War, Professor Harry Hinsley, did not attempt to cover the war in the Far East when this was so much the concern of the United States. While US historians addressed aspects of it from their own perspective, there was no official history of Australian intelligence in the Second World War. As a result, Australia's story was not told. For many years after the Second World War, the Australian public was none the wiser. This work outlines the antecedents and major developments up to and include in the Second World War. For the reasons noted above, the final post-war chapters draw on publicly available information. These final chapters reveal that Australian forces were involved in SIGINT in the Korean War and particularly during Indonesia's Konfrontasi. In Vietnam, in 1966, Australia's 547 Signal Troop located the North Vietnamese Army Unit ahead of the Battle of Long Tan, but the intelligence was not well understood. After the battle, Australian and Allied staff officers paid more attention to SIGINT. In subsequent operations, notably in Iraq and Afghanistan, the support provided built on the expertise developed earlier and capitalised on emerging technology. Gradually, the general public also began to pay more attention. In 1975, Australian SIGINT came under public scrutiny over East Timor, with accusations of an Australian government cover-up for the murder of five Australian journalists and, again in 1999, when there were allegations of distortion and misuse of intelligence. A Royal Commission under Justice Hoek and the report by Philip Flood in 2004 respectively dismissed these particular allegations. But in Australia, as in other democratic societies, popular demand for transparency and accountability is on the rise. Today, Australia's SIGINT faces new challenges presented by cyberspace and greater public scrutiny than ever. The book concludes with some observations on contemporary challenges with managing the SIGINT and cyber domains and some reflections on what the future may hold. It notes that SIGINT is also enabling more intense scrutiny of the private and public lives of citizens than ever before. This is causing public concern, hence the need for Australia's SIGINT and cyber security functions to be better understood by the people they are intended to protect.