Текст
                    
AI, AUTOMATION, AND WAR
AI, Automation, and War The Rise of a Military-Tech Complex Anthony King PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD
Copyright © 2025 by Princeton University Press Princeton University Press is committed to the protection of copyright and the intellectual property our authors entrust to us. Copyright promotes the progress and integrity of knowledge created by humans. By engaging with an authorized copy of this work, you are supporting creators and the global exchange of ideas. As this work is protected by copyright, any reproduction or distribution of it in any form for any purpose requires permission; permission requests should be sent to permissions@press.princeton.edu. Ingestion of any IP for any AI purposes is strictly prohibited. Published by Princeton University Press 41 William Street, Princeton, New Jersey 08540 99 Banbury Road, Oxford OX2 6JX press.princeton.edu All Rights Reserved ISBN 9780691265148 ISBN (e-book) 9780691265155 LCCN 2024047701 (print) | LCCN 2024047702 (ebook) Version 1.0 British Library Cataloging-in-Publication Data is available Editorial: Rebecca Brennan and Rebecca Binnie Production Editorial: Natalie Baan Jacket Design: Karl Spurzem Production: Lauren Reese Publicity: William Pagdatoon and Kathryn Stevens Copyeditor: Will DeRooy, will@intelligent-editing.com
CONTENTS Preface vii Acknowledgements xi 1 Robot Wars 1 2 What AI Can Do 23 3 AI Strategy 43 4 A Military-Tech Complex 60 5 The Special Relationship 82 6 AI and Planning 99 7 AI and Targeting 115 8 AI and Cyber Operations 131 9 The Human-Machine Team 149 10 War at the Speed of Light 166 Notes 185 Bibliography 205 Index 219
PREFACE No one of a certain generation can forget the scene. It begins with a vision of Los Angeles at night, in the year 2029, after the supercomputer Skynet has tried to annihilate the human species in a nuclear holocaust. Drones fly above the ruined city, firing lasers at the last human survivors. A lone human soldier runs for cover, only to be liquidated by a laser strike. Vast robotic tanks crush human skulls beneath their tracks. Then the famous prologue appears: ‘The machines rose from the ashes of the nuclear fire. Their war to exterminate mankind had raged for decades, but the final battle would not be fought in the future. It would be fought here, in our present. Tonight …’. No one would ever describe Arnold Schwarzenegger as a great actor, and he was—perhaps appropriately enough—cast as a robot, with only a few staccato lines which mimicked the phrases of other actors, but James Cameron’s 1984 film The Terminator remains a classic. Taut and tense, it has become a cultural landmark. The Terminator is science fiction, of course. However, in the last ten years, concerns about the threat posed by autonomous weapons and increasingly intelligent computers have become palpable. More and more commentators and experts have voiced worries that war is about to be automated, that technology will finally eclipse humans, and that Cameron’s dystopia will be realised. The question of AI and war has become a central issue—perhaps the central issue—in strategic studies. As a sociologist specialising in the study of the armed forces and war, I first became interested in AI as I was working on divisional command in 2018. That research did not address the question of AI and military decision-making, but as I finished the project, the
question of how AI might transform command began to appear over the horizon. As I worked on urban warfare between 2019 and 2021, the issue became ever more pressing. Soon, the profound significance of AI to military affairs was clear to me. However, it was not easy to begin my research. Let me be honest: I was worried. If AI is as capable as experts were pronouncing, I feared, sociology might soon be of only historic interest. Sociology, which emerged as a discipline in Europe in the late nineteenth and early twentieth centuries, is predicated on the premise that human social groups have irreducible properties. Groups have constitutive powers over all forms of practice, no matter how apparently personal. Sociology prioritises the collective powers of the human group as the prime explanatory variable. Yet here was a technology which might supersede humans altogether. Thus, sociology might go the way of scholasticism or alchemy, because it cannot begin to explain AI. AI is an alien agency which may supersede human social action. Since I had begun to work on military sociology in 2004, my research on the armed forces had addressed questions of cohesion, coordination, collaboration, social interaction, and teamwork. It was all profoundly human, founded in traditional sociology. I was interested in explaining how professional military groups worked together and evolved. AI seemed to threaten my entire approach to the study of war and the armed forces. As I started to read about AI, I was not comforted. Not only were there the jeremiads by Geoffrey Hinton, Stuart Russell, and the Stop Killer Robots campaign; leading scholars in security studies, with carefully evidenced and reasoned arguments, also professed the powers of AI. They declared that AI was about to make strategy; it would decide how, where, and when to fight. AI was birthing autonomous drone swarms and combat robots. The machines are taking over. I was intimidated—concerned. Yet, as I continued to research, I came to a different conclusion. AI is a highly capable technology with prodigious and growing powers. Nevertheless, instead of imputing powers to an autonomous, pristine technology—to AI—it
may be useful for us to situate AI within its social, institutional, and organisational context. It may be productive not to assert AI’s autonomy but to show how human social groups have sustained, developed, and used AI so that it is able to exercise the powers which it has. Sociology teaches us that we should see AI—though it may be a revolutionary technology—as a manifestation of collective human expertise, not as a thing-in-itself; inscrutable, implacable, ineffable. From this perspective, the fascinating thing about the military application of AI is not its unique processing powers but rather that it has relied on so many people to sustain and apply it. It has depended on the development of novel, unusual social groups for it to have any military use at all. In particular, the application of AI to war has depended on an unlikely alliance between tech companies and defence ministries, between military professionals and civilian experts. In this book, I explore this new social configuration. While I fully recognise the impressive capabilities of AI and try to document them, I also try to decentre the analysis of AI, moving it away from the technology and back to the people—to the commanders, staff officers, and technicians (many of whom who are not military). It has been a challenging project in many ways. As everyone knows, AI and its military application are developing very fast. It is difficult to analyse a phenomenon which is moving so quickly. Researching AI has presented empirical and methodological obstacles. Sociology aspires to be generalising and abstract; it aims to offer not just a narrative history of an event but a theory of social practice. Nevertheless, the best empirical sociology is almost always situated in a concrete, definable location; it studies a particular group of people, in a defined organisation, doing something specific, at an identifiable time and place. In my work on the armed forces, I have certainly always tried to employ this method. I have studied rapid reaction forces, infantry platoons, and divisional commanders. These actors are all located at specific places; it is possible to engage with them directly. As a result, I have often visited infantry battalions and divisional headquarters.
This AI project has sometimes defied that method. The military development of AI is diffuse. It is not located in one military unit, or one headquarters, or even just a few places. Its application is diverse, multiple, and temporary. It is occurring at different times and different speeds in disparate elements of the armed forces. One cannot apprehend its military application without also engaging with tech companies and defence ministries. International and transnational linkages in the defence and tech sectors have also been significant. The military application of AI is an ecology, rather than a discrete technology; it is a complex, intangible, half-obscured web of practices and interactions, rather than a single, solid edifice. It eludes easy investigation and description. So, AI, Automation, and War is avowedly preliminary; it is merely the beginning of an investigation into AI and the armed forces. I plan to continue to work on AI and, in particular, to explore a question implied but not fully answered here: how does AI actually influence commanders’ military decisions? This book might even be read as an essay, rather than a definitive monograph. I hope that the discussion herein of the military application of AI is interesting and pertinent. It is likely to be controversial. Although I have found many allies in the literature and in discussions, I am certainly proposing a more unusual, perhaps a less fashionable, sociological view of AI. Yet it is difficult for me to know whether my analysis is successful. Readers will decide. However, even with its inevitable shortcomings, omissions, and misreadings, I hope that this book might affirm the importance of AI as an object of study to military scholars, as well as to social and political scientists much more generally. At the same time, the book will have been successful if it persuades some scholars to focus less on automation and technology and more on organisation and expertise. A short note on the use of the word ‘data’: strictly speaking, ‘data’ is a plural noun, from the Latin datum, meaning ‘the given’, or ‘the fact’. ‘Data’ means ‘the facts’, not ‘a fact’. However, in colloquial spoken English, ‘data’ is now normally used as a singular noun. Although this convention is technically wrong, I have followed it
throughout the book, for the sake of readability. For example, I have used the phrase ‘data is’, not ‘data are’.
ACKNOWLEDGEMENTS In November 2020, I was awarded a Leverhulme Major Research Fellowship for a project titled Urban War: Past, Present and Future. The fellowship ran from September 2021 to September 2024. The support of the Leverhulme Trust has been invaluable to this project and is deeply appreciated. I am indebted to Leverhulme. However, it is obvious from the title of the fellowship and the topic of the current book that some explanation is required. I have written several articles about urban warfare and revised my book on urban warfare in the course of my fellowship, but this is a book about AI, not urban warfare. In 2020 and 2021, the UK went into a series of lockdowns as a result of Covid. It was unclear when normality would return. I had been working on urban warfare, with a view to writing a monograph in due course. As a result of the lockdowns, and with no clear idea of when they might end, I began to write my urbanwarfare book in 2020. I was surprised to finish it quickly in the autumn of 2020—before I was notified of my award. In the final chapter of my urban-warfare book, I discussed the question of whether AI would transform urban operations. The book you now hold initially began as an analysis of AI and urban warfare, then. However, as I was writing it, the urban element receded, and the book became a work about AI and military automation more generally. For me, the connections between AI, automation, and urban warfare are close. I discuss them in the final chapter of the book. Nevertheless, I must admit that this project has taken a rather different course to the fellowship application which the Leverhulme Trust so kindly agreed to support. I hope that the trustees will regard this work as a valid substitute for the formally planned
project. I have continued to work on urban warfare during the fellowship and intend to write a historical sociology of urban warfare in due course. Much of the research I did during this fellowship will form the basis of that book. The people at the Leverhulme Trust were the prime patrons of this research and deserve full thanks. However, other institutions have also supported me. In January 2016, I became the chair of war studies at Warwick University. I am grateful to the support of the university and the politics and international studies department throughout my time there in making the application to Leverhulme and in supporting my research; I was allowed to work on this project with almost no interruption for two years. At Warwick, I am especially indebted to Jill Pavey, Richard Aldridge, Nick VaughanWilliams, Stuart Croft, Christopher Moran, George Christou, Kieran Moore, Daz Kittendorf, Matthew Clayton, Stuart Elden, Christopher Hughes, and Max Warrack. In October 2023, I moved back to the University of Exeter, where I was eventually appointed the director of the Strategy and Security Institute. It has been a pleasant return to old friends, and I am grateful for the support of colleagues at Exeter, including Rob Lamb, Gareth Stansfield, Ally Sutherland, Roo Haywood-Smith, and Natalie Rees, as I completed this project. This was not an easy project. In the end, I discussed AI with 126 interlocutors, who worked in the armed forces, defence ministries, think tanks, and tech companies in the UK, the US, and Israel. It would probably have been impossible and certainly would have been much harder without the support, guidance, and help of many friends and colleagues. They are too many for me to record their precise contributions, but I am especially grateful to Jack Shanahan and Randall Collins. Despite having just retired from the US Air Force and the directorship of the Joint Artificial Intelligence Center, Jack Shanahan read and commented on the entire manuscript in detail, eliminating some important misunderstandings. I met Randall Collins for the first time in about 2000, when he was visiting Exeter. He has remained a mentor and friend to me since that time. I appreciate his careful comments on the manuscript and his affirmation that this project was valuable, even when I was struggling to clarify my
thoughts. One of the best things about returning to Exeter has been renewing my friendship with Richard Everson, professor of computer science at the University of Exeter; he provided very helpful expert comments on chapter 2. I could not have written the section on Covid testing in Liverpool in chapter 7 without the generous assistance of Joe Fossey, Iain Buchan, Tom de Silva, and Luke Palmer. Christopher Earls at Cornell was also extremely patient with his advice. The comments of two anonymous reviewers at Princeton University Press were pertinent and most appreciated. Many other colleagues and friends helped me. I cannot document their contributions in full here, but I hope they remember how they supported me: Christopher Dandeker, Angus Dodd, Julian Rimmer, Hew Strachan, Tim Edmunds, Andrew Dorman, Patrick Bury, James Rogers, Rob Bassett-Cross, Justin Lynch, Luke Vannurden, Eitan Shamir, Yagil Levy, Eyal Ben-Ari, Paul O’Neil, James Kidner, Paul Hollingshead, Lucinda Kirk, Oliver Lewis, Rob Smyth, Imogen Verret, Eyal Ben-Ari, Will Blyth, Steve Rasburn, Geraint Evans, Fiona Freely, Chris Copland, Jules Buczacki, Matthew Murphy, Dean Pace, Stephen Houldsworth, James Chiswell, Oliver Denning, Sean O’Connor, Alex Case, Graham Fairclough, Tom Quinn, Edward Stringer, Jim Hockenhull, Rajat Pasal, and Amos Fox. I am very grateful to Rebecca Brennan and Becca Binnie for their support as editors and to Will DeRooy for his skilful and diligent copy-editing. Finally, but not least, I remain, of course, indebted to my wife, Cathy King.
1 Robot Wars The New Prometheus Are we witnessing the birth of a new Prometheus? In 2005, computer scientist and futurist Ray Kurzweil declared that ‘the singularity was near’.1 He believed that computer superintelligence would appear within three decades and, therefore, that machines were about to transform civilization in ways we could only begin to imagine. In 2024, he remained convinced of the epochal powers of artificial intelligence (AI). The ‘singularity’ was, for him, even nearer.2 He predicted that with the help of AI, ‘We are going to extend our minds many millions-fold by 2045’.3 There is little doubt that in the last two decades AI has developed startling—near miraculous—powers. Kurzweil is not alone in his belief that AI will transport humans to a new era. James Lovelock, the celebrated originator of the Gaia Theory, believes that the Anthropocene, the age of humans, is over: we are at the dawn of the Novacene, an age in which AI will control and manage humans’ lives. AI could soon function a million times more quickly than the human brain. Eventually, according to Lovelock, the Novacene will regulate the ‘chemical and physical conditions to keep the Earth habitable for cyborgs’.4 In his recent bestseller, The Coming Wave, Mustafa Suleyman, an AI pioneer and one of the founders of DeepMind, professed a similar view of AI: ‘And now we stand at the brink of another such moment as we face the rise of a coming wave of technology that includes both advanced AI and biotechnology.
Never before have we witnessed technologies with such transformative potential, promising to reshape our world in ways that are both awe-inspiring and daunting’.5 Suleyman is well-placed to know whether we are, indeed, on the edge of an AI revolution. The London-born son of a Syrian taxi driver and an English nurse, he was abandoned at sixteen. He then gained a place at Oxford University to study philosophy but dropped out before beginning to work on AI, a field in which he became a major figure. Indeed, he was integral to an event which is widely regarded as one of the seminal moments in the advance of AI in the last decade: he helped to develop AlphaGo, the AI program which defeated the world Go champion in 2016. AlphaGo was a remarkable achievement. Even after IBM’s Deep Blue defeated Garry Kasparov, the world chess champion, in 1997, experts claimed that ‘it may be a hundred years before a computer game beats humans at Go— maybe longer’,6 because Go is a far more complex game than chess. Go is played on a board of 324 squares. Players place black and white stones on any of the interconnections between the squares, with a view to surrounding each other’s pieces. The player who surrounds more of the other player’s pieces wins. Go, therefore, has a vastly greater number of potential moves than chess does. After three pairs of moves in a game of chess, there are about 121 million possible configurations; after three moves in a game of Go, there are on the order of 200 quadrillion (2 × 1017) possible configurations.7 Experts’ scepticism was not wholly misplaced. Yet the experts were wrong. In 2010, Demis Hassabis, Mustafa Suleyman, and Shane Legg set up a small tech company called DeepMind. They were interested in exploring the possibility of developing artificial general intelligence; as Suleyman put it, ‘We wanted to build truly general learning agents that could exceed human performance at most cognitive tasks’.8 It was a hugely ambitious undertaking. In 2013, DeepMind developed an algorithm, Deep Q-Network, which could play Atari computer games. The success of Deep Q-Network attracted the attention of major companies in Silicon Valley, and in 2014 Google bought DeepMind. In 2015, the DeepMind team began to work on AlphaGo, training the
program by having it watch 150,000 games of Go.9 Initially, the computer failed badly in its attempts to play the game. Gradually it learnt on massive datasets, teaching itself a system purely on the basis of trial and error. Because it was a computer program, it could run through games almost infinitely. Eventually, on 15 March 2016, AlphaGo beat world champion Lee Sedol in a five-game series, 4–1. Famously, in move 37 of game 3, AlphaGo made an extraordinary play, positioning a stone on its own on the edge of the board. Amazed observers declared, ‘It’s not a human move’.10 Sedol was plainly disturbed by the unexpected move, and AlphaGo went on to win the game. What was most striking was not just that a computer program had learnt to play Go but that it seemed to have developed a creativity that exceeded the imagination of even the best human player. AlphaGo was designed purely to play a game. The program was thus, in a certain sense, trivial. Yet the evidence for an AI revolution is becoming incontrovertible. AI has already made major contributions to scientific and medical research, fields in which it has affected the lives of perhaps millions and will soon affect the lives of billions. Since 1972, biologists have been working on the problem of protein structure: ‘Determining the crumpled shapes of proteins based on their sequences of constituent amino acids has been a persistent problem for decades in biology. Some of these amino acids are attracted to others, some are repelled by water, and the chains form intricate shapes that are hard to accurately determine’.11 In 2020, DeepMind announced that AlphaFold, a program created to identify new chemicals, had developed a method of mapping the structure of folded proteins. By the middle of 2021, the program had mapped 98.5 per cent of the proteins in the human body. AlphaFold had solved the problem of protein folding in just eighteen months.12 AI programs now routinely read scans for cancer more accurately than doctors do.13 AI may have even more radical medical uses. One of the pressing needs of modern medicine is to develop new drugs to overcome antibiotic-resistant bacteria. The Broad Institute at MIT and Harvard, led by Dr Felix Wong, has used AI to make major progress in this area.14 As Jeremy Hsu has described, Wong’s team
‘tested the effects of more than 39,000 compounds on Staphylococcus aureus and three types of human cells from the liver, skeletal muscle and lungs. The results became the training data for AI models to learn about the patterns in each compound’s chemical atoms and bonds. That allowed the AIs to predict both the antibacterial activity of such compounds and their potential toxicity to human cells. The trained AI models then analysed 12 million compounds through computer simulations to find 3646 compounds with ideal drug-like properties’.15 Wong explained the significance of AI to the research: ‘Our [AI] models tell us not only which compounds have selective antibiotic activity, but also why’. Because AI can process huge quantities of data, plotting thousands of variables, it is able to identify patterns which are quite undetectable to human researchers. Humans simply cannot hold all those variables in their minds. Consequently, AI programs have detected new molecular qualities, identifying relations between a molecule’s structure and its antibiotic capacity that humans had neither perceived nor defined. Algorithms have been employed with increasing success in many other fields, including oceanography, in which they have been trained to distinguish between submarines, mines, and sea-life better than humans can.16 AI has also become integral to business and commerce. It has been essential to the competitive advantage of the largest companies.17 The rise of Amazon is substantially due to data and AI. Amazon’s algorithms have automated buying and selling, and, as a result, they have been able to predict consumers’ tastes on the basis of the things that other customers, with similar data profiles, have bought.18 Walmart has also successfully harnessed the predictive power of AI. Walmart’s algorithms noted that when a hurricane was announced, consumers in the American South stocked up on PopTarts (likely because Pop-Tarts are easy to prepare and are high in calories). As Linda Dillman, Walmart’s chief information officer, observed, ‘Strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane’.19 Consequently, Walmart shipped more of that product to its stores in the affected states when a hurricane warning was issued.
A similar transformation has been evident in financial markets. Originally, the stock market consisted of human traders physically communicating with each other on the trading floor and exchanging notes documenting the deals they had done in real time, face to face. The process involved famously chaotic scenes in which jacketed traders gesticulated and shouted at one another. In the 1980s, stock markets became predominately computerised; exchanges began to be communicated over email. Electronic trading displaced physical trading. In 2006, entrepreneurs recognised the potential of high-volume electronic trading. High-volume trading involved thousands of small transactions which exploited small price shifts in the market. Traders sold and bought rapidly as prices rose and fell; speed was everything. Since trading had already been digitised—it was electronic—high-volume traders realised that algorithms might be developed to compute and execute trades; software could process financial data far more quickly and accurately than humans could. As a result, from 2006, high-volume traders automated their activities, employing algorithms to sell and buy shares ‘at the speed of light’.20 The stock exchange has been revolutionised by AI; buyers and sellers are connected digitally. Sales are now processed automatically. AI’s progress shows no sign of slowing. On the contrary, it has been exponential and self-reinforcing. On 30 November 2022, OpenAI released ChatGPT, a large generative language model capable of trawling the entire internet for data and producing useful responses to prompts. Many people see generative AI as the next breakthrough. Indeed, some hope that generative AI will help to alleviate poverty in the developing world, by increasing education and economic productivity: ‘AI stands to transform lives in the emerging world, too. As it spreads, the technology could raise productivity and shrink gaps in human capital faster than many before it’.21 In the global south, a lack of teachers, educators, doctors, engineers, and managers is a major obstacle to development; ‘AI could ease this shortfall, not by replacing existing workers, but by helping them become more productive’.22 Locals could draw on AI to help bridge the gaps in expertise. AI might act
as a proxy teacher or doctor for locals, accessing the internet for help. For instance, Tonee Ndungu, an entrepreneur in Kenya, has developed two apps to help children learn through engaging with a chatbot.23 In the West, too, many leaders have advocated AI as a way of transforming the economy and improving the well-being and livelihoods of citizens. For instance, in 2023 the UK prime minister Rishi Sunak, in response to worries about economic stagnation after Brexit, declared that AI represented ‘one of the greatest opportunities for the UK’: ‘Combined with the computational power of quantum we could be on the precipice of discovering cures for diseases like cancer and dementia or ways to grow crops that could feed the entire world’.24 It is difficult to predict how AI will evolve even in the next five years. As a result of advances in AI, humanity is now on the edge of a Fourth Industrial Revolution. The powers of computing, data, and digital communications—all enhanced, enabled, and integrated by AI —are converging to transform every aspect of human existence, just as coal, electricity, and nuclear power successively transformed society in previous eras. A new Prometheus is appearing. Frankenstein’s Monster Or perhaps, rather than a Prometheus, we are waking a monster. Experts in the field of computing have highlighted the dangers AI may pose. For instance, in an interview with The New York Times in May 2023, Geoffrey Hinton, one of the pioneers of neural networks and a seminal figure in the development of machine learning (ML), confessed his fears that AI was approaching a tipping point, when the interconnection of existing systems might trigger the rise of a new class of intelligence. Computers, he said, by sharing their data automatically with each other, could soon become vastly more intelligent than humans: ‘Whenever one [model] learns anything, all the others know it. People can’t do that. If I learn a whole lot of stuff about quantum mechanics and I want you to know all that stuff about quantum mechanics, it’s a long, painful process of getting you to understand it’. Indeed, Hinton described AI as not just troubling
but an ‘existential threat’. In the near future, he feared, AI would be harnessed neither to play games, nor to make medical advances, but for war. He urged, ‘What we want is some way of making sure that even if they’re smarter than us, they’re going to do things that are beneficial for us’, adding, ‘but we need to try and do that in a world where there [are] bad actors who want to build robot soldiers that kill people. And it seems very hard to me’.25 Hinton warned that AI may be used for military purposes. Indeed, AI may automate war, as killer robots, directed by non-human machine intelligence, take over. By February 2024, Hinton’s fears had increased. He said: ‘If I were advising governments, I would say that there’s a 10 per cent chance these things will wipe out humanity in twenty years. I think that would be a reasonable number’.26 Hinton might be considered alarmist. Nevertheless, it is striking that many policymakers and scholars who are experts on strategy and security have also worried about the development of AI and its implications for war. They too fear the automation of war. The former US secretary of state Henry Kissinger has been prominent here. Kissinger, controversial though he is, was probably the most important Western diplomat, strategist, and strategic thinker of the late twentieth century. He was well-placed to make a judgement about the strategic implications of AI for global security. In one of his final acts as a public figure, Kissinger expressed his worries about AI. In 2021, just two years before he died, he published a book with Eric Schmidt and David Huttenlocher about AI and ‘our human future’. Kissinger’s choice of co-authors was well considered. Schmidt has had a long and illustrious career in Silicon Valley and is profoundly aware of the potential of AI. He served as CEO of Google from 2001 to 2011 and subsequently chaired the Defense Innovation Board and the National Security Commission on Artificial Intelligence (NSCAI) in Washington. Huttenlocher is the dean of computing at MIT, having previously served as the dean of Cornell Tech; both are new centres dedicated to the analysis of digital technology and AI. The book is, then, a statement from a pre-eminent strategist and two tech and AI specialists. It is an important contribution.
In The Age of AI and our Human Future, Kissinger, Schmidt, and Huttenlocher avoid the jeremiads of which Hinton is perhaps guilty. They explore the question of how AI will change strategy and the prosecution of war as a political enterprise. In the near future, might AI replace human strategic judgement? Might computers decide when, where, and how to fight wars? The authors’ assessment of the political implications of AI for strategic affairs is sobering: ‘Only very rarely have we encountered a technology that challenged our prevailing modes of explaining and ordering the world. But AI promises to transform all human experiences’.27 It is predicted that ‘the introduction of non-human logic to military systems and processes will transform strategy’. If the armed forces accrue an advantage from using AI, then they will surely use it—even if only to defend themselves from those states which do use AI. Yet the perils of AI are clear. The authors are particularly concerned with cyberwarfare, lethal autonomous systems, and nuclear weapons. AI operates at a speed which no human can achieve, offering very real benefits to states and their armed forces. The authors warn that if AI gains control of weapons, including nuclear weapons, the norms of deterrence which have operated since 1945 would collapse. The rationale and motivations of AI might be different to those of humans: ‘In contrast to the field of nuclear weapons, no widely shared proscription and no clear concept of deterrence (or of degrees of escalation) attend such uses of AI-assisted weapons […] Such reliance will introduce unknown or poorly understood risks’.28 Kissinger, Schmidt, and Huttenlocher are therefore disturbed by the prospect of the automation of war, a scenario in which AIs, not humans, develop strategy and prosecute war. In the context of interstate competition, military automation carries many risks. The authors declare that, in order to keep humans in control of lethal weapons, ‘We will need to overcome, or at least moderate, the drive to automaticity before catastrophe ensues’. They conclude, ‘Defence will have to be automated without conceding the essential elements of human control’.29 States need to employ AI, but they must also mitigate and control it.
Kissinger, Schmidt, and Huttenlocher are perhaps the most prominent strategic commentators to address the question of AI. Yet they are certainly not alone among strategic experts in their disquiet that AI is about to automate strategy. Many other scholars believe that AI will inform and even automate strategy. In the last decade, the British military scholar Kenneth Payne has been an eloquent voice in these debates. In a series of books and articles exploring the automation of strategy and war, Payne concurs with Kissinger, Schmidt, and Huttenlocher. In the past, human commanders have been inhibited by a range of emotional commitments. They have been scared to put themselves or their troops at risk; fear has emasculated decision-making. In many cases, imagined fears have been crippling. Pity and morality have also constrained decisionmaking. Commanders have often sought to preserve human life. On other occasions, hatred and vengeance have compelled extreme actions which have no military logic. For Payne, AI potentially cures these strategic inhibitions: ‘AI is primarily a decision-making technology. Its effect is on the nature of warfare, insofar as it alters the long-standing human psychology of the decisions made in combat’.30 That is, AI will not be hampered by the foibles of human psychology. AI will calculate the strategic situation entirely on the basis of a logical analysis of the data, in order to make rational decisions. It will make strategic decisions quickly and accurately to execute those decisions instantaneously. Consequently, ‘AI alters the nature of war by introducing non-human decision-making’.31 War will become an automated competition between computers, not a visceral struggle between peoples. It is a radical claim, but other scholars have concurred. For instance, the political scientist and arms control expert Denise Garcia echoes Payne exactly: ‘The development of artificial intelligence and its uses for lethal purposes in war will fundamentally change the nature of warfare’.32 In her recent monograph, she claims that ‘militarised artificial intelligence’ represents an existential threat. She believes that AI will determine how, when, and where wars are fought: ‘What is at stake is the potential loss of human control to machines that will kill autonomously in response to an algorithm,
with no humans involved’.33 For Garcia, the only solution to this future is regulation and human control.34 Roberto Gonzalez, a military anthropologist, also decries the military application of AI. Unlike Garcia, he claims that the pursuit of AI is practically misguided; AI is not nearly as effective as military leaders believe. Yet the armed forces are on a ‘quest for the automated battlefield’35 and are actively committed to using AI to automate their operations. In his work on AI and military decision-making, James Johnson has also warned that military automation, and the substitution of human commanders by AI, is imminent. Like Gonzalez, he rejects the claim that AI might perform the functions of human commanders. He insists, ‘Machines cannot reliably complement or augment, let alone replace, the role of humans in command decision-making’.36 War is non-linear, unpredictable, and chaotic; AI models are unsuited to the complexity of military decision-making. Nevertheless, Johnson believes that, because of weaknesses in human psychology, AI will begin to assume command responsibilities. In a crisis situation, where there is an overwhelming amount of data, humans are liable to place a false trust in AI. They will become victims of the ‘automation bias’; they will defer to AI because they are uncertain what to do themselves. Consequently, ‘as deep human–machine symbiosis alters and shapes the psychological mechanisms that make us who we are, thus as they learn and evolve, AI agents will likely become—either inadvertently or more probably by conscious choice—de facto strategic actors in war’.37 Johnson continues: ‘The logical end of this trajectory is an AI commander—planners, warfighters, and tacticians. The danger is that decision-makers may seek to reconcile the paradox of war by outsourcing our consciences in the use of lethal force to non-human agents who are ill-equipped to fill this ethical-moral void’.38 The literature is troubling. Many scholars believe that artificial intelligence is about to assume the role traditionally taken by human commanders and political leaders; AI will make the decisions. AI will arrogate the fatal decision of whether to go to war. AI will assume
the role of politician and commander-in-chief. AI will automate strategy. War itself will be directed not by humans but by machines. Kissinger, Schmidt, and Huttenlocher, and all these other commentators, then, profess an imminent revolution in strategic affairs. It is possible to identify a second theme in the literature on AI and war. Many scholars are not so exercised by the thought of AI automating strategy, but they are deeply concerned that AI will automate warfare. They are disturbed by the prospect that AI will automate weaponry. Above all, scholars in this camp fear that AI will necessarily lead to a proliferation of lethal autonomous weapons; drone swarms and robots controlled by AI will dominate.39 The fear that AI will automate weapons has been apparent for about a decade. In 2013, activists concerned about the military threat posed by autonomous weapons created a group called the Campaign to Stop Killer Robots. This group advocates the regulation of the military application of AI. As part of its campaign, at the International Joint Conference on Artificial Intelligence on 28 July 2015, Elon Musk, Stephen Hawking, Demis Hassabis, and many other leading AI experts published an open letter documenting their concerns about the military application of AI; AI could be used to turn weapons against humans. Unlike nuclear weapons, autonomous weapons will be easy and relatively cheap to develop. ‘The key question for humanity today is whether to start a global AI arms race or to prevent it from starting. If any major military power pushes ahead with AI weapon development, a global arms race is virtually inevitable, and the endpoint of this technological trajectory is obvious: autonomous weapons will become the Kalashnikovs of tomorrow’.40 Stuart Russell, who played an important role in the development of AI from the 1980s, has been a leading opponent of the proliferation of AI-enabled autonomous weapons. He has campaigned vociferously for the regulation of AI for military purposes and was one of the signatories of the Campaign to Stop Killer Robots’ 2015 letter. He is scared by the prospect of AIautomated weapons. To highlight his fear, he created a short fictional film called Slaughterbots.41 The film, released in November
2017, dramatized the assassination of a senator by a swarm of killer drones which then attacked a university campus. The implication was that once they have been automated, such swarms will kill without constraint.42 Following Slaughterbots, Russell dedicated one of his BBC Reith Lectures in 2020 to the military application of AI. He discussed the issue of automated weapons and killer drone swarms almost exclusively.43 The lecture reached a climax when Russell described a scenario in which a lethal quadcopter the size of a jar could be armed with an explosive projectile device: ‘A regular shipping container could hold a million lethal weapons […] The inevitable endpoint is that autonomous weapons become cheap, selective weapons of mass destruction’.44 He continued: ‘Anti-personnel autonomous weapons could wipe out all the males in a city between 12 and 60 or all the visibly Jewish citizens in Israel. Unlike nuclear weapons, they leave no radioactive crater’. As evidence, he cited the Turkish use of an autonomous Kargu-2 drone in Libya in March 2020. Russell concluded that unless governments acted to regulate the military application of AI, ‘there are eight billion people wondering why you cannot give them protection against being hunted down and killed by robots’.45 Eric Schmidt has, apart from in his work with Kissinger, articulated similar concerns about lethal autonomous weapons. He takes an entirely different political and ethical stance to Hinton and Russell, believing that the US must invest in AI in order to retain its supremacy—and to protect democracy and freedom. Yet he, too, sees the cataclysmic military potential of AI: Eventually, autonomous weaponized drones—not just unmanned aerial vehicles but also ground-based ones—will replace soldiers and manned artillery altogether. Imagine an autonomous submarine that could quickly move supplies into contested waters or an autonomous truck that could find the optimal route to carry small missile launchers across rough terrain. Swarms of drones, networked and coordinated by AI, could overwhelm tank and infantry formations in the field.46
Warfare will be automated. Drones and robots, controlled by algorithms, will dominate the battles of the future. The Campaign to Stop Killer Robots, Stuart Russell, and Eric Schmidt might exaggerate the potential of lethal autonomous weapons. It is striking, then, that many prominent security-studies scholars, while eschewing the language of slaughter-bots, have often concurred with their view. They too claim that AI will automate weapons, making war easier and more likely. For instance, in an important article, Jürgen Altmann and Frank Sauer observe: ‘Today’s unmanned systems have already increased the risk that military force will be used in scenarios where manned systems would previously have presented decision-makers with bigger, cautioninducing hurdles’.47 The anthropologist Lucy Suchman claims that states will use AI and autonomous weapons to prosecute dehumanised targets anywhere in the world at will: ‘These [AIenabled targeting systems] become ever more dangerous in the contemporary moment, as the figure of the ‘imminent threat’ is expanded into a horizon of anywhere and of endless war’.48 Consequently, these and other scholars call for the regulation of autonomous weapons.49 In their recent monograph on AI, Ben Buchanan and Andrew Imbrie describe AI as the ‘new fire’. For them, AI represents a potentially revolutionary development for the armed forces, and they draw a striking historical parallel: Humanity has also wielded fire’s destructive forces. The Byzantine Empire used it to great military success, first during the siege of Constantinople in 672 AD, and then in the centuries that followed. In battle, Byzantine troops shot a specially formulated compound at their enemies, one that would burn even when it came into contact with water. Once the compound hit the target, the power of fire would kick in, torching enemy equipment and causing soldiers to flee. Since then, the flames of war have become deadlier. Could there ever be another force so productive and perilous, one so essentially defined by the exponential growth of its core components? Welcome to the age of artificial intelligence.50
For Buchanan and Imbrie, AI is the equivalent of ancient Greek fire or the gunpowder weapons of late medieval Europe. AI-automated weapons will magnify the destructive power of weapons. Buchanan and Imbrie have suggested that with the help of AI, ‘missiles would fly to an area of concentrated enemy forces and hover. Each missile would release smaller munitions, and each of these would select and attack an enemy target’.51 In the last two decades, David Hambling has established himself as a leading expert on drones and remotely controlled systems. Like Buchanan and Imbrie, he has claimed that military automation is approaching. Autonomous drone swarms, in particular, will be revolutionary:52 ‘A swarm of ten thousand small drones could level a town […] A small perching drone could deliver multiple incendiaries the size of bats […] Acting together drones might bring down a bridge or skyscraper, but they could do more than that’.53 Kenneth Payne’s work on the strategic implications of AI was already discussed; Payne sees great potential for AI in weapons development too. In his view, AI will facilitate the rise of automated weapons; ‘warbots’, as he felicitously calls them in his most recent book. In his analysis of warbots, he discusses the now-famous AlphaDogfight experiment at the US Air Force Research Laboratory in 2016 and 2020, which tested AI in a virtual simulation of aerial combat. AI programs, which had used massive amounts of data to teach themselves the best aerial manoeuvres, flew fighter jets in simulated combat against a human pilot. Heron Systems’ Falco program proved successful in the trials in 2020. Displaying ‘superhuman precision in its flying and fighting’, Falco beat the human pilot 5–0. There were several reasons for Falco’s victory; one of these was that ‘the AI agent could pull manoeuvres that a human pilot simply could not physically withstand’.54 Another was that Falco calculated that the most effective way to attack an opponent was frontally: ‘The AI agent showed a strong favour for what pilots called forward-quarter gunshots, when the two aircraft are racing toward each other head-to-head’.55 Such an approach is extremely difficult and dangerous; human pilots tended to avoid it. Indeed, one pilot described it as ‘a gunshot that is almost impossible’. Many pilots
flinch when a plane flies at them. By contrast, Falco, experiencing no emotional response, fired its weapons coolly, no matter how likely the chances of a head-on mid-air collision. These simulated dogfights seemed to demonstrate that AI could automate aerial combat. AI could be quicker, more skilled, and more lethal than even the best human pilots. On the basis of the AlphaDogfights, many other commentators assume that soon AI will automate combat. In his recent best-seller, Paul Scharre, for instance, fears that military forces are developing autonomous weapons systems which will be able to identify and engage targets independently of human control: ‘Militaries around the globe are racing to deploy robots at sea, on the ground, and in the air—more than ninety countries have drones patrolling their skies. These robots are increasingly autonomous and many are armed. They operate under human control for now, but what happens when a Predator drone has as much autonomy as a Google car?’56 James Baker claims that because AI has the ability ‘to outperform humans in pattern recognition and anomaly detection’, it will soon direct weapons independently of human control.57 John Antal confirms the point. He has claimed that the Second NagornoKarabakh War was ‘the first war won primarily by robotic systems’. The future, for him, is clear: ‘When these [autonomous] systems are connected into a network and form a multi-domain strike capability that leverages the synchronization in time, space and effect with artificial intelligence (AI), the ability for anyone or anything to hide in the battlespace will become much harder, if not impossible’.58 Similarly, Seth Frantzman claims that once drones are AI-enabled, war will start ‘to look a lot more like a computer game’.59 In the 1990s, John Arquilla, with David Ronfeldt, made an important intervention into discussions about the evolution of war, claiming that ‘cyber war was coming’. Arquilla was impressed by the Revolution in Military Affairs in the 1990s, when the US harnessed the potential of new surveillance systems, digital communications, and precision munitions. These new systems would soon allow US forces to coordinate seamlessly with each other, converging on decisive locations on the battlefield. Arquilla has been similarly
impressed by AI and the potential of military automation. For him, AI will accentuate the trends of the Revolution in Military Affairs. Robots and drones will replace humans: ‘Future battles between advanced forces will be incredibly fast-paced, replete with weapons empowered by artificial intelligence and coordinated to strike in networked “swarms”’.60 The ethicist and philosopher Ronald Arkin has developed an unusual position in these debates. He, too, claims that lethal autonomous weapons will proliferate to become extremely important. However, he welcomes the development. He claims that because their judgement is motivated not by fear or hatred but by reason, AI will make decisions more ethically than human combatants would. AI will not kill unnecessarily.61 Nevertheless, he still believes that autonomous drone swarms will dominate the battlefield of the near future. A consensus is developing across the study of security, armed conflict, and war. In a field which is typically riven with debate and disagreement, it is surprising that so many scholars and commentators have eventually converged on essentially the same position regarding the military application of AI. Despite the wide divergence in their political and critical viewpoints, Henry Kissinger, Ken Payne, David Hambling, Roberto Gonzalez, Denise Garcia, Jürgen Altmann, Frank Sauer, and many others believe that AI is about to automate war—or significant parts of its prosecution. AI is about to displace humans to make strategic decisions as to when and how to go to war. AI will increasingly direct weapons, killing people independently of human control. It is a troubling vision of the future. AI Scepticism The concerns of Kissinger, Hinton, Russell, Payne, and others are not baseless. On the contrary, these authors have good reasons to argue the way they do. It is absolutely true that, today, states are actively seeking to harness the power of AI for military advantage. China, for instance, has announced its intention to become the world leader in AI by 2030. Its ‘New General AI Plan’ proclaimed that ‘AI is a
strategic technology that will lead the future’.62 China is determined to have the world’s premier AI-enabled military within a decade. Similarly, the Russian president Vladimir Putin declared, ‘Whoever becomes the leader in this sphere [artificial intelligence] will become ruler of the world’.63 Although Putin has suffered a terrible setback in Ukraine, there is little doubt that he and his successors will attempt to enhance the Russian armed forces with AI as quickly as they can. In response to the challenge posed by China and Russia, in 2014 the US committed to a ‘Third Offset Strategy’. The US has invested heavily in AI, autonomy, and robotics to sustain its advantage in defence and will continue to do so. Some have declared that the US is in an ‘AI arms race’.64 Indeed, Alex Karp, the CEO of Palantir Technologies, a leading tech defence company, went further: ‘The power of advanced algorithmic warfare systems is now so great that it equates to having tactical nuclear weapons against an adversary with only conventional ones’.65 In September 2018, the Defense Advanced Research Projects Agency (DARPA) announced a $2 billion campaign to develop the next wave of AI.66 The US Department of Defense issued its AI strategy in 2019, accompanied by a major increase in AI funding;67 in 2024, the Department of Defense budget for AI was $1.8 billion.68 The US has established the Defense Innovation Unit and the Joint Artificial Intelligence Center to germinate, accelerate, and enhance its armed forces’ AI capability. Smaller states are equally committed to the military development of AI. The UK and Israel, for instance, are developing their AI capabilities. AI has become an existential security question which no serious military power can ignore any longer. It is becoming as central to defence policy as aircraft carriers, tanks, or atomic bombs were in the twentieth century. Today, of all the automated and robotic systems being developed for military usage, the drone swarm has attracted by far the most attention and seems to show the most potential. The trajectory of the drone, or the uncrewed aerial system (UAV), over the last two decades is remarkable. The drone first began to be commonly used by the US in 1999, as a surveillance system; by 2024, it was a ubiquitous weapon, used by almost every combatant on a daily
basis. There have already been notable developments in autonomous swarming. In October 2016, the US Department of Defense demonstrated a swarm of 103 Perdix micro-drones capable of ‘advanced swarm behaviours such as collective decision-making, adaptive formation flying and self-healing’.69 The Chinese have also made significant advances in swarm intelligence. In 2017, a formation of a thousand UAVs flew at Guangzhou Airshow, and China Electronic Technology Group flew a swarm of 119 fixed-wing drones.70 The US Army has procured and tested the TSM-800 drone swarm, manufactured by Booz Allen. At Fort Irwin, California, in recent trials in 2023, operators successfully flew a preprogrammed swarm of ninety-seven TSM-800 drones to attack a designated target; one human controller oversaw the attack remotely (with the capability of aborting the mission), but the swarm was essentially autonomous. The swarm was divided into five subgroups of twenty drones, programmed to attack on different vectors, so that it was more difficult to defend against them.71 The US Navy has tested super-swarms which look and fly like flocks of birds in order to deceive enemy radar. The possibility of automated drone swarms controlled entirely by AI is evident. No one should doubt the military utility of AI, then. Yet it is easy to be entranced by AI. AI has, after all, made extraordinary advances in a very short time. No matter how tempting it is to enthuse about AI, though, scepticism is in order. Current predictions about AI are more fragile than they appear. Hinton, Russell, Kissinger, Schmidt, Payne, and other experts project a vision of the future based on their understanding of AI today. Yet they take a relatively narrow view of AI, examining only a few exceptional cases; there is little discussion in their work of the difficulties and shortcomings of AI. In addition, they propose the most extreme future scenarios, on the presumption that further major strides are inevitable and obvious. There are serious epistemological dangers to prognostications of this type, especially in a field as empirically complex as war. Many scholars have been too quick to draw causal conclusions about AI and the inevitable automation of war. They predict an AI military revolution on the basis of thin, narrowly
selected evidence which supports only the case for automation while ignoring the limitations of AI and the difficulties of applying it to strategy, to war, and to warfare. Indeed, there is a tendency towards circularity in contemporary work. Because these scholars presume the future of AI, they read the evidence about the performance of AI in the present in only one categoric way, which, they claim, leads to that inevitable, already assumed future. It is a pure case of teleology. Recently, some scholars have begun to question some of the presumptions which have become so established in the debates around security. Rather than advocating a single AI future, they have highlighted the limitations of AI and the difficulty of applying it to military operations. For instance, in an important recent article, American security-studies scholars Avi Goldfarb and Jon Lindsay have punctured the hyperbole around AI, saying that ‘AI, from this perspective, is not a simple substitute for human decision-making’.72 They fully recognise that AI is capable of better, faster, cheaper statistical prediction than humans are. AI has consequently proved highly successful in the commercial world, allowing companies to predict customer demand and market trends with striking accuracy. There is no doubt AI will be useful to the armed forces. Nevertheless, Goldfarb and Lindsay stress the distinctiveness of military operations: ‘the conditions that have made AI successful in the commercial world—quality data and clear judgement—may not be present or present to the same degree for all military tasks’.73 In the commercial sector, markets are relatively stable; demand is predictable. The data on which companies make their decisions is generally clean, reliable, and adequate. Rival companies are serious competitors, but their actions, too, are broadly predictable, operating from within regulatory parameters. Not so in war: ‘In contrast with assumptions about rapid robot wars and decisive shifts in military advantage, we expect AI-enabled conflict to be characterized by environmental uncertainty, organizational friction, and political controversy’. The authors conclude, ‘War, by contrast, occurs in a more anarchic environment’.74 In war, data will, therefore, be incomplete, messy, and inaccurate. Moreover, the enemy will actively
seek to corrupt and poison data: ‘The importance of data and judgment creates incentives for strategic competitors to improve, protect, and interfere with information systems and command institutions’.75 Moreover, the military decision-making process cannot be reduced to statistical prediction; it is not reducible to an algorithm. A command decision is a complex process. Commanders do not just order a weapon to fire at a threat. They have to define a mission, in which all their forces and all their weapons are organised and oriented to a single goal. Commanders, therefore, must consider many different factors before they make a decision. They have to understand the situation; they must comprehend what they have been directed to do by political leaders. Balancing that direction with a variety of military, civil, and political stakeholders, they must work out what is possible, not only militarily but politically. No matter how impressively it processes data, AI does not possess the judgement that underlies decision-making.76 ‘AI will alleviate some of the data processing burden’, Goldfarb and Lindsay allow, but, in war, human intelligence will remain critical. Indeed, AI, data, and machine learning will make ‘human beings even more vital’.77 Goldfarb and Lindsay are not alone in their scepticism about AI. In a closely related article, Benjamin Jensen, Christopher Whyte, and Scott Cuomo also take a sceptical view of AI. They fully recognise the potential of AI for military affairs, as AI can perform and indeed has already performed a variety of useful military functions. They acknowledge that ‘deep learning has the potential to create combatadvising software agents that anticipate both the natural and human environment, offering predictions about enemy actions’.78 AI could prove very useful in military logistics; it could anticipate supply needs, thereby revolutionizing military readiness. It could simulate defence scenarios to improve reactiveness and decision-making. Alternatively, ‘AI advances have the potential to perform a wide range of intelligence tasks faster and with higher accuracy than human analysts’.79 There are many military practices to which AI might be usefully applied. However, the authors also highlight the operational limitations of AI. War is a complex, bewildering phenomenon: ‘As a nonlinear system, every battle and campaign is
contingent and subject to emergent properties’.80 On contemporary battlefields, civilians, friendly and enemy forces are often intermingled and indistinguishable from one another in blasted, ruined urban areas. War is an agonistic enterprise: ‘The enemy gets a vote, producing a complexity unique to war. Every change to military capabilities—the hardware—and their battlefield employment through new concepts and organizations—the software—is subject to a corresponding reaction’.81 The smallest bias or gap in the dataset would generate egregious targeting errors. AI would be extremely susceptible to errors of targeting in the confusion of a battlefield:82 ‘Consider what would happen if military intelligence professionals entered into a similarly flawed image recognition system hundreds of pictures of adversary fighters assessed to be located in an urban area filled with hundreds of thousands of noncombatants’.83 The risks are obvious. An AI agent might easily target civilians or friendly fighters; more likely, it might simply stop functioning at all. AI is powerful, but it is also limited. It is very unlikely, according to Jensen, Whyte, and Cuomo, that combat could be completely automated. As a result, the authors dismiss the utopian vision promoted by so many other commentators and scholars. They do not see AI taking over: ‘AI does not yet promise to change states’ abilities to prevail in major conflict’.84 In an indignant recent article, Cameron Hunter and Bleddyn Bowen have made a similar argument and rebutted the claim that AI could ever supersede human commanders. Because AI has been successful under closed conditions, they explain, AI proponents describe war as a similarly prescribed system: ‘Decision-making in war under this implied vision is within a closed, rule-based system […] Conceiving of war as a kind of game or closed system allows AI optimists to envisage a future in which AI will be able to make or advise on command decisions’.85 Hunter and Bowen vehemently disagree with that view; war is an open, complex—indeed, chaotic— environment. Strategy, command, and military decision-making, therefore, require more than mere calculation: ‘Command in strategy and tactics requires abductive logic—an ability to think and make decisions based on the constant presence of unknowns and
unknowable things that may never appear in a historical dataset or past experience’.86 Strategists need a subtle awareness of other actors and the range of factors at play as a state moves to war: ‘AI currently cannot make judgements, but rather makes probabilistic inferences. Nor can it make useful decisions in the absence of comprehensive data in a closed system’. It is, therefore, difficult to see how second-generation AI could automate military decisionmaking—much less war more widely. It will be a good deal more difficult for AI to automate war than many scholars presume. There is much evidence to support the arguments of sceptical scholars like Goldfarb and Lindsay. For instance, there is a common error in much of the literature about the application of AI to military affairs. Many AI advocates extrapolate from military simulations that make use of AI to presume that the same situation would pertain on the battlefield itself. On this account, the evidence from simulations transposes immediately onto the battlefield; what happens in virtual reality will soon inevitably happen in reality. The heavily referenced AlphaDogfight trials illustrate the problem of this evidential carelessness rather well. AlphaDogfight has been recurrently cited by AI advocates to prove the superiority of AI over human pilots. In the simulations, the AI pilots won. On this basis, it is presumed—by Kenneth Payne and by Kissinger, Schmidt, and Huttenlocher—that AI will soon fly real planes in combat more successfully than human pilots can. Yet the conditions in those trials were vastly in the favour of AI: ‘The AI agent [Falco] was given perfect situational awareness of the simulated environment, including the location of the opposing fighter’.87 In addition, the human pilot was constrained in a way which Falco was not. In training, human pilots are not permitted to conduct head-on shots; it is too dangerous to practice them in the air.88 And human pilots do not like taking head-on shots. However, in actual combat, human pilots might well adopt this kind of tactic. Trained on large amounts of pristine data from previous simulations, Falco performed supremely. Yet the real world is vastly more complex than the world of such simulations. Human pilots have to deal with weather— clouds, rain, wind, unusual lighting conditions, and so on—
unexpected enemy action, mechanical failures, human errors, airdefence systems, and more. They have to land and take off; they have to coordinate with their colleagues. Their mission changes. It is sometimes difficult to distinguish friend from foe. To an AI pilot, by contrast, even the smallest change to the environment might be confusing. US Air Force commanders know all this full well and recognise that, in reality, a completely autonomous aeroplane is improbable. They were certainly still interested by the results of the AlphaDogfight trials, but they saw the experiment as a way of improving the performance of human pilots by augmenting them with AI, rather than replacing them.89 The air force recognised that this trial was only a simulation in a virtual world. For the air force, it is important to distinguish between the virtual and the real. Yet, in many discussions of AI, evidence taken from simulations is assumed to apply immediately in the real world. A recent furore surrounding the US Air Force demonstrates the fallacy with even more force. In May 2023, Colonel Tucker ‘Cinco’ Hamilton precipitated a Twitter storm when he seemed to claim that, in a recent exercise, a rogue autonomous drone had attacked its own command post. It was reported that the drone had been unable to find an enemy headquarters and, therefore, logically following its algorithms, attacked a friendly one instead. Many people took this incident as proof that military automation was imminent. They presumed the incident was real. Hamilton later admitted that he had ‘mis-spoken’. The episode had not really happened at all; it had occurred within a simulation. Although the US Air Force is certainly experimenting heavily with AI—with autonomous and quasiautonomous aircraft—the replacement of piloted combat planes with completely autonomous ones is unlikely. As Bill ‘Evil’ Gray, a test pilot, observed: ‘We are trying to figure out how to integrate artificially trained neural networks, trained in a simulation[, …] into the real world’.90 That is not easy. Prophecies about the imminent AI revolution in military affairs are overstated and under-evidenced.
AI at War Is AI about to automate war? This question is the central theme of this book. In order to address this issue, I focus on recent and contemporary military practice, rather than projecting into the future. Specifically, I answer two subordinate questions: first, in the last two decades, how has AI been employed in military operations? Second, how have the armed forces reorganised themselves in order to exploit AI? I eventually address a third issue: how has AI changed the character of war in the last decade, and, consequently, how might it change the character of war in the next ten years? The method is deliberately historical; it looks to the past and present. It looks at how militaries have actually applied AI to their activities and operations in the recent past and how they are planning to use AI in the near future. Their plans cannot be taken as a reality in themselves, though they may be organisationally relevant for the present practices. I try not to speculate about how AI might be used or how it might change war and warfare ten or more years from now. In focusing on the past—and therefore on actual evidence—I adopt a sceptical, empiricist approach to AI. I consciously follow the philosophy of the eminent Scottish philosopher David Hume here. A great deal of contemporary scholarship on AI presumes the future. This is a problem, because there is no evidence about the future. So, any prediction, however plausible it might seem, can be only speculation in the proper philosophical sense. Hume highlighted the dangers of prediction over two hundred years ago from his position of ‘determined scepticism’. In a famous passage in his Treatise of Human Nature, he showed that cause and effect, so often presumed by philosophers and theologians, can never actually be assumed. He considered the example of billiard balls striking each other. Because billiard balls have collided in the same way in the past, observers naturally presume that they will interact in the same way in the future.91 Although practically—and empirically—it is correct to assume this eventuality, there is no logical necessity that the balls should strike each other as they have before. Philosophically, there is no necessary bridge forward from the
present to the future. In any future case, anything might happen; factors of which we were ignorant might suddenly influence events. Cause and effect are not inevitable or obvious. Humans infer necessary cause from seeing events repeat themselves regularly; they presume a certainty to which they are not entitled. Consequently, Hume famously concluded, ‘We have no other notion of cause and effect but that of certain objects which have always conjoin’d together and which in all past instances have been found inseparable’.92 In the future, even the most apparently ineluctable causal links might not operate. The future development of AI and its application to war is far more indeterminate than billiard balls colliding on a flat baize-covered table. In this book, I try to avoid prediction and prognostication. Instead, I consciously look backwards to what has actually happened. I examine military developments in the present and the recent past. I look at how the armed forces have sought to adopt AI, to train with it, and to apply it to military operations. Recent wars are plainly a vital part of the evidence base. Since 2001, conflicts have proliferated and intensified in Ukraine, Georgia, the Middle East, Afghanistan, the Sahel, sub-Saharan Africa, and South-East Asia. War has been a constant, and the amount of potentially relevant material is vast. In particular, the Russo-Ukraine War is of prime significance; it continues to generate nearly endless, often surprising, evidence about war in the twenty-first century, defying many predictions. For instance, as General Mark Milley, the chair of the US Joint Chiefs of Staff, claimed, the Russo-Ukraine War has delivered a unique insight into the potential of AI for war: ‘We are witnessing the way wars will be fought, and won, for years’.93 In a speech to the Royal United Services Institute in November 2022, General Sir Jim Hockenhull, the head of the UK’s Strategic Command, discussed the Russo-Ukraine War and AI at length. He used the conflict as a way of illustrating the growing importance of AI, data, and open-source intelligence, declaring, ‘The conflict in Ukraine can in some ways be viewed as the first digital war’.94 The war has involved an explosion of data. The wars in Nagorno-
Karabakh and Gaza are also immediately relevant for understanding the military application of AI. The recent operations of the Israel Defense Forces (IDF) and the current war in Gaza are also instructive. In May 2021, the IDF conducted an eleven-day campaign, called Operation Guardian of the Walls, against Hamas in Gaza. It was described as the ‘first AI war’, as AI was employed extensively to facilitate targeting. Following the attacks of 7 October 2023 (see chapter 10), Israel has been engaged in a major war with Hamas. The campaign has been brutal, with many thousands of civilians killed and hundreds of thousands more displaced. Nevertheless, the IDF have once again drawn on AI to help them target Hamas militants. The war in Gaza and the Russo-Ukraine War may be a turning point in the history of war. They may mark the moment when AI first began to be indispensable to military operations. These wars may disappoint the AI proponents, though. There is no sign in Ukraine that AI is about to take over, despite both sides’ profligate use of drones and loitering munitions. On the basis of the evidence from Ukraine, AI will not automate war—that is more fantasy or science fiction than reality. Nevertheless, the war in Ukraine has categorically demonstrated that AI has indeed become crucial to military operations. Although President Zelensky, General Zaluznyi, General Syrskyi, and their subordinates still make all the decisions for the Ukrainian military, AI has played a crucial role, harvesting intelligence from a great quantity of diverse data. AI algorithms have helped the Ukrainians to plan and helped them to target the enemy. It has enhanced their military capability. The war shows the salience of AI in contemporary warfare. This connection is likely to deepen in the next decade. AI is becoming more potent every week, and the armed forces will draw ever more heavily on it. Even if we dismiss the apocalyptical claims about military revolution, AI will inevitably continue to reconfigure warfare. Like gunpowder, railways, telegraphy, automobiles, aeroplanes, wireless, and nuclear weapons, AI will inevitably have a major impact on the way wars are fought now and in the future.
To this end, it is necessary to examine how the armed forces are actually harnessing AI for military operations. Most major military powers are already trying to use AI. A global survey of all these powers would be welcome. Yet a complete survey of how every military force is using AI would be impractical. The empirical focus of this book is, therefore, deliberately circumscribed to achieve a level of evidential adequacy. However, while it is impossible for me to be comprehensive, it is useful to employ a comparative method. In their excellent recent work on technology and civil-military fusion, Yoram Evron and Richard Bitzinger use comparison to great effect.95 They select four case studies—the US, China, Israel, and India—to show how these states have differentially adopted or failed to adopt new military technologies. The comparisons provide a better understanding of the process in each state, as well as the general pattern of change. I have followed Evron and Bitzinger’s method here, adopting a comparative approach focusing on examples from the US, the UK, and Israel. Because the armed forces of these states are Western or Westernised powers, it has been easier for me to gain access to them than it would be to gain access to those of Russia or China. There are also good substantive reasons for focusing on these three powers. The US and Israel are pioneers in the application of AI to military operations. They provide excellent evidence about the military application of AI. And despite the small size of the British armed forces, the UK remains a major European power; as such, it is a leading proponent of the military application of AI. France, Germany, and other European countries are certainly looking to employ AI, but the UK usefully stands as an example of how a medium-sized NATO member is adopting this technology. The evidence presented here is certainly not comprehensive. In the following chapter, I discuss AI as a technology. However, a major part of the analysis examines not AI as a discrete technology but rather the way in which the armed forces have reorganised themselves in order to be able to employ AI. This is a vital and often under-appreciated issue. AI has not simply automated war or the armed forces, nor will it. In order to exploit AI, the armed forces have already begun to reform their organisational structures and
practices. Profound institutional reconfigurations are occurring. The organisational transformations are just as important as the technological developments, because without those alterations in human organisation, it is impossible to use AI. The armed forces are, therefore, changing their command hierarchies and the structure of their headquarters; they are altering their doctrine and practices. Above all, a profound organisational transformation is taking place. A new partnership between the armed forces and commercial tech companies—such as Google, Amazon Web Services, Microsoft, SpaceX, Palantir, and Anduril—is appearing. In this book, I plot the emergence of this new relationship between the armed forces and tech companies. The armed forces have, of course, long been dependent on the private sector. In the twentieth century, and especially during the Cold War, private arms companies were contracted to produce weapons and platforms for the armed forces. A military-industrial complex developed. Since the 1990s, private military and security companies have been contracted to perform specific services in support of the armed forces; for the most part, they have provided dining facilities, technical support, and close security. Occasionally, they have provided combat forces—traditional mercenaries. Outsourcing has become a major feature of the defence sector. The relationship which is crystallizing today between the armed forces and tech companies is different. Tech companies are not providing the armed forces with pristine platforms or weapons. Neither are they supplying peripheral support services. They are providing software, data, and expertise. In addition, they do not merely deliver these AI-enabled services and then leave it to the armed forces to apply them—on the contrary, software and data are immediately related to current operations and need constant revision. Consequently, to harness AI, tech companies are being integrated into the armed forces and into military operations themselves. They are actively partnering with active military forces and deploying their employees forward into operational headquarters. There, the civilian data scientists, programmers, and coders are integrating with military personnel. The pursuit of AI is
thus precipitating a major organisational restructuring. A hybrid private sector–public sector, civil-military configuration—a militarytech complex—is emerging. The appearance of this strange new complex is of profound significance not just to warfare but also to civil-military relations. The rise of a military-tech complex raises serious political, legal, and ethical questions which are equally as vexing as current debates about military automation. The problem is not that computers are about to take over strategy and war but that private-sector tech companies are increasingly influencing the conduct of war. It is already possible to see the emergence of a military-tech complex in Ukraine. In order to harness AI, the Ukrainian armed forces have relied not only on traditional allies, such as the US, but also on close partnership with private-sector tech companies; they have needed the support of Google, Microsoft, Starlink, Palantir, and Anduril. They have fought the Russian invasion with the assistance of tech companies which have provided them with the data, the AI, and the software to be able to execute operations effectively. As General Hockenhull himself noted, ‘Much of that digital capability is coming from commercially available services rather than necessarily traditional military capabilities’.96 ‘Commercial networks’ have been a ‘force multiplier’. For instance, ‘The availability of commercial satellites has enabled an extension of reach in the Ukrainian military’s situational awareness and their ability to conduct surveillance and reconnaissance. We’re seeing artificial intelligence used alongside commercial software applications to increase the speed of action.’97 It is vital that we recognise and try to understand this militarytech complex, especially since, as we have seen, so much of the literature has fetishized AI as a technology, ignoring its organisational aspects. It is also necessary that we acknowledge that any account of the military application of AI and the military-tech complex can be only preliminary. The armed forces are only just beginning to employ AI. The military application of AI is a very new development, one that in most cases has transpired in the last five years. The armed forces and tech companies are at the very
beginning of a profound transformation. Studying AI may, therefore, have some equivalence to studying the genesis of strategic bombing forces or tank warfare in the 1920s and 1930s. The potential of AI is clear. The outlines might be visible, but we are examining a volatile process, not a stabilised institution. Analysing the process of construction is always far more difficult than understanding the finished edifice. In the case of AI, it may be even more difficult. The military application of AI is diffused across a transnational organisational complex. The network is still crystallizing. Finally, AI and the military’s use of it are developing so rapidly that it is nearly impossible to map the landscape with complete confidence. Even the AI pioneers themselves have been staggered by the pace and scale of the changes—as Hinton’s and Russell’s warnings illustrate. Consequently, this book is avowedly provisional. In it, I describe the application of AI to recent military campaigns, especially in the Russo-Ukraine War, and I take examples from US, British, and Israeli armed forces as they try to apply AI to their operations. Only at the end of the book do I offer some tentative predictions about the future trajectory of military transformation and therefore the likely character of warfare between AI-enabled, digitised militaries. Yet, for all my efforts to adopt an empirical method and to limit my claims to what is empirically verifiable, I must allow that even if the picture I depict in this book is broadly accurate for now, it may be overtaken by events. No one knows, for instance, how quantum computing will transform AI and therefore military operations too. However, AI is an existential security issue. Scholars are duty-bound to analyse its development and its implications as best they can. Although this book must be only preliminary, offering conditional findings, it seems imperative for me at least to proffer some interpretation about the military implications of AI. It would be a dereliction of duty to do otherwise.
2 What AI Can Do Is AI really about to automate war? In order to assess the probable influence of AI on war, we must establish what AI can actually do today. The question is whether contemporary AI, as it is currently constituted, might be able to precipitate a revolution in automation as its advocates suggest. Of course, it is possible that, in the future, AI will develop quite new powers. Perhaps, as a result of quantum computing or some other development, the powers of AI will be transfigured. Nevertheless, it would be mere speculation to profess a revolution in warfare on the basis of an AI which does not yet exist. The question is, will AI, as we currently know it, be able to automate strategic, operational, and tactical decision-making and coordinate and control weapon systems autonomously? To understand how AI might be applied to war, we therefore need a clear definition of what AI is and what it can do. The Origins of AI It is not easy to define artificial intelligence, not least because there are many types of artificial intelligence. AI is a field, rather than one specific thing. The field of AI includes planning, reasoning, searching, knowledge representation, machine learning, deep learning, probabilistic learning, decision trees, and evolutionary algorithms. Machine learning, a major sub-field, consists of three methods: supervised, reinforcement, and unsupervised learning.1 However, as a field, artificial intelligence is unified by a broadly shared capacity. Consequently, many scholars use a performative
definition for AI: AI refers to computer programs which do things usually associated with intelligent beings. On the basis of this performative definition, AI might be said to refer to computer software programs which manipulate data, independently of human direction, to produce unprogrammed but still coherent and useful results. AI sees, correlates, and analyses data to draw conclusions which humans sometimes cannot. Because it processes data to produce unprogrammed results, it seems to be intelligent. Artificial intelligence itself is not new. The prospect of artificial intelligence has existed for nearly two hundred years. From the 1830s until his death in 1871, the mathematician and inventor Charles Babbage, with the help of Ada Lovelace, worked on an ‘analytical engine’. Widely considered to be the first programmable computer, this invention was a rudimentary form of artificial intelligence. Genuine artificial intelligence emerged, following a series of technical developments, in the 1930s. In the Second World War, the mathematician Alan Turing and his colleagues at Bletchley Park, a top-secret code-breaking headquarters, developed a massive and powerful computer, Colossus, capable of deciphering the German Enigma code. Enigma was a completely random code, changed each day, that scrambled the letters in military communications. It could not, therefore, be broken by a human, and certainly not within twenty-four hours (i.e. before the code was changed again). No individual could possibly go through the hundreds of thousands of possible permutations in a single day. However, since in the German language certain letters and words are more common than others, some priming was possible. Employing vacuum tubes to perform Boolean operations, Colossus was able to process thousands of possible configurations quickly. It was, therefore, eventually able to decipher Enigma—purely on the basis of mathematical probability. Primitive computers were also employed in a number of weapon systems in the Second World War. For instance, the US Army Air Force’s Norden Bomb Sight was a partially automated, computerised system. In the 1940s, prototype computers were in use. In 1946, the Electronic Numerical Integrator and Computer (ENIAC)—an
eight-foot-tall machine of eighteen thousand vacuum tubes capable of three hundred operations a second—was completed at the University of Pennsylvania. ENIAC was used in the development of the hydrogen bomb.2 It was an important precursor to the modern computer. In 1947, Bell Labs invented the transistor, a semiconductor creating ‘logic gates’ to perform calculations. The transistor was foundational to the digital age.3 In 1948, Norbert Wiener, taking note of many of these technical developments, published his famous book on cybernetics, the art of steering. A seminal text on the possibilities of computing and artificial intelligence, it was an essay about the potential of automated feedback systems to revolutionize technology. Babbage’s analytic engine, Colossus, ENIAC, and cybernetics were important developments that eventually facilitated the rise of what we now call AI. However, the field of artificial intelligence was officially inaugurated in the 1950s. In 1950, Alan Turing published a famous paper, ‘Computing and Intelligence’, which considered the possibility of intelligent machines. In the 1930s, Turing had addressed the question of AI in two papers, in which he expressed scepticism that a machine might ever be capable of human intuition. Famously, in his 1950 paper, Turing seemed to reverse his position. He proposed a test to discover whether a machine could be classed as intelligent. He believed that a machine could be said to think if it could pass the ‘Imitation Game’—that is, if a human, on the basis of blind questioning, could not tell whether the responses came from a human or a machine. ‘Computing and Intelligence’ invigorated the field of computing and encouraged widespread discussion about the possibility of machine intelligence. It led to the famous 1956 Dartmouth seminar which is widely viewed as the origin of artificial intelligence. Between June and August 1956, the four eventual founders of AI—Marvin Minsky (from MIT), John McCarthy (from Stanford), and Herbert Simon and Allen Newell (from Carnegie Mellon)—as well as leading figures in mathematics and computing, John Nash, Julian Bigelow, and Ray Solomonoff—gathered at Dartmouth University. The term ‘artificial intelligence’ was first employed to describe the purpose of
the meeting to the invitees. The Dartmouth seminar consisted of a series of workshops over two months, in which the group concentrated on devising a way of programming a calculator to form concepts and to develop generalizations. Following the Dartmouth seminar, artificial intelligence developed as a dedicated research programme in several leading computer sciences departments in the US, especially at MIT, Stanford, and Carnegie Mellon, which remain leaders in the field today. The first wave of research into AI, from the late 1950s, was founded on a distinctive approach to the problem of machine intelligence. The philosophical—and technical—issues of artificial intelligence had been considered by the mathematician Kurt Gödel, Alan Turing, and other important figures in the field for several decades. Gödel and Turing proposed that machine intelligence should or would operate in a manner consistent with human intelligence. Gödel, Turing, and others assumed an account of human cognition which was consistent with dominant schools of analytic philosophy at that time. On this account, human thinking—and human language, in particular —was constituted by logic. The Vienna School, the logical positivists, Bertrand Russell, and the young Ludwig Wittgenstein all attempted to show how the operation of human language—and, therefore, meaning itself—was reducible to logic. Language divided reality up into a series of discrete values which could be organised into logical propositions. Meaning was constituted by these propositions. The job of the philosopher was to identify the logical mechanics of language and, therefore, to adjudicate between sense and nonsense. Consequently, it was assumed that machine intelligence would also operate on the basis of logic. The philosopher Brian Cantwell Smith has described the philosophical premises of the early AI pioneers as ‘neurophysiology, logic, and computation […] bundled together’.4 Operating on this ontology, AI that originated in the 1950s and 1960s has become known as first-wave AI or good oldfashioned AI (GOFAI). First-wave AI assumed ‘that the world comes chopped up into neat, ontologically discrete objects’.5 As the philosopher Hubert Dreyfus noted, ‘GOFAI is based on the Cartesian
notion that all understanding consists of forming and using appropriate symbolic representations’.6 First-wave AI operated on a deductive methodology, founded in logic. Operating with these ontological and epistemological assumptions, first-wave AI programmers assigned symbolic values to all the variables which GOFAI was going to process. GOFAI, then, manipulated these pre-coded symbols logically to produce solutions: ‘first wave (GOFAI) systems were built to entertain and explore the consequences of symbolically articulated discrete propositions implemented as formal symbols representing objects, properties, and relations in terms of a presumptively given formal ontology’.7 For instance, in 1957, Allen Newell and Herbert Simon sought to abstract the heuristics used in a logic machine and apply it to other, similar problems. They called this project the General Problem Solver.8 Simon was confident that the General Problem Solver would be successful. He boasted that, within ten years, ‘a digital computer will be the world’s chess champion[, …] a digital computer will discover and prove an important new mathematical theorem[, … and] most theories in psychology will take the form of computer programs.’9 Symbolic AI made some progress, especially on very narrowly defined problems, but it fell far short of its inventors’ aspirations. In 1972, fifteen years after Allen had predicted the General Problem Solver’s successes, the philosopher Hubert Dreyfus mocked its failings, saying ‘The decade is up and the computer is at best a class C amateur [at chess]’.10 It became clear that AI pursued on the basis of symbolic logic was of limited potential. In the 1970s, the research programme stalled, and an AI winter began that lasted for nearly two decades. Second-Generation AI In the late 1990s, the field of AI was reignited. The development of second-generation AI has been remarkable, and its achievements have been staggering. Second-generation AI operates on a quite different system to first-wave AI, though. It does not manipulate symbolic logic, pre-coded by programmers. It is an inductive system,
building statistical models generated from the data, not a deductive method, reaching conclusions through logic. Second-generation AI operates by means of statistical probability. It is connectionist. It plots recurring correlations between data points and then, on the basis of those correlations—tested and retested thousands, sometimes millions, of times—it generates a model. The more data a program has, the more accurate the model becomes. For instance, when Michele Banko and Eric Brill were developing their natural language program, they discovered that the raw quantity of data proved vital. ‘With a standard 1 million words of sample data, programs chose the right word about 82 per cent of the time’.11 With 10 million words, the success rate increased to 90 per cent or more; with 100 million words, 95 per cent; and with 1 billion words, 97 per cent.12 Second-generation AI has produced remarkable insights into data because it can plot almost an infinite number of correlations. Yet it cannot be said to ‘understand’ anything. It never identifies a causal connection; it identifies only statistically common conjunctions between data points. In order to develop a coherent inductive system, second-generation AI has required three critical enablers: data, software (algorithms), and computing power. Since 2000, as a result of the rise of digital communications, mobile phones, and the internet, there has been an explosion in data. The term ‘data’ refers to information which exists virtually in computer networks. Data ultimately consists of a series of binaries, of which every item in cyberspace and virtual networks is composed. Crucially, because data is numeric, consisting of ones and zeros, it is eminently computable. As Hal Varian, the chief economist at Google, has stated, ‘Between the dawn of civilization and 2003 we only created five exabytes of information; now we’re creating that every two days’.13 The production of data increased twenty-fold from 2010 to 2020. Currently 18 million gigabytes of data are created globally every minute.14 Almost all human activities—and many non-human ones— now leave a digital trace. An average internet user generates 1.7 megabytes of data a second, or 6,120 megabytes an hour; a family
can create about 506,736 megabytes daily.15 Every single transaction on the internet is recorded somewhere in cyberspace as data. Since almost every activity has left some kind of digital footprint which might be analysed by a computer program, radical new insights have been possible. As Alex Pentland, a professor of media arts and science at MIT and an AI expert, has articulated: ‘For the first time in history the majority of humanity is linked […] In short, we now have the capacity to collect and analyse data about people with a breadth and depth that was previously inconceivable’.16 Vast troves of data exist beyond the internet too. The results of this data revolution have been striking. So fertile is it that many have claimed, ‘Data is the new oil’.17 The major tech companies Google, Amazon, Microsoft, Facebook, Apple, Baidu, and Alibaba are all based on this explosion of data. Algorithms, some of which are relatively old, have also become very effective, allowing these companies to exploit the deluge of data. These algorithms have been employed to process data and to build models. Consequently, second-generation AI has operated in a different way to first-wave AI. First-wave AIs processed logical operations with a small number of symbols. They worked deductively. Sometimes, with deep learning, symbolic logic has played some role. However, the algorithms of second-generation AI have generally used three main forms of machine learning to build their models: supervised learning, unsupervised learning, and reinforcement learning. In supervised machine learning, programmers normally label the dataset for the algorithms, specify an outcome, and identify potential features: ‘In supervised learning, the programmer “trains” the system by defining a set of desired outcomes for a range of inputs (labeled examples and non-examples), and providing continual feedback about whether it has achieved them’.18 The streaming media company Netflix, for instance, has employed supervised learning with labelled datasets in order to classify people, who are deemed to be similar on the basis of their choices of movies and TV shows. On this basis, it is able to make accurate recommendations to its users.
In unsupervised machine learning, the program learns to structure the data for itself; ‘the user provides no desired outcomes or error messages’.19 On their own, the algorithms find natural groupings in the data—which is not seen by the programmers. This learning is ‘driven by the principle that co-occurring features engender expectations that they will co-occur in the future’.20 Reinforcement machine learning requires only a starting point and a performance function; it ‘is driven by analogues of reward and punishment; feedback messages telling the system that what it just did was good or bad’.21 It finds associations which accord with the reward. AlphaGo, discussed in chapter 1, operated by means of reinforcement learning. Second-wave AI has involved some major advances in software. There have been some critical developments in hardware, the most important of these being neural networks. Walter McCulloch and Warren Pitts began their work on neural networks in the 1940s. In the 1950s, Frank Rosenblatt developed a programme he called connectionism. Rejecting the symbolic logic of first-wave AI, he suggested that an inductive method might be more effective. He built a single-layer neural net called a perceptron. It consisted of transistors, organised physically into a layer of silicon chips, through which data was processed. The perceptron, as a neural layer, was intended to imitate the way the human brain operates as it processes information through the synapses.22 The breakthrough in neural networks came only much later, in the 1980s. At that point, it was realised that while one neural layer was insufficient, three (or more) layers might prove effective. In 1986, Geoffrey Hinton and David Rumelhart began to explore the potential of multiple neural layers. They created a ‘back-propagation’ algorithm as part of this work. Back-propagation referred to the process by which an algorithm might weigh and re-weigh data through its neutral layers. Gradually, the program was able to calibrate the data until it produced the correct answer.23 In the 1990s, there were advances in shallow neural networks. However, the real breakthrough came in 2012, when it had become evident that ‘deep’ neural networks consisting of many
layers might be much more powerful than ones that used only a few layers. Such networks could accurately process and weigh vast datasets with multitudes of parameters. Deep learning had arrived. Two years earlier, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) had been established to see whether computer scientists could develop a program which might recognise facial images autonomously. In 2012, two of Hinton’s students, Alex Krizhevsky and Ilya Sutskever, won the competition. Their deep learning model, AlexNet, was able to identify a range of digital images automatically and with a high degree of accuracy. As they developed this winning facial recognition program, Krizhevsky and Sutskever had realised that it was possible to use graphical processing units (GPUs) as part of deep neural networks; they were able to do the calculations. Krizhevsky and Sutskever used two GPUs to train AlexNet. GPUs proved central to the success of deep neural networks and have become ever more important to deep learning; programmers now use thousands of GPUs. The AI research company OpenAI has estimated that, to optimise its future large language models, it might need 1 million GPUs.24 It may be useful to give a very simple example of an AI program being trained to recognise images of dogs. Each image the program views consists of thousands of data points. The algorithm processes the data points which denote a canine appearance. Each data point has a mathematical value which is processed through the layers and correlated with thousands of other data points. The more images— and data—the program processes, the more parameters are identified and the more accurate these values become, until eventually the algorithm is able to recognise a picture of a dog with almost complete reliability.25 At this point, the algorithm has created a model, which processes any future data automatically. Neural networks might consist of many layers, increasing their processing capability and accuracy. By 2012, deep learning and neural networks had reached an inflection point, after which they became very effective. Indeed, the DeepMind team employed neural networks, back-propagation, and reinforcement learning to develop AlphaGo.26 Supervised, unsupervised, and reinforcement learning
have all been used by AIs to train themselves by trial and error. The more data that is processed through the neural layers, the more accurate the AI becomes. In addition to the data explosion and the development of algorithms, computing power has increased exponentially in the last two decades. Between 2012 and 2018, computing power increased by 300,000 times.27 The tech primes have constructed vast computing facilities to store and process their massive datasets. For instance, Amazon is estimated to have around two million servers. It has eleven cloud regions, which each have multiple data centres, totalling twenty-eight sets globally, some of which contain 50,000– 80,000 servers. Google has three regions, with eight sets. Microsoft has seventeen regions, with over a million servers. These servers store and process petabytes (1 petabyte = 1 quadrillion bytes) of data. In 2012, Facebook’s Hadoop stored 100 petabytes of data. In 2016, Google’s Tensor Processing Unit was able to conduct thirty times as many calculations a second as a GPU. Google’s OpenAI system is able to make 3,640 quadrillion calculations every second.28 Consequently, by 2018, it was possible for algorithms to work on huge datasets quickly. AI became practical for the first time. The Limitations of Second-Generation AI The capabilities of second-generation AI are remarkable. However, it is important that we recognise what second-generation AI cannot do. Second-generation AI is very different from first-wave, good oldfashioned AI. Second-generation AI, working through neural networks, operates inductively. A machine-learning program processes huge amounts of data through its neural networks until it generates answers which accord with the results a human programmer wants. The system is completely probabilistic and inductive. Algorithms know nothing. They are unaware of the real world and, in a human sense, unaware of the meaning of the data they process. They simply build models of statistical probability on the basis of reiterated trial and error. Machine-learning programs
identify statistical correlations in quantitative data. Most AIs do not recognise causation or intention. They do not ‘understand’ in a human sense; they have no innate comprehension of context. They are incapable of interpreting meaning. However, because AIs are trained on so much data, it is very likely that these correlations will prove accurate. Because it is an inductive system, second-generation AI is, therefore, constitutionally brittle.29 The remotest bias or gap in the dataset can generate egregious mistakes in the subsequent performance of AI. There have been several cases in which AI programs have produced poor results because of biases in the data. Infamously, some AI facial-recognition programs have recognised Caucasian faces quite accurately but—because there were so few Black faces in the data on which they were trained—classified African Americans as ‘gorillas’.30 And on 23 March 2016, Microsoft released Tay, a chatbot that had been trained on the interactions of users. An activist group flooded the chatbot with racist and sexist input. Because the offensive tweets were statistically frequent and Tay had no understanding of their meaning, Tay happily posted tweets, as it had been trained to do, that read ‘I fucking hate feminists’ and ‘Hitler was right: I hate the Jews’.31 The next day the project was cancelled. These incidents are extreme cases. There are more telling tests which show the potential weakness of AI. The Winograd schemas are perhaps the most instructive illustration of the fragility of inductive AI. Winograd schemas are intelligence tests which play on ambiguous syntax. For instance, one Winograd schema involves the sentence ‘I poured water from the bottle into the cup until it was full’. The test subject is asked to explain what was full. AI programs have struggled to answer these questions correctly. In 2013, Hector Levesque published a paper called ‘On Our Best Behaviour’, on the use of Winograd schemas to evaluate how close second-generation AIs came to passing the Turing test. Levesque had made small revisions to a series of Winograd schemas to produce questions like the following:
Joan made sure to thank Susan for all the help she had received. Who had given the help? a) Joan b) Susan Levesque found that AI programs, such as Google Translate, struggled to respond correctly, attaining only about 61 per cent accuracy on Winograd schemas. That success rate was little better than guessing.32 Comprehending a Winograd schema requires not just inductively knowing the statistical probability of the cooccurrence of two bits of data; an agent has to understand the context. For all its extraordinary potency, second-generation AI could not do that. As Oren Etzioni, director of the Allen Institute for AI, observed in 2017, ‘When AI can’t determine what “it” refers to in a sentence, it’s hard to believe it will take over the world’.33 Second-generation AI is based on statistical probability. Because it has so much data to work with and such vast computing power, it has produced remarkable results. Yet second-generation AI is fallible and limited. The fallibility of AI is, of course, of immediate relevance to its likely military application. Military operations, war, and combat are complex, confusing, and disordered environments. It seems intrinsically unlikely that an inductive and probabilistic system could be capable of taking over major military functions, such as decisionmaking. Its capabilities are too narrow. It is improbable that AI, as it is currently constituted, could automate war. Generative AI Second-generation AI is brittle. Will generative AI or large language models resolve this vulnerability? Do they have the potential to overcome the inherent limitations of probabilistic algorithms? The development of large language models has certainly been striking. On 30 November 2022, OpenAI launched its large language model, Chat Generative Pre-trained Transformation-3.5 (ChatGPT). In March 2023, OpenAI launched ChatGPT-4, another major breakthrough. By early December 2023, only one month after its launch, the platform had a million users; a month later, there were 100 million users.
Other companies, such as Anthropic, Meta (Facebook), Microsoft, and Google (Alphabet), have been working on large language models too. Google’s Gemini, Anthropic’s Claude, and Meta’s LLaMA have all proved powerful. Indeed, Google’s Gemini 1.5 AI model is able to ingest eight times as much data as ChatGPT-4.34 There is no inherent difference between large language models and other forms of second-generation AI. Large language models are founded in the same probabilistic statistical methodology; they use unsupervised learning, sometimes refined by reinforcement, but they are fundamentally the same. The difference resides in their architectures, which are distinctive; it is this architecture which gives them their power. Large language models employ deep learning neural networks to predict the next word, or token, in any sequence, on the basis of statistical patterns in their training data. Large language models are trained on a gargantuan dataset, more or less the whole of the internet; the models routinely learn from reading billions of words.35 It has been estimated that ChatGPT, for instance, has ingested 10 per cent of all human written language. To process this quantity of data, large language models are all built on a ‘transformer’ architecture. A transformer refers to an attention layer in the neural network which allows a model to focus on a specific part of the data, plotting multiple relationships among that data. These models are able to compute a multitude of adjustable parameters which are weighted by the model. In 2019, ChatGPT-2 used 1.5 billion parameters.36 However, this proved insufficient and did not work well. The breakthrough came once the transformers exceeded 100 billion parameters. By 2020, ChatGPT-3 was able to work with 175 billion parameters.37 There are no firm figures for ChatGPT-4, but it probably uses about 1.8 trillion parameters, perhaps divided into eight ‘experts’ of 220 billion parameters each. To conduct all these calculations, large language models require truly vast amounts of computing power. Accordingly, in the past decade, computing power—quantified as a floating-point operation (FLOP)—has multiplied exponentially. The computing power of the best AI models has increased by nine orders of magnitude, from two peta-FLOPs to ten billion.38 ChatGPT-4 uses forty times as many
FLOPs as 2012’s AlexNet. As a result, large language models can learn the relationships between words, phrases, and even paragraphs to hitherto unachievable levels of accuracy. Consequently, these models can generate responses on almost any topic, delivered in prose that imitates human speech. Since large language models depend on massive computing power and numerous chips, they are prohibitively expensive, costing $100 million to train.39 Future iterations may be even more expensive. The scale and scope of large language models has been remarkable. ChatGPT, for instance, is able to summarise arguments, articles, and books; to write songs; to draft lectures; to solve logic puzzles; to write computer code on the basis of natural language descriptions by programmers; to identify film plot summaries written in emojis; and to provide answers to examination questions. Winograd schemas had been a consistent problem for secondgeneration AI. Yet, in 2020, ChatGPT-3 attained 88.3 per cent accuracy on a Winograd test. It seems probable that as generative AI develops, it will be improved yet further, and its performance on these tests will one day be as good as that of humans. Generative AI has proved impressive in other areas, including specialist professional expertise, where its performance is improving rapidly. While ChatGPT-3.5 passed the bar examination in only the 10th centile, ChatGPT-4 passed in the 90th centile and has also passed the qualifying examination for neurosurgery.40 In April 2023, commentators lauded the model, saying ‘The technology exceeded human performance in areas that would have been unimaginable only a few months ago’.41 Alphabet (Google) claimed its large language model, Bard, taught itself to translate Bengali on the basis of only a few instructions. There is some dispute about this; but even if some training was involved, it is an impressive achievement. It is also ‘ludicrously easy to use’, as a writer for The Economist states: ‘Type a prompt and see a result’.42 Ben Tossell, a British AI entrepreneur, notes, ‘You can just open your laptop and write a few lines of code that interact with the model’.43 As a result of all these developments, second-generation AI seems close to passing the Turing test. Indeed, large language
models may have already done so. In many cases, they appear intelligent. In 1990, Hugh Loebner launched a challenge with a $100,000 prize for the first computer program which could pass the Turing test. However, by 2020, with the rapid advances in AI, the challenge was defunct. The Turing test is no longer a useful metric for assessing the performance of AI. Even though large language models remain founded in machine learning and probabilistic calculation, they often seem creative, even intuitive. Indeed, large language models have enjoyed stellar success. As one tech writer noted, ‘New “large language models” (LLMs)—the sort that powers ChatGPT […]—have surprised even their creators with their unexpected talents as they have been scaled up”.44 Mustafa Suleyman has been astounded by the recent advances of large language models, saying ‘The pace of progress in AI has been breathtaking’.45 The tech primes are investing in large language models. They believe that the capabilities of these models are so prodigious that the market for them will be vast. Microsoft, for instance, has invested heavily in the program. In February 2023, Microsoft added ChatGPT–like functionality to its web browser, Bing. In September 2023, Microsoft launched Copilot, an AI-powered app that is able to change a computer’s settings, generate images, and summarise web pages. Soon, Copilot ‘will sort through employees’ emails in Outlook and summon up information from their Word documents and PowerPoints’.46 Copilot is able to find and summarise files, to retrieve emails, and to create meeting agendas and PowerPoint presentations. Copilot works through Azure, Microsoft’s cloud. Microsoft executives believe that Copilot is so effective that it will help Microsoft regain its position of supremacy in the tech sector. Large language models may improve human productivity as much as computers themselves did. Copilot, for instance, will make offices and businesses more efficient and faster; vital pieces of information will not go missing. Copilot will perform or alleviate many tedious bureaucratic procedures for employees. Some processes may indeed be automated. Bureaucracies might become smaller; they should become better. The human employees should be liberated to interact
with and service the needs of human clients. Large language models have already proved extremely powerful in helping to conduct scientific research. The potential value of these models is therefore huge. For instance, in the second quarter of 2023, AI, including ChatGPT, increased Microsoft’s turnover by about $120 million. In 2025, Microsoft estimates it could earn another $40 billion, largely through Azure’s AI tools and Copilot.47 Large language models are a significant advance over other forms of second-generation AI, then. They seem to be intelligent; in some cases, their performance is remarkable. They are based on huge datasets—ultimately the whole internet—and have developed very considerable powers. Yet it is unclear whether they will overcome the fundamental limitations of second-generation AI. After all, they are trained on the same inductive system of statistical probability. In particular, large language models are as susceptible to hallucination as any other AI program.48 Because they process so much data, considering so many parameters, they routinely generate random findings. They find irrelevant, and erroneous, correlations in their data; these correlations may be mathematically correct, but they have no meaning. Large language models have no understanding of context in the real world; the data is the only context they see.49 So, they cannot recognise when they are wrong. Indeed, a model might give contradictory answers to successive prompts. As a result, some scholars have called ChatGPT and similar programs ‘stochastic parrots’.50 Egregious hallucinations are often easy to identify, but small mistakes which users do not notice are more dangerous. In addition, large language models are only as good as the data on which they are trained. Consequently, they are constitutionally biased by their training data: ‘The content of these datasets tends to reflect society’s prejudices’.51 For instance, in the field of recruitment, models replicate existing biases in favour of white workers and males; ‘the geographic location and ideological background of machine learning engineers and AI policy scholars also limits AI’s perspective’.52 Large language models may reflect the ideologies of people at Stanford, Berkeley, Cambridge, and Oxford. On certain
tasks, they may be considered intelligent. They are powerful tools, augmenting human expertise, but they are a long way short of displaying the practical intelligence which even the least intelligent human possesses. Initially, there were fears that generative AI would automate many jobs, displacing humans. Massive job losses would follow its introduction. In 2013, two Oxford University researchers, Carl Benedikt Frey and Michael Osborne, proposed that 47 per cent of US jobs might be automated by AI.53 In the light of the limitations of generative AI and its practical application in the last three years, those fears have receded. A 2023 International Labour Organisation report claimed that only routinised clerical work was at risk of automation; 24 per cent of those jobs were defined as ‘highly exposed’. The authors of the report concluded that generative AI was more likely to ‘augment occupations, rather than to automate them’.54 Still, as recently as 2023, Goldman Sachs predicted that 300 million jobs globally would be eliminated or degraded by artificial intelligence.55 It is improbable that commercial enterprises or administration, as a whole, will be automated by generative AI. Generative AI may be able to perform specific functions in support of humans, but it is unlikely to replace human workers entirely. It will increase human productivity rather than supplant it. Of course, many scholars fear that generative AI is a major step on the route to full military automation. Generative AI models will take over command of military forces; they will be able to strategise and to conduct war. This seems fanciful, even though these models are young and developing very quickly. If the commercial sector is any guide, it seems more likely that generative AI will help the armed forces to conduct operations, perhaps by automating some bureaucratic military and staff functions. Just as Copilot aids company executives, generative AI will surely assist in helping a commander to make decisions. Yet it seems highly doubtful that an AI program, even one as capable as a large language model, would be able to automate military decision-making or weapon systems as a whole, much less be able to fight a war. Of course, unlikely though
it is, there might conceivably, in an unspecified future, be a genuine revolution in the capabilities of generative AI. At that point, a superintelligent computer might direct killer robots to fight wars autonomously. Yet, since we can have absolutely no idea what such a scenario would look like, it is dangerous and unhelpful to speculate. We should focus on how the prodigious capabilities of large language models might be applied now and in the immediate future. Amazon In order to avoid speculation about the application of AI to military affairs, it may be useful for us to consider some concrete organisational examples. It is pertinent to examine successful organisations that have already applied second-generation AI to their operations. This exposition will show what AI can and cannot do. Walmart, Uber, Airbnb, Baidu, Google, and Tesla would provide good case studies. These companies have applied AI to real-world problems; they have delivered goods to consumers or booked taxis or hotel accommodation for travellers. In each case, they have employed AI to process data about the market, their customers, and their own operations. AI has allowed them to see patterns in their markets which in the past would have been difficult, even impossible, to discern. It has allowed them to profile individual consumers. In particular, data has allowed them to connect with, and to communicate to, customers directly. In this way, these and other AI companies have transformed a series of sectors. Big data and AI have been transformational, but have they actually automated these sectors as a whole? That seems much less clear. It is worth considering a specific example of the application of AI to one organisational problem, the example of the self-driving car. In his work on military automation, Paul Scharre cites the example of the Google car, the Waymo, as a possible model for future autonomous military platforms; like a military vehicle, the Waymo moves in the real world, finding routes to its destination.56 For
Scharre, since autonomous vehicles have already been introduced into the civilian sector, it is only a matter of time before combat drones, robots, or uncrewed vehicles appear on the battlefield too; automated war is imminent. This may seem plausible. Yet the reality of applying AI to the battlefield is much more challenging than is often appreciated. Google has developed Waymo through machine learning. Waymo has ingested thousands of hours of video footage of driving, as well as millions of images of roads, vehicles, and traffic signs. The car’s program responds automatically to each situation that arises. Because there is so much training data, Waymo operates well in many cases. Waymo can drive down normal streets to its destination —most of the time. However, Waymo remains vulnerable. Its weaknesses are deeply relevant to the difficulty of automating war. Waymo is entirely dependent on its data. It cannot identify any object which is not already in its database.57 The car ‘understands’ nothing; it does not have situational awareness in the human sense. It does not know— in the way that humans know—what a street is, what the traffic regulations imply, or what other road users might be trying to do. It just processes data. Consequently, because it cannot understand context or intention, when a Google car confronts conditions for which it was not trained—unusual traffic, poor weather, unpredictable pedestrians, ambiguous situations—it performs poorly.58 In March 2018, a self-driving Uber car killed a cyclist in Tempe, Arizona, because ‘there was a low but nonzero probability that a human was in its path’.59 It was simply following its data. Obviously, human drivers regularly make mistakes. However, selfdriving cars are even more susceptible. In 2017, self-driving cars were involved in one crash every fifty thousand miles; human drivers have one accident every 1.5 million miles,60 which leads to the conclusion that human drivers are thirty times safer. Self-driving cars have of course improved in recent years. Yet human drivers perform even better in unusual conditions. Humans are exceptionally good at responding to new situations, extrapolating from what they know to what they do not.
Even the billionaire Elon Musk, one of the most radical and famous technologists alive, has been forced to admit the limitations of AI. He created his car company, Tesla, with the aim of building a completely autonomous car which would outperform humans; Tesla’s Autopilot system was intended to drive the car for its human occupants. Musk was forced to reduce his ambition when a Tesla driver, taking Musk’s boasts about Autopilot too seriously, was killed when his vehicle collided with the side of a truck; the Tesla driver was watching a Harry Potter movie, not the road, and Autopilot did not notice a white tractor-trailer against the bright sky.61 Waymo and Tesla prove the difficulty of completely automating civilian driving. The history of automated cars suggests that organisations will employ AI where they can, but that full autonomy is likely to be limited. The challenges in the application of AI are affirmed by examining the operations of a whole company. Amazon is, of course, an excellent example here. It is particularly useful as an analogue for the armed forces because Amazon is a logistics company; or at least it was until 2020, when it was able to redefine itself as a data company. Like the armed forces, Amazon operates in the real world. It delivers physical packages to specific consumers across the world. Amazon organises the movement of its employees, its services, and its goods in order to supply individuals with their orders. The armed forces, of course, ultimately deliver violence or threat of it. Yet they too deploy personnel around the globe to engage adversaries and enemies with specific weapons. Like Amazon, the armed forces identify, target, and deliver a physical service. When Jeff Bezos set up Amazon in his garage in Seattle in 1996, his initial ambitions were modest. He wanted to create an online bookstore to challenge major high-street retailers such as Barnes and Noble. Bezos had no shop; buyers ordered books from him on the internet. He posted them out direct to his customers. Amazon rapidly acquired vast quantities of data about its customers in this way. By trading on the internet and employing data, Amazon reduced costs, fulfilled orders more quickly, and could identify
market trends early. The process was not easy: ‘The technology was broken all the time and because the technology was broken, the data was often wrong. We would bring it to Jeff Bezos and it was all contradictory and he would be yelling and screaming at us’.62 However, Amazon was eventually able to expand its customer base. It was able to eliminate the high-street shop and connect the customer immediately with the supplier. Amazon revolutionised retail by the use of data. The company broke down book purchasing into five steps, each of which it digitised. Amazon ‘created automated tools that allowed buyers to order merchandise based on dozens of variables such as seasonal trends, past purchasing behaviours, and how many customers were searching for a particular product at certain times’.63 Data, processed by AI, was central to Amazon’s retail revolution. Yet it is important that we understand the reality of Amazon as an organisation. Without question, Amazon has always been a datacentric company. Its operations are facilitated by AI. It could not have been as successful without AI as it has been. However, the company still relies on its human workforce. Indeed, the decisive elements of Amazon remain human. When it comes to strategic decision-making, management, the physics of Amazon’s logistics, and its actual supply of products, its operations are quite traditional. They have not been automated at all. Human employees, helped by data and AI, define the organisation. Amazon is managed by a group of senior executives, called the Steam. The executives are responsible for defined areas of activity, and they answer directly to Bezos. Strategic decisions are based on the company’s massive and rich datasets, processed by AI. Yet neither data nor AI determines S-team strategy. Some of the Steam’s most important strategic decisions are entirely political and have little to do with AI at all. For instance, Amazon developed a policy of leveraging its relationships with publishers to reduce prices for books sold on Amazon. Bezos and his executives targeted specific publishers, whom they then persuaded, manipulated, and sometimes simply bullied: ‘When a publisher did not capitulate and the company shut off the recommendation algorithms for its books, the publisher’s
sales usually fell by as much as 40 percent’.64 Executives also lobbied governments for favourable treatment in terms of taxation and transport. Because of problems with the delivery labour force, Amazon also formed alliances with UPS Next Day and FedEx Express. Eventually, the company developed its own air-transport division, Amazon Air.65 Not one of these decisions was automated; it is not easy to see how they could be. Indeed, Bezos himself emphasised the importance of human judgement and attention to strategic decisions. In an angry exchange with one of his executives, Roy Price, Bezos raged: ‘You are telling me we are making $100 million decisions and we don’t have time to evaluate whether they are good decisions?’66 The S-team meetings illustrate the importance of human actors to Amazon and the subservience of AI and data to those human actors. The S-team assembles regularly for major management meetings. These events, which are often chaired personally and aggressively by Bezos, are fundamental to Amazon’s activities. Even now, after the institution of Amazon Web Services (AWS), marking Amazon’s transition to a data company, these meetings remain crucial. Executives are expected to be in complete control of their departments, fully apprised of their data. As reported in Brad Stone’s Amazon Unbound, every Wednesday at AWS there is a ‘ninetyminute midday business review, where the top two hundred managers [discuss] the minute details of customers, competitive developments, and the financial health of each product unit’.67 This meeting is followed by a two-hour operations review assessing the technical performance of each web service; the forum is ‘the centrepiece of the week’.68 In it, more than forty vice-presidents and directors sit at a large table in the centre of the board room, surrounded by hundreds of others stood in the wings or listening in over the phone. Those who fail to meet Bezos’s expectations or who ‘[err] with their data’ are quickly removed: ‘For managers, a failure to understand and communicate the operational posture of their service could amount to career death’.69 AI does not automate the decisions of the S-team or obviate its disputes. On the contrary, disputes arise around political and
strategic issues that cannot be solved by probabilistic calculation alone. At one point, the S-team was described by several of its members as ‘a highly combustible forum, a group in which everyone felt the need to be outspoken and curry favor with the boss and where political disputes were allowed to fester’.70 There have been many arguments about authority and jurisdiction. For instance, in 2005, Kal Raman, a software technician, was hired from Walmart to overcome problems in retail. He wanted Amazon to become a datacentric company which exploited AI. Nevertheless, he argued furiously with Diego Piacentini, who was formerly solely in charge of retail, over who had authority over fulfilment centres. AI and data (sometimes) informed those debates, but they did not determine them. They addressed strategic questions about the structure of the company—and the fate of specific departments in it. AI did not develop Amazon’s marketing strategy; its executives did. Indeed, Bezos developed a number of collaborative techniques, such as ‘twopizza teams’, to enhance creativity and innovation.71 Amazon’s logistics have also defied automation. Amazon’s logistics operations are organised around its fulfilment centres, massive warehouses in which it stores and dispatches its goods. Automation has played a role in the fulfilment centres, but the human workforce remains essential. Amazon has employed robots in its fulfilment centres wherever it can. Stocking the warehouses and loading orders can be onerous; a warehouse worker might walk more than ten miles over the course of the workday. Robots have been introduced to reduce this physical burden on human workers: ‘Stowers in older Amazon facilities used to walk up and down long aisles pushing a cart full of products, placing them randomly on shelves where they found space, and scanning them with a handheld device to mark their location in a system. Now Amazon robots carry empty shelving units—known as pods—to the workstations of stowers, who take products placed in front of them and fit them into open shelf space inside the shelving pods’. Robotic stowers bring goods to the pickers, the people who pack the orders. Currently about two hundred thousand robots are used in Amazon’s fulfilment centres.72 As a result, pickers’ productivity has tripled, though—
demonstrating the importance of human labour—injury rates increased from 2.9 per 100 workers in 2015 to 11.3 per 100 workers in 2018. Even with this automation, humans still vastly outnumber robots. In 2020, Amazon employed 1.3 million people globally.73 Many of them are poorly paid workers who stock and collect goods from shelves in massive warehouses. Indeed, precisely because it is so dependent on its human labour force for profit, Amazon has vigorously eliminated all forms of unionism. Even so, the company has to tolerate significant deviance from its indispensable human employees. For instance, theft in fulfilment centres is a problem. In 2006, a temporary employee in Coffeyville, Kansas, was clocking in and out for his shifts at the fulfilment centre but could not be found while at work. Eventually, he was discovered in a cavern he had created in a pile of empty pallets in the corner of the fulfilment centre.74 Presumably, if Amazon could automate its warehouses, eliminating the inconvenience of human employees, it would have done so long ago. Yet it is a long way from automating its fulfilment centres. Home delivery and the circumvention of the high street is the decisive part of Amazon’s activities; it has always been its defining characteristic, and its success is vital to its competitive advantage. Amazon orders are loaded onto vans by human employees, who drive them direct to the specific abode of the Amazon customer, whatever the weather or other circumstances. They may leave packages at neighbours’ houses or decide not to leave them if they might get stolen. There is no automation at the point of delivery. It would be very difficult to program an autonomous vehicle or a drone to drive around a town—negotiating traffic, roadworks, and unexpected hazards—to find problematic addresses. There are numerous mundane physical tasks at which humans are far more proficient than any robot or drone ever could be. Although Amazon has experimented with drones, there seems little likelihood that Amazon’s delivery system could be populated by automated vehicles directed by Amazon software. Its algorithms sift data to accelerate orders but are not able to execute delivery
operations themselves. Amazon is a data-centric, AI-driven company, but its enterprises ultimately rely on human employees in the board room, in the fulfilment centres, and in its delivery vans. Amazon is not the only company which demonstrates the improbability of full automation under second-generation AI. Elon Musk, a pre-eminent Silicon Valley pioneer since the 1990s, has aggressively sought to realise a vision of the future through the development of AI, aspiring to automate human practice wherever possible. For instance, his SpaceX programme was initiated on the frankly unfeasible ambition to travel to Mars. There is no doubt that AI has been central to Musk’s projects—and his successes—but he now recognises its limits, despite his determination to transform humanity through technology. I have already discussed the problems of Autopilot in Tesla cars. In 2018, Musk tried to automate the production of batteries for Teslas at his factory in Nevada. Yet, even with automation, the factory failed to produce enough units. Musk eventually realised that automation was the problem, not the solution. Visiting the site, Musk watched a robot struggle to adjust a small seal on a window for several minutes. Musk stepped in and completed the simple task in seconds. In that moment, as his biographer remarked, ‘Musk learned that there are certain tasks, sometimes very simple ones, that humans do better than robots’. Consequently, Musk issued an immediate directive to managers: ‘You have seventy-two hours to remove every unnecessary machine’. Later he announced publicly: ‘Excessive automation at Tesla was a mistake. To be precise, my mistake. Humans are underrated’.75 Total automation is not an ideal but a fallacy. The best companies have exploited AI to augment their human workforce, not to replace it. In China, which is rapidly catching up with the US in terms of AI, there is a similar phenomenon. In Chinese cities, services have been revolutionised by AI. There has been an explosion in online delivery, ride-hailing, personal styling, and services. Chinese tech companies have tended to go ‘heavy’: ‘They don’t want to just build the platform—they want to recruit each seller, handle the goods, run the delivery, supply the scooters, repair those scooters, and control the payment’.76 These companies have vertically integrated the human
workforce which delivers their services. The point here is that these companies are data- and AI-centric. There has been a digital revolution in China—but the human labour force remains essential. AI has been introduced to reconfigure the management, administration, and productivity of the companies and to enhance their relations with their customers, not to supersede their human employees. Human, All too Human What can AI do? Many AI experts and security-studies scholars worry about the military implications of AI. They believe that the automation of war is imminent. That eventuality is not impossible; nothing is. Yet the history of artificial intelligence, the capabilities of second-generation AI, and the application of AI by leading tech companies such as Amazon are salutary. There is no questioning the potency of second-generation AI, especially large language models. Working inductively on massive datasets, second-generation AI models have produced almost miraculous results in an unimaginably short time. Nevertheless, the mechanics of second-generation AI suggest that we should be cautious to presume that in the near future computers will display practical human intelligence and judgement, much less superhuman intelligence. AI operates probabilistically, on the basis of statistical likelihood. It is specific and narrow. It knows nothing; it understands nothing. Consequently, while it is able to perform impossible calculations on data, it cannot define—much less redefine—even a simple social situation. It does not recognise context or meaning. While a model can be programmed to respond on the basis of a vast dataset, it ultimately lacks human judgement and imagination. It cannot interpret a situation, weigh up a variety of factors it has not seen before, and decide on a set of novel actions. It cannot plot across a variety of cases to develop a coherent response. It cannot go beyond its data. The history of tech companies affirms both the power and the limitations of contemporary AI. Google, Facebook, Amazon, Airbnb,
and other tech companies use data, processed by trained models, to interact immediately with their customers and to understand the market with greater fidelity than their competitors do. Because they can store every single digital transaction, they are able to map the market with unparalleled accuracy. They are therefore often able to anticipate market demand in a way which old analogue companies could not. Data is knowledge. Nevertheless, the executives in these companies, informed by data, still have to make strategic and operational decisions about what these companies will do. They have to negotiate contracts with suppliers, retailers, workers, host governments, and venture capitalists. Executives have to make strategic and political decisions about how to employ the AI technology itself. While AI allows companies to understand their markets, those companies still rely on human employees to make, organise, and deliver the services and goods in the physical world. Those decisions are not reducible to probabilistic induction. AI has facilitated human enterprise; it has not replaced human labour and expertise, and it appears unlikely to do so in the foreseeable future. Plainly, the armed forces are special, as they alone exercise lethal force in the defence of a state. Yet the practices of tech companies which have invented and utilised AI are instructive. AI has allowed tech companies to innovate and to dominate old markets and discover new ones. AI-enabled companies operate differently to traditional ones. They are structured differently, with an often smaller workforce. Automation has been employed where it has contributed to efficiency. Yet the commercial application of AI is salutary. AI has not even partially displaced executive boards, managers, or workers. The limitations of second-generation AI that are evidenced in the commercial sector suggest that full military automation, in which AI is entrusted with strategic decision-making or command of military operations as a whole, is extremely unlikely. It therefore seems unwise to presume that AI will take over strategy and war any time soon.
3 AI Strategy Many AI experts fear that in the very near future war will be automated. But the actual capability of second-generation AI and the way in which civilian companies have utilised it in the last two decades recommends scepticism. Military automation will be far harder to achieve than is often presumed. However, to help us understand how AI might transform the battlefield of the near future, let us examine how states and the armed forces think they will actually employ AI. From Science Fiction to Policy In the past five years, almost all the major powers have published formal policy statements on their plans to use AI for military purposes. They have developed AI strategies describing what they intend to do with their forces and how they will modernise them with the application of AI. These statements are deeply interesting—and highly pertinent to understanding the military potential of AI. They discuss not what might happen in the long term but rather what governments and their militaries have done and are going to do in the next five years to harness the power of AI. Because they move the discussion of AI from speculation to concrete policy, these documents are salutary. Of course, there is often a divide between policy and actuality. There is always slippage between strategy and execution, and in many cases, the aspirations of policy are never achieved. Nevertheless, policy statements and written military doctrines provide, at least, a horizon of possibility in which current
and future developments are most likely to take place. It is improbable that military forces will quickly develop capabilities which are utterly un-envisaged in existing policy. Consequently, it is worth examining current AI strategy in some detail, as these statements will indicate whether the armed forces are really about to use AI to automate war, or whether, in fact, they intend to use AI for other purposes. American AI Strategy In the 1950s, the US faced an increasing military threat from the Soviet Union. The Soviets had developed a nuclear capability in 1949, when they tested their first bomb, and their arsenal was growing quickly. Because the armies of the Warsaw Pact massively outnumbered nascent NATO forces in Europe, President Eisenhower announced a new security strategy: the build-up of nuclear weapons to ‘offset’ Soviet superiority in conventional forces. The First Offset Strategy generated the ironically named MAD (mutually assured destruction) defence doctrine. In the 1970s, the US once again faced a security threat. In 1973, after the withdrawal from Vietnam, the American armed forces abandoned conscription. Although the US hoped to benefit from professionalisation, its forces contracted to about half their previous size; the militaries of the Warsaw Pact countries then outnumbered NATO forces in Europe by three to one. The US defence budget for 1975 fell by $100 million. At the same time, Soviet forces had made some considerable technological advances. Consequently, the US committed itself to a ‘Second Offset Strategy’. The Second Offset Strategy involved advanced new technology, including better surveillance systems, precision weaponry, stealth technology, and satellite communications. The Second Offset Strategy was designed to win the Cold War. It may have contributed to the collapse of the Soviet Union, as the Soviets could not compete economically with US military investments. Although after the end of the Cold War in 1990, the US faced no serious competitor for twenty years, in the 1990s and
2000s the Second Offset Strategy underpinned US military operations in the Balkans, Iraq, and Afghanistan. However, from about 2012, it was manifest that major power competition was returning. Russia, having invaded Georgia in 2008, was becoming more assertive. In 2014, Russia annexed Crimea and parts of Luhansk and Donetsk provinces. More disturbingly for the US, China was plainly emerging as a great power rival. China was rapidly improving its technology; it was advancing quickly in the areas of computing and space, and it was investing massively in its armed forces. It now intends to develop its ‘informationized’ warfare into full ‘intelligentization’, in which AI will be integral. In 2017, Lieutenant General Liu Guozhi, director of the Central Military Commission’s Science and Technology Commission, proclaimed that the People’s Liberation Army was ‘on the eve of a revolution’ in which AI would accelerate the process of military transformation: ‘Whoever doesn’t disrupt will be disrupted’.1 China’s policy has provoked the US. In November 2014, Defense Secretary Chuck Hagel announced a ‘Third Offset Strategy’ to ensure that the US could maintain its military advantages over China. The Third Offset Strategy involved a number of new technologies, including new long-range bombers. Yet, overwhelmingly, this strategy sought to exploit the potential of AI to enhance US military capabilities. For instance, on 16 July 2019, Secretary of the Army Mark Esper was asked by Congress what the priority for modernization of Department of Defense technology ought to be. He replied: ‘For me it’s artificial intelligence. I think artificial intelligence will likely change the character of warfare, and I believe whoever masters it first will dominate on the battlefield for many, many, many years. It’s a fundamental game changer. We have to get there first’.2 Robert Work was appointed deputy secretary of defense in 2014 by President Barack Obama; he remained in that post until 2017, into the Trump administration. Serving for three years in this key post, he was one of the architects of the Third Offset Strategy. In order to understand the role of AI in the new strategy, we should pay attention to some of Work’s explanations of the Third Offset Strategy. In 2015, not long after his appointment, in the closing
comments of a speech to a defence forum, Work famously declared, ‘10 years from now if the first person through a breach isn’t a friggin’ robot, shame on us’.3 The phrase has become synonymous with him and has often been taken as evidence that for Work, the Third Offset Strategy and AI will primarily involve automation of the armed forces. There is no doubt that Work wanted the US armed forces to employ autonomous weapon and robotic systems wherever they could. However, his statement is more modest than it may at first appear. It does not imply that robots should replace human personnel in all or even most military jobs. Rather, it identifies a specific military task—an infantry soldier breaking into a defended building. This is one of the most dangerous actions a soldier can perform and has almost always incurred heavy casualties. Work’s suggestion is that, for this one specific task, the US Army and the US Marines, whose troops might have to break into a building, might employ a robotic system instead of a human. For Work, automation should be part of the Third Offset Strategy, but it is by no means the only—much less the most important—application of AI to military operations. In other statements, Work articulated this position clearly. For instance, on 28 April 2016, he gave a speech to the North Atlantic Council in Brussels in which he explained the Third Offset Strategy, encouraging council members to pursue their own innovation programmes in support of it. Autonomy was certainly not irrelevant to this speech. Work argued that AI and autonomy constituted the technological ‘sauce’ of the Third Offset Strategy. Nevertheless, the content of the speech was intriguing. Work did not suggest that robots of any type were about to predominate. On the contrary, rather than equating his vision of automation with fully autonomous systems, he compared it with the AI used in automobiles. Although, as already noted, autonomous cars, such as Waymo, do exist, and eventually more such cars might, AI has generally been employed to support specific driving functions, rather than to drive cars by itself:
Now just looking at the commercial, just think about the developments in self-driving cars. In just a few years we’ve gone from cruise control. Everybody knows about cruise control. That is artificial intelligence and autonomy that you have delegated to the car the authority to keep a certain speed. And it is narrow, artificial intelligence and the car does it very, very well. It’s very narrow […] We’ve delegated authority to the machine to warn you and you take action afterwards […] Well, that’s all we’re talking about here, using AI and autonomy in ways which help the human, not to create Terminator or Skynet, now this is to allow humans to operate better or without a human.4 Eventually some weapon systems might be completely autonomous. Yet, in the near term, military automation will be much more limited, restricted to specific functions which a computer can do more quickly than a human can: above all, data processing. Indeed, Work disparaged the notion of ‘killer robots’, saying, ‘This third offset is about making [humans] better’, not replacing them. There were certain areas, he said, in which AI-enabled autonomy might be crucial, especially in defence: ‘We are going to leverage AI technology, particularly in things like cyber defense, electronic warfare defense, missile defense’. Cyber, electronic, and missile attacks might exceed the capacity of humans to intercede; they take place at a speed which is beyond human reaction times. At that point, when data has to be processed at speed, Work acknowledged, AI might play the decisive role: ‘Now, there may be times where you’re under attack in the case of missile defense. You’ve got 60 missiles coming at you. There’s no way that a human is going to be able to sort all that out. The human will make the decision, but the machine protects. And with that going on, the machine will do exactly what it is programmed to do’. In this way, AI would increase the effectiveness, efficiency, and lethality of US forces. AI would help create ‘joint and combined collaborative human-machine battle networks’.5 Work described the specific role AI might play in these digital battle networks. Battle networks consist of three elements: a sensor
grid (intelligence feeds); a command grid, which analyses that intelligence and makes decisions based on it; and an effects screen, which acts (weapon systems). AI will not automate any one of these elements entirely. However, it will play a crucial role in filtering the information from the sensor grid to allow commanders to make more accurate decisions; it will automate data processing at specific points in the network. For instance, AI will play an important role in targeting. Work gave the example of how, in July 2014, AI helped Ukrainian intelligence to date and geo-locate material which Russian troops posted online. On 17 July 2014, a Malaysian airliner, Flight MH-17, was brought down over Ukraine, killing over 300 passengers and crew. The data proved, despite the Kremlin’s denials, that Russian forces were responsible. AI will enable US commanders to have better situational awareness across the entire battlespace— they will see everything. They will therefore be able to plan more quickly and comprehensively, coordinating the full spectrum of capabilities. Finally, they will be able to target enemies more accurately. Work came to a significant conclusion. He fully recognised the potential of AI as a technology; it was the ‘sauce’ for the Third Offset. However, in the end, the success or failure of the Offset would depend not on the technology but on ‘new organisations and new different, operational constructs’.6 In order to harness the power of AI to process data automatically and at scale, the US armed forces have to reorganise themselves radically and to re-imagine the character of their military operations. They have to decide how to apply AI’s data-processing capability to a suite of applications. Work’s public statements are a fertile source for understanding the Third Offset Strategy. During and just after Work’s time in office, the US published formal documents which affirmed Work’s statements and laid out a plan of how the US armed forces would harness AI. As part of the Third Offset, the US, therefore, articulated an AI strategy in the 2017 National Security Strategy and the 2018 National Defense Strategy. The latter document, written by Secretary of Defense James Mattis, was compelling. It began by identifying China and Russia as the nation’s main rivals: ‘China is a
strategic competitor using predatory economics to intimidate its neighbors while militarizing features in the South China Sea. Russia has violated the borders of nearby nations and pursues veto power over the economic, diplomatic, and security decisions of its neighbors’.7 Against the threat which China and Russia posed, the US proposed to ‘build a more lethal force’: ‘Our aim is a Joint Force that possesses decisive advantages for any likely conflict, while remaining proficient across the entire spectrum of conflict’.8 The National Defense Strategy identified eight areas of development. The term ‘AI’ appeared only twice in the whole document. Nevertheless, it was clear that improving performance in most of these eight policy areas—especially (1) space and cyberspace; (2) command, control, and communications, computers and intelligence, surveillance, and reconnaissance (C4ISR); and (3) advanced autonomous systems—depended on AI. Indeed, the Strategy was explicit about the role of AI: The drive to develop new technologies is relentless, expanding to more actors with lower barriers of entry, and moving at accelerating speed. New technologies include advanced computing, ‘big data’ analytics, artificial intelligence, autonomy, robotics, directed energy, hypersonics, and biotechnology—the very technologies that ensure we will be able to fight and win the wars of the future.9 Mattis’s National Defense Strategy articulated a vision of how the US would prepare to defend itself in the near future. In 2018, the Joint Chiefs of Staff published the National Military Strategy. Framed by the National Security Strategy and National Defense Strategy, it described the role of AI in the Offset. That document described how AI might enable the US Navy, Air Force, Army, Marine Corps, and emergent Space and Cyber Command to work more effectively together: To achieve military advantage over competitors and adversaries, the NMS [National Military Strategy] introduces the notion of joint combined arms, defined as the conduct of operational art through the integration of joint capabilities in all domains. The Joint Force and its leaders must be as comfortable fighting in space or
cyberspace as they are in the other three traditional domains of land, sea, or air.10 [emphasis in original] This was the first time the US had proposed the development of what would later be defined as multidomain operations. AI was implicitly central to this vision because integrating military activities from such diverse forces required a massive quantity of data. Only AI-enabled systems might be able to process such a quantity. The National Military Strategy did not explicitly define itself as an AIdriven programme. Yet, in aspiring to create a multidomain force, it adopted AI at its core. At the same time as the Department of Defense was working on the National Security Strategy and the National Defense Strategy, the US government had appointed a National Security Commission on Artificial Intelligence. This commission, led by Eric Schmidt and Robert Work, is an important statement about how the US intends to use AI. The NSCAI’s Final Report, published in 2021, begins with a sobering claim that the US is behind other nations in the AI race. The document discusses autonomy and robotics. A chapter dedicated to autonomous weapon systems considers the ethical risks they raise. Like Work, the authors of the NSCAI’s Final Report wanted AI to be applied across defence functions. Echoing Work’s statements, automation and lethal autonomy are not the prime focus of this document. The report defines AI as a ‘constellation of technologies’: data, computer power, and algorithms, ‘mathematical operations to tell the system how to navigate to provide the answers to specific questions’. AI will enable the armed forces to process data on a hitherto unachievable scale and at a hitherto unachievable speed. Since data will be central to US military capabilities, the report emphasises the radical potential of AI for military intelligence in particular: AI will revolutionize the practice of intelligence. There may be no national security function better suited for AI adoption than intelligence tradecraft and analysis. Machines will sift troves of data amassed from all sources, locate critical information, translate
languages, fuse data sets from different domains, identify correlations and connections, redirect assets, and inform analysts and decision-makers.11 In this strategy, the prime function of AI is not military automation or lethal autonomous weapons. It is the processing of massive datasets. There are many possible applications here, but in this capacity, AI will allow the US forces to fuse intelligence for greater situational awareness and understanding: ‘Traditional confines of the battlefield will be expanded through AI-enabled micro-targeting, disinformation, and cyber-operations’.12 The report continues: ‘In war, many of the military uses of AI will complement, rather than supplant, the role of humans. AI tools will improve the way service members perceive, understand, decide, adapt, and act in the course of their missions’.13 The goal of the strategy is ambitious: ‘Once the IC [Intelligence Community] has automated its processes within individual intelligence disciplines, it should fuse those individual processes into a continuous pipeline of all-source intelligence analysis processed through a federated architecture of continually learning analytical engines’.14 The National Military Strategy plans to automate some elements of data processing—not strategizing, decision-making, or warfighting more generally. The strategic aim of the US is clear. The problem is, according to the NSCAI, that the ‘DoD [Department of Defense] remains locked in an Industrial Age mentality’, with ‘massed forces and monolithic platforms and systems’. Today, the US does not face traditional military threats which can be overcome by building ever-moreexquisite platforms. It now confronts a new challenge: ‘A new warfighting paradigm is emerging because of AI. Our competitors are making substantial investments to take advantage of it. This idea has been called “algorithmic” or “mosaic” warfare; China’s theorists have called it “intelligentized” war’.15 Here, a traditional, platformcentric approach will fail. Instead, ‘Advantage will be determined by the amount and quality of a military’s data, the algorithms it develops, the AI-enabled networks it connects, the AI-enabled weapons it fields, and the AI-enabled operating concepts it embraces to create new ways of war’.16 Embracing data processed
by AI, US forces in each domain will be able to integrate fully with each other to achieve a new level of lethality. The National Defense Strategy of 2018 and the NSCAI’s Final Report of 2021 were strategic-level statements. In March 2022, the Department of Defense published a related policy, The Joint AllDomain Command and Control (JADC2) Strategy. This strategy describes how the Pentagon intends to operationalise the nation’s AI strategy. Above all, the Pentagon wants to construct an AI-enabled situational awareness and communications systems common to all the services. JADC2 will unite diverse data from all the services, on which all US military forces can draw: ‘This Joint All-Domain Command and Control (JADC2) strategy describes the urgent need for a focused Departmental push on actions to empower our Joint Force Commanders with the capabilities needed to command the Joint Force across all warfighting domains and throughout the electromagnetic spectrum to deter, and, if necessary, defeat any adversary at any time and in any place around the globe’.17 JADC2 will enable the ‘Joint Force Commanders to “sense”, “make sense”, and “act” in the operational environment’.18 Data is fundamental to JADC2: ‘This data and information sensor ecosystem exploits remote sensors, intelligence assets, and open sources to sense and simultaneously integrate information from and within all domains to enable the Joint Force Commander to achieve information and decision advantage’.19 However, because there is so much data, from so many sources across the domains, traditional human-centric analysis methods are no longer adequate. Artificial intelligence will be essential: ‘JADC2 developed capabilities will leverage Artificial Intelligence and Machine Learning to help accelerate the commander’s decision cycle. Automatic machine-to-machine transactions will extract, consolidate and process massive amounts of data and information directly from the sensing infrastructure’.20 JADC2 is a highly ambitious enterprise, and its success is uncertain. Nevertheless, the plan illustrates the American understanding of AI. AI will be primarily used to process data across a range of functions to help planning, intelligence, situational
awareness, cooperation, and military decision-making. It will support military operations, not automate them. JADC2 is a Pentagon-level, joint, and cross-domain effort. Notably, the services have been developing their own doctrines about AI too. The US Air Force has been at the forefront of this thinking. It recognises that data has become essential to its operations. However, because its datasets are so huge, they are exceedingly difficult to mine. It is very easy for a commander to become overwhelmed by data and, therefore, for a force to miss important data points. The US Air Force has had considerable experience of this problem of data overload. In 2017, an assault on an al Qaeda safe house in Afghanistan produced 40 terabytes of data. If just a quarter of that data were in the form of videos, it would take a person 208 days, working twenty-four hours a day, to review it all. However, with ‘AI preparing and triaging raw collected data, human analysts would have more time to evaluate the material and apply their expertise’. In the CIA, for instance, AI is already saving all-source intelligence analysts forty-five days a year.21 It is simply impossible for the air force to operate effectively without using AI. Consequently, on 20 September 2021, Secretary of the Air Force Frank Kendall announced that the air force had deployed ‘AI algorithms for the first time to a live operation kill chain’.22 The air force will not allow AI to decide on targets or to displace human analysts, commanders, and pilots. But by processing a sea of data, AI is helping it to operate more effectively. The US Army has also recognised the utility of AI. In October 2022, it published its latest version of Field Manual 3-0 Operations. This field manual is a foundational piece of doctrine for the army. Crucially, it revised AirLand Battle, which had been the army’s doctrine since 1982. FM 3-0 introduces a new operational concept: multidomain operations. The army accepts that in order to conduct land operations, it can no longer operate independently. It is not even sufficient for it to cooperate with air and maritime forces. It has to be able to conduct multidomain operations, harnessing space and cyberspace too:
Multidomain operations are the combined arms employment of joint and Army capabilities to create and exploit relative advantages that achieve objectives, defeat enemy forces, and consolidate gains on behalf of joint force commanders. Employing Army and joint capabilities makes use of all available combat power from each domain to accomplish missions at least cost. Multidomain operations are the Army’s contribution to joint campaigns, spanning the competition continuum.23 Of course, the US Army believes that its mission is crucial to the success of any multidomain operation: ‘Military operations on land are foundational to operations in other domains’.24 Yet, in the age of multidomain operations, new interdependencies appear. Above all, the US Army now requires the support of the cyber and space domains. The army needs cyber capabilities in order to protect its digital systems from virtual attacks and to mount cyberattacks on enemies. It needs space capabilities in order to communicate, navigate, and, increasingly, target opposing forces; satellites have become a critical military enabler for land operations. FM 3-0 does not use the term ‘artificial intelligence’ once, though it mentions data fourteen times. Nevertheless, its description of how the US Army will operate implies a central role for AI. For instance, when it describes how a joint force commander (JFC) would plan a response to a crisis, it includes the following passage: As Army forces prepare to respond to a crisis, the JFC conducts a final review of deploying forces, ensuring they are deployed in the proper sequence and are able to be task-organized effectively for the anticipated mission. Threat forces are likely to detect force projection activities using space and cyberspace capabilities, human intelligence, and open-source collection efforts. Planners should anticipate adversary forces using all available means to contest the deployment of forces, beginning from home station, during transit, and upon arrival in theater.25 The second sentence is crucial. It would be very difficult to exploit and fuse information from cyberspace, human intelligence, and open sources without using some form of machine learning. The amount
of data from open sources alone is often vast. Some automation of the collection and selection process is normally required. Consequently, in order to conduct multidomain operations on land and to share data with the other branches, the US Army will need to embrace artificial intelligence. By contrast, lethal autonomous weapons and battlefield robots are absent from the document. AI will play a more mundane but no less significant a role of processing data in order to aid in situational awareness, to help plan, to identify targets, and to warn of real and virtual threats against the deployed force. In the last five years, the NSCAI, the US Air Force, and the US Army have each described an AI strategy. Autonomous weapon systems are certainly not irrelevant. Robert Work, for instance, has emphasised the potential for robotics. The armed forces are interested in AI-enabled autonomous systems, such as drones and uncrewed surface and underwater vessels; these are likely to soon play an important role. The armed forces will automate where they can. Yet, in these documents, AI’s primary role is not the automation of military operations but the processing of data. AI is critical because with mass data processing, the forces will be able to see more deeply, accurately, and quickly through the entire battlespace. With AI, US forces hope to be able to see what the enemy is doing early on, and, because data can be transferred between domains, US forces aim to orchestrate a response which fuses the capabilities of all their assets. Reflecting written policy, a consensus is already developing among military professionals in the US. Air Force Lieutenant General Jack Shanahan is an important figure here. After a long career, he was directed to establish the Algorithmic Warfare Cross-Functional Team (Project Maven) in 2017. Following that, he was the inaugural director of the Defense Department’s Joint Artificial Intelligence Center in 2018. His role as director was to accelerate the uptake of AI by the armed forces. Shanahan is, therefore, well-positioned to make a judgement about the application of AI to military affairs. When interviewed for this book, he was dismayed by the recent declamations of Geoffrey Hinton and Stuart Russell about the
imminent dangers of AI and the rise of slaughter-bots. Indeed, he feared these pronouncements might be damaging for the armed forces: ‘Civilians are scared by the vision, so they are anti-AI. In fact, at base, the opposition is not to AI but to US military and foreign policy’. He voiced concern about ignorance of AI among the public and political elites: ‘Autonomy has been totally overstated. It is one function. The lethal autonomous weapons obsession is inaccurate and not healthy’. For Shanahan, ‘Data and AI are absolutely an intelligence function. They allow targeting across the depth of the battlefield’. Harnessing AI is, therefore, imperative. He concluded, ‘Warfare at the front end will be horrible, but AI and data will enable the force’.26 Shanahan is a very senior officer. Yet junior personnel working in related fields shared his view. For instance, a US Army officer working on the introduction of AI into the army proposed a similar account of the application of AI. He roundly dismissed the obsession with military automation and autonomous weapons, saying ‘Terminator is not going to happen’. Instead, he described the empirical problem facing commanders today; there is so much data that commanders risk being overloaded: The intelligence apparatus is now over-engineered. There is a proliferation of sensors. Finding stuff is not hard. What is hard is working out why it is there. When you have so much data, how do you know if that [data] is important? In the Cold War, in the 1970s, you had a satellite photo and a radio intercept to find a truck. But what if you can see the entire Red Army; should each have the same weight? If I can see everything, what do I look for?27 In On War, Clausewitz described the fog of war. In the eighteenth and early nineteenth century, generals often lacked sufficient information. Now, ubiquitous sensors, acting as universal data feeds, generate too much information. Because commanders can potentially see everything, they may understand nothing. For the US Army officer quoted above, AI can play a crucial role here: ‘AI is just software, but now you have such a daunting task, AI helps. I have every sensor in a five-kilometre square area: which ones are
important? AI filters it down for humans’. Crucially, AI will allow data from one domain to inform command decisions in another. The US military will employ AI primarily for military intelligence purposes in the broadest sense. It will allow US armed forces to process a vast amount of data so that they can understand more accurately, plan more accurately, and target their enemies more accurately. British AI Strategy In the last decade, the UK, one of America’s closest military and intelligence allies, published a series of strategy documents which echo the US publications closely. There is little doubt that the UK has been influenced by the US and motivated to accelerate its use of AI for military purposes so that it can continue to cooperate with the US. In September 2021, the UK government published its National AI Strategy, which stated that the UK planned to invest in AI in order to exploit the potential of data. The National AI Strategy is, therefore, usefully read alongside its sister publication, the Ministry of Defence’s Data Strategy for Defence, which was also published in 2021. Data Strategy for Defence was explicit about the challenge the UK faced: ‘Defence Data is an enduring strategic asset, effectively exploited and driving sustainable battlespace advantage’.28 Admiral Sir George Zambellas, the First Sea Lord, stated, ‘The future performance in war will be dominated by the relentless and competitive exploitation of data’.29 The document laid out a scenario for the UK to achieve: We urgently need to invest in the technologies that will revolutionise warfare. In the future a soldier in hostile territory will be alerted to a distant ambush by sensors on satellites or drones, instantly transmitting a warning, using Artificial Intelligence to devise the optimal response, and offering an array of options, from summoning an air strike to ordering a swarm attack by drones, or paralysing the enemy with cyber weapons.30 As in the US, the aim is to use data to orchestrate operations in the air, on the sea, on land, in cyberspace, and in space. AI will be crucial to this effort because it will be impossible to process the
quantity of data involved without the help of machine learning. For the UK, AI is primarily about intelligence and information management. In 2021, the UK published its new security and defence strategy: the Integrated Review and the Defence Command Paper. These documents affirmed the policy laid out in Data Strategy for Defence. The 2021 Integrated Review and Defence Command Paper proposed a tilt to the Indo-Pacific in the light of Brexit and the rise of China . In order to achieve this tilt, they envisaged a coherent, crossdepartmental approach to security which would be enabled by the exploitation of data processed by AI. In March 2023, only two years later, the British government published a revised version of the Integrated Review and the Defence Command Paper. The Defence Command Paper of 2023 is not limited to technological development, much less to the potential of AI. Nevertheless, it illustrates the Ministry of Defence’s thinking about AI. The word ‘data’ is used forty-four times in the document. The message is clear. In order to compete in the coming decade, the UK armed forces must exploit the power of data: Over the last year, the Armed Forces of Ukraine have shown the game-changing impact of the most advanced intelligence, surveillance and targeting software ever deployed. We have witnessed how communications infrastructure, digitisation of data, and increasing automation and autonomy are vital for data security, information operations, communications, targeting, interoperability, and lethality.31 The second sentence does not explicitly mention AI, but the term is implied with the words ‘automation and autonomy’. AI will automate data processing to secure information and communications systems, and to enable multidomain targeting. Harnessing AI in this way, the UK will learn a new way of warfare, one that will be ‘joint and alldomain, underpinned by data and information, both open-source and highly classified’.32 According to the Defence Command Paper, the UK must pursue AI aggressively.
British service personnel have concurred with published AI strategy—and with their US counterparts. For instance, a senior British Army general involved in the development of AI capabilities has dismissed the wilder claims about AI and military automation. For him, AI is not principally about drones and robots: ‘We should be about demystifying AI. It is about reducing the cognitive load. We want to do anything that can reduce the cognitive load of humans to have better precision, tempo, lethality, reach and understanding […] We want to be a data-driven army’.33 Another senior British officer described how AI might be employed by the army: AI is just a technology. Some people say in the face of a problem, “We need to do more AI”. This is wrong. The question is “What is your problem?” They are reaching for the stars, but they don’t know their own galaxy. They need to see the potential of AI. AI is involved in analytics. The prime function of AI, in this officer’s opinion, is data processing. Senior officers working on AI are clear. Junior British officers expressed a similar understanding of AI. For instance, a Royal Engineer highly trained in computer programming described how AI might be harnessed to process all the high-resolution satellite imagery of a square kilometre identified as a target area: One kilometre square is a huge area to go through and see what is there. But you can do that—go through one kilometre square a week. You look for key signatures. You have hundreds of pictures of vehicles. You feed that into an algorithm. Once it is trained on pictures, it can identify vehicles. Then, you can compare to where they were before; for instance, has an aeroplane moved? You are seeing if things are moving or not. You are beginning to get insights without analysts sat there.34 Massive datasets already exist from which it is possible to develop a more faithful and comprehensive intelligence picture. British armed forces are trying to exploit the potential of these datasets. The only way of processing them is by applying AI to them. For the British armed forces, AI’s primary function is processing data; AI facilitates
the exploitation of data in order to improve intelligence, understanding, and situational awareness. The UK, therefore, needs to develop an AI capacity—not in order to automate the force but in order to process all that crucial data. The UK lags behind the US in its application of AI to military operations. Yet its published strategy articulates a similar ambition to the US. Like the American armed forces, the UK aims to use AI primarily for intelligence and targeting, processing data at a scale and speed which humans cannot. NATO’s AI Strategy Because it is a multinational organisation operating by consensus, NATO has always been slower to develop new strategies and policies than its member states. However, even NATO has recognised the potential of AI to transform military operations, and it has recently sought to embrace AI. On 22 October 2021, NATO published a summary of its Artificial Intelligence Strategy. The document was far less practical than American or British AI strategies. For the main part, it concentrated on the principles and ethics of AI, rather than the application of it. The central point was that NATO should develop a responsible AI strategy. To this end, the document established principles for the use of AI: lawfulness, responsibility and accountability, explainability and traceability, reliability, governability, and bias mitigation. Given the sensitivities about national AI capabilities, Alliance politics seems to have encouraged an abstract approach to the problem of AI. Nevertheless, the strategy did propose the construction of a ‘robust, relevant, secure data infrastructure’.35 The aim was to collate and fuse all NATO data to create a common digital backbone on which space, cyber, air, land, and maritime forces could draw seamlessly across the entire alliance. NATO’s Draft Operating Concept for Multidomain Operations in the Urban Environment provides greater detail on the application of AI. NATO has been working on the urban environment for a few years, and this publication summarises its understanding of how
NATO might adopt a multidomain approach to urban operations. AI is certainly not the main focus of the document—the urban environment and its challenges are—but important parts of the text discuss AI. In particular, the volume of data in urban areas may become unmanageable, because cities are replete with sensors. Consequently, ‘emerging disruptive technologies such as AI and Machine Learning might replace humans for repetitive and process based requirements’; ‘the transmission and processing of this data will need to be significantly increased through AI and Machine Learning’. In particular, AI could ‘facilitate the extraction, interpretation, and analysis of data’; ‘supervised AI could select and prioritise targets’.36 NATO policy is sometimes more nebulous than national strategy or military doctrine. It has to be read rather more obliquely for the reader to discern the military implications of its often-bland passages. Yet NATO officers share the views of their national colleagues, such as General Shanahan and the cited senior British general. Autonomous weapons are not the prime concern. Automation of that type has been overstated. AI will, however, be crucial to the processing of data and, therefore, to the acceleration and refinement of intelligence collection and analysis. Rather than superseding commanders, AI will help them make decisions. For instance, following an exercise to test NATO’s multidomain urban concept conducted in February 2023 at the Joint Force Command in Naples, the deputy commander emphasised the importance of AI and data processing to NATO at the after-action review. He declared, ‘Data is king’. AI has become crucial in processing massive datasets. It has converted data into information in order to enhance the commander’s understanding of the situation. NATO is certainly much slower to promote AI than are its member states—and, especially, of course, the US. Yet, despite the obliqueness of its doctrine, NATO’s strategy for AI is evident. Like the US, the UK, and other allies, NATO intends its members to develop its AI capacities in order to process data in support of military decision-making.
Data is King Many observers have worried that full military automation will follow the introduction of AI—that AI will supersede commanders and will direct weapon systems without human intervention; that it will automate war and warfare. An analysis of written AI strategy would suggest otherwise. Robert Work has suggested that robotics might be one of the applications of AI for limited functions, but he advocates a much wider application of AI. Using algorithms, AI will help process data to improve situational awareness and support decision-making. Following the Third Offset Strategy, the US, the UK, and NATO are not planning to automate war itself. They want to employ AI to automate data processing and, in that way, to improve a range of activities which might be broadly defined as military intelligence: situational awareness, planning, understandings, and targeting. The US, the UK, and NATO are not alone in this regard. Many other allied forces are embracing AI. They too recognise the power of data to enhance situational awareness, understanding, and decision-making. Although its operations are controversial to some, Israel is instructive here. Israel’s security concept comprises four pillars: deterrence, early warning, decisive defeat, and defence. AI has been applied to all four areas. Indeed, since 2014, Israel has understood itself to be in an AI arms race too. Autonomous weapons systems are one area of development; Israel has some of the most advanced autonomous systems in the world. Yet AI has been employed principally to process the massive datasets which a constellation of sensors collects on the Palestinians in the Occupied Territories every day. The AI policies of the US, the UK, and NATO are not unique; they are in line with the efforts of every major military power today. The US, the UK, and NATO do not dismiss the possibility of robotics and lethal autonomous weapons. Remotely controlled systems are already ubiquitous, and it is very likely that autonomous weapons will play a greater role in future. The policy documents, therefore, discuss the ethics of automation at considerable length. They are
deeply concerned with developing an AI strategy which is consistent with democratic norms. Ultimate human control will remain. The legitimacy of the armed forces depends upon it. Yet the focus is elsewhere. Above all, data processing is the central point of all these policies. In each case, the US, the UK, and NATO want to harness AI to process big data—to automate some parts of its collection and analysis—in order to improve their forces’ understanding of the battlespace. With the help of AI, the armed forces aspire to fuse and analyse massive amounts of data from satellite, signals, image, human, and open-source intelligence, so that they can plan and target more rapidly and with greater fidelity. AI—and the data it analyses—is not a weapon system itself, much less a platform. It serves a situational-awareness and intelligence function, conceived in the broadest sense. It helps commanders to understand and locate the enemy and to decide how best to deploy their forces. The entrepreneur Peter Thiel and his defence tech company, Palantir (discussed at some length in chapter 5), have been at the forefront of some of these developments. Thiel has articulated the military utility of AI eloquently: ‘Forget the sci-fi fantasy; what is powerful about actually existing AI is its application to relatively mundane tasks like computer vision and data analysis’. He has observed that ‘these tools are nevertheless valuable to any army—to gain an intelligence advantage’.37 Data and AI are a medium for situational awareness, planning, targeting, and coordination. The armed forces aim to exploit data and algorithms in order to outcompete adversaries through ever-greater efficiency, rather than simply to replace humans at key points in the chain. Indeed, it might be argued that AI’s broad intelligence application to military affairs can be distilled into three main functions: planning, targeting, and cyber operations. With regard to planning, AI has helped and will help the armed forces to understand the operating theatre more quickly and with greater accuracy. It will therefore help them to develop plans and coordinate forces, especially across domains. AI also plays an important role in targeting by processing a massive amount of data and identifying signatures in cyberspace. Finally, the armed forces intend to employ AI to augment and
enhance operations in cyberspace itself, defending against and mounting digital attacks while prosecuting information campaigns in this domain. In the previous chapter, I examined how Amazon has employed data and AI to improve its operations. If the published AI strategies discussed in the present chapter are accurate, the armed forces may be following the example set by the commercial sector. AI has not automated Amazon’s operations, but, by processing a massive quantity of data, it has provided Amazon executives new insights into their organisation, their markets, and their customers. For the commercial sector, data amounts to knowledge. The armed forces are undergoing a similar transformation as they harness the power of AI and data. Data presents commanders with new opportunities for situational awareness and intelligence. It provides them with new insights into their own units, their operations, and their enemies. Scholars are concerned that AI will lead to another military revolution, one in which autonomous weapons will colonise the battlefield. In truth, by automating data processing, AI is more likely to improve the armed forces’ understanding of the operating environment. For the armed forces, data represents a new source of intelligence that can help them deploy their forces more quickly and more efficiently. In short, AI is less about autonomous weapons and more about intelligence gathering and analysis. AI is most likely to refine situational awareness. It will, therefore, support military decision-making, rather than automate command decisions. Data may indeed be king, but AI will support military commanders, not supersede them.
4 A Military-Tech Complex The armed forces will use AI to process a vast amount of data. In this way, AI will improve situational awareness and understanding and, therefore, help with specific military functions—above all, planning, targeting, and cyber operations. How will AI do this? It is understandable why so many commentators have been bewitched by AI’s potential. After all, AI is able to process data and produce impressive results autonomously and without human direction. Precisely because of its powers, it is easy to imbue AI with independent agency. As explained in chapter 1, scholars have routinely presumed that AI is a pristine technology which will influence—and even determine—the conduct of war on its own. On this account, AI is an autonomous technology. It is a discrete, identifiable device which is able to exert influence over its human users; it has causal power. Yet there is a danger here that scholars, preferring the autonomous agency of AI, fall into technological determinism. They reduce their explanations of the armed forces to pre-existing and pristine pieces of technology. Technology determines action. The sociology of science and technology has consistently highlighted the weakness of determinism of this type. Scholars in this field have repeatedly shown that, far from being independent, even the most impressive and advanced forms of technology are developed in a specific social milieu. Technology is a social product. If we are to understand a technology, therefore, it is vital that we appreciate the culture of the scientists and researchers who created
and designed it, as a community, as well as appreciate the collective intentions and values of users. Technologies can never be separated from the interests, understandings, and purposes of the scientific communities which developed them and the practitioner communities which use them. Technology is always a social artifact, then, the functions and use of which cannot be separated from the organisations and groups which invent, develop, and employ it. This position in no way denies the powers of technology. It is often impossible to perform certain tasks without a specialist piece of technology—for example, one cannot slice bread without a knife. Yet technology, despite its utility, has no agency independent of the human groups which create and use it. When scholars claim that a technology determines human practice, they therefore contract their analysis, excluding the constitutive organisations and social activities which are always vital to making a technology what it is. There is always a social and organisational basis to technology. This sociological insight applies even to AI. There are many examples of technological determinism in military history and security studies. For instance, scholars have often asserted that in the late fifteenth and early sixteenth centuries, there was a gunpowder revolution: gunpowder weapons in and of themselves transformed war. For instance, since gunpowder weapons could easily breach medieval fortifications, they compelled instant re-fortification; tall medieval city walls were replaced across Europe with the low ramparts of trace italienne, the bastion fort, in the first decades of the sixteenth century. Bert Hall and Kelly DeVries have rejected this argument as technologically determinist. In fact, the trace italienne ‘is not something that sprang fully formed from the head of Zeus’. The trace italienne had medieval precedents; ‘during the fifteenth century no castle or town wall could ensure its safety against an opponent’s cannon without (finance permitting) some anti-gunpowder weapons alterations’.1 DeVries proposes a more important point. The evolution of warfare in the early modern period cannot be reduced to a single technology, the gunpowder weapon, which by itself changed everything. Rather, a multitude of organisational, social, political, economic, and military factors,
operating over a longer period, facilitated the development of the cannon in Europe. To understand the ‘gunpowder revolution’, it is essential to understand this context and situate these new weapons in it, rather than explain the entire transformation by reference to the cannon. The gunpowder revolution may be a pertinent analogue for AI. For instance, notwithstanding its extraordinary powers, it is better to see AI not as a pristine piece of technology, either. It was never developed, and does not operate, in isolation. On the contrary, it emerged in an institutional context. It was and remains a product of a unique techno-industrial ecology: Silicon Valley. There is a surprising, ironic conclusion to this. In practical terms, AI is indivisible from the tech sector which originally developed it and which continues to employ, expand, and refine its models. If we are to understand AI, it is imperative that we have some appreciation of this organisational background. Following the example of Kelly DeVries, we need to situate AI in a long, rich social context. As the armed forces have begun to apply AI to military operations, they have not simply taken a pristine, discrete, preexisting technology and applied it in isolation. On the contrary, the military application of AI has necessarily involved an engagement with the tech sector which invented these technologies. The military application of AI is just as much an organisational story as a technological one. It involves collaboration between defence ministries, the armed forces, and tech companies. This collaboration is not so much about the emergence of a hermetic AI-enabled military as it is about a private sector–public sector, civilian-military partnership. Silicon Valley is a critical actor in the application of AI to military operations. Without Silicon Valley and the tech sector, there could be no military application of AI. It is, therefore, important that we have some understanding of the history of Silicon Valley. Silicon Valley In 1955, William Shockley established Shockley Semiconductor Laboratory to develop new kinds of transistors. Transistors, which
had only recently been invented, consist of silicon chips capable of being programmed to perform mathematical calculations. Shockley presciently recognised that they were going to revolutionise computing, and he hired leading computer scientists to develop a programme to develop semiconductors. Unfortunately, Shockley was an intolerable authoritarian. He quickly alienated his scientific talent, and, in 1957, a year after the Dartmouth seminar on AI, eight of his employees—Julius Blank, Victor Grinich, Jean Hoerni, Eugene Kleiner, Jay Last, Gordon Moore, Robert Noyce, and Sheldon Roberts—resigned from his company to establish Fairchild Semiconductor. The ‘Traitorous Eight’ are now legendary in the history of Silicon Valley. Their mutiny was the origin of Silicon Valley and the rise of the tech sector in the US. For instance, in 2014, 70 per cent of all traded technology companies in Silicon Valley could trace their lineage back to Fairchild Semiconductor and the Traitorous Eight. The emergence of Silicon Valley as a technology hub in the late 1950s was perhaps surprising. In the 1950s, defence technology research and development was primarily located on the East Coast. Vannevar Bush, the dean of MIT, was the founder of Raytheon and had been President Franklin D. Roosevelt’s leading scientific administrator during the Second World War. With MIT and Harvard University, it might be expected that Cambridge—and the East Coast —would have dominated technological developments. However, Silicon Valley had some competitive advantages. Many commentators have emphasised that it had the ‘hippies’ anticorporate vibe’. It is easy to overstate the Bohemian culture of Silicon Valley, as ‘there was no lack of inventiveness outside of Silicon Valley’ either.2 Still, there was a perhaps more liberal ethos in California than on the East Coast. Most important, innovators there were willing to share their ideas with each other, and they were organisationally more flexible.3 For all the self-imagery, there were two other, rather more mundane, factors in the rise of Silicon Valley and the tech sector. First, and contradictory to its own self-identity, Silicon Valley was always supported by the Pentagon. Defence funding might not have
been as important to Silicon Valley as it was to the defence industries in the east, but neither was it negligible. The Valley enjoyed significant funding from the Department of Defense from the 1950s. Second, the emergent tech companies in Silicon Valley were able to draw on venture capital to support their activities. Venture capital was the central and enduring competitive advantage of Silicon Valley. In 1957, the Traitorous Eight lacked capital to develop their products. Consequently, Eugene Kleiner, one of the Eight, sent a letter to his father’s lawyer asking for support; the letter was passed on to Arthur Rock, a New York financier. In late June 1957, Rock met a team from Fairchild in a San Francisco restaurant. He offered them $1 million, recruiting Robert Noyce as a leader. Noyce approached Sherman Fairchild, a prominent businessman and investor, who offered each scientist 10 per cent of the shares of the proposed investment. With this capital, the Eight were able to set up one of the first and most important tech companies, Fairchild Semiconductors. The enterprise was hugely successful. Noyce and his co-founders earned $2.4 million. Rock, as a passive investor, earned more than forty times that figure.4 Following the success of Arthur Rock and Fairchild Semiconductor, a new financial ecology coalesced around the emergent technology companies in the Valley. Financiers began to recognise the potential of technological developments. Unlike more traditional forms of stock, innovations in computing and technology had, as the Fairchild episode showed, the potential to generate exponential dividends. Consequently, from 1957, a network of venture capitalists began to congregate around Silicon Valley. From then on, venture capital remained at the very heart of Silicon Valley; it provided the resources for the revolution which took place there. Moreover, venture capitalists quickly formed an integrated network. Investors knew each other; they cooperated (and competed) with each other to fund enterprises. They came to know the entrepreneurs and tech experts themselves. A tight ecosystem developed between the tech start-ups in the Valley and venture-capitalist funders. This milieu was critical to the success of the Valley and, ultimately, to the
development of AI: ‘In the place of a few smart individuals, there was now a thick web of start-up connoisseurs, significant because the combined force of their actions was greater than the sum of their separate endeavours’.5 The links between the tech primes and venture capital thickened quickly. By the 1970s, the network of venture capitalists was so deep and dense that persistent start-ups, like Apple, were likely to acquire some backing even if several potential backers refused.6 Venture capital underwrote tech enterprise—and continues to do so. By the first decade of the twenty-first century, the ecology was so fertile that entrepreneurs could almost take funding for granted. The relationship between venture capital and the tech sector has been vital in the emergence of AI and, therefore, to its military application. The armed forces are dependent on the tech sector. They would not have been able to begin to develop an AI capability without it. This dependence can be demonstrated by comparing the respective capital of the tech sector and the Pentagon. By the early 2000s, the financial and human capital of the tech sector was huge. In 2023, the US defence budget of $766 billion assigned $875 million to AI and approximately $3.98 billion to software.7 The Combatant Commands were also awarded $200 million as an AI Development Fund. An additional $1.6 billion was allocated to AI and machine learning in the same year. In 2023, the Pentagon’s actual spend on AI research and development was approximately $1.1 billion.8 Plainly, the Pentagon is taking AI seriously and is investing heavily in it. Of course, the US defence budget is vast by global defence standards. For instance, the UK, France, and Germany have far smaller defence budgets, and they invest ‘only a fraction of SinoAmerican budgets in AI/ML R&D’.9 In 2024, France and Germany, respectively, spent $64.3 billion and $77.6 billion on defence.10 In 2023, the UK’s defence budget was $73.5 billion.11 It invested $2.2 billion in research and development. Of that, about $500 million was dedicated to AI.12 In 2023, Israel had a defence budget of $70.1 billion,13 of which $133 million was dedicated to AI research.14 These
AI research and development figures are approximations, but they represent useful guides to expenditure. These defence budgets are not insignificant, and US expenditure is large. Yet, the contrast with the tech sector is stark. The turnover of the US tech primes is huge: in 2023, Google’s (Alphabet’s) annual revenue was $305.6 billion;15 Amazon’s, $574.7 billion;16 Apple’s, $383 billion;17 Facebook’s (Meta’s), $134 billion;18 and Microsoft’s, $211.9 billion.19 Yet it is not just the raw turnover of the tech primes which is so significant. For instance, the US defence budget is bigger than the earnings of any of the tech primes and more than double the earnings of Google, Apple, or Microsoft. Yet the Pentagon’s budget is overwhelmingly pre-committed to operating costs and prior programme commitments. By contrast, each year, the tech primes are able to invest far more in AI research and development and in data, software, and computing power for AI. In 2023, Apple invested $29.9 billion in research and development;20 Google, $45.4 billion;21 Facebook, $38.4 billion;22 Amazon, $85.6 billion;23 and Microsoft, $27.2 billion.24 In 2021, Alphabet (Google) and Meta (Facebook) spent $25 billion to $35 billion, between 12 and 21 per cent of their revenue, on AI research and development. In 2019 to 2020, Palantir and Anduril—important but relatively small specialist defence tech companies—invested $560 million in AI research. Defence Ministries’ and Tech Primes’ Revenue and Research and Development Spending on AI in 2023 TABLE 4.1. Organisation US Dept of Defense UK Ministry of Defence Israeli Ministry of Defence Google Amazon Facebook Apple Microsoft Turnover/Budget ($ billions) 766 60 70.1 305.6 574.7 134 383 211.9 AI R&D ($ billions) 1.1 0.5 0.133 45.4 85.6 38.4 29.9 27.5
On average, in 2023, the five tech primes invested $45.3 billion in research and development. The US Department of Defense—even though its budget dwarfs the defence budgets of allied nations—has not even begun to come close to that level of investment in research and development. The Pentagon’s $1.1 billion investment in AI research and development in 2023 was 2.4 per cent of the tech sector’s average expenditure, while the UK’s investment constituted 1.1 per cent and Israel’s 0.3 per cent (see table 4.1). The tech primes far exceed traditional defence industries in research and development. In terms of the development of AI and machine learning, there is no comparison. The tech primes have invested vast quantities of capital in the development of AI, quantities that would be impossible for defence ministries to match. As a result, they alone have the data, the capital, and the computing power to develop AI for military purposes. They have digital capabilities which the armed forces and the traditional defence industries could never develop but which are imperative to military operations today. Moreover, innovations in the sector are accelerating, as each development reinforces and amplifies others. The innovations in the tech sector are becoming ever more advanced, superseding developments in the defence industries. Human capital has also been vital here. The development of AI has relied on highly skilled, expert labour. All recent AI breakthroughs, such as DeepMind’s AlphaGo (see chapter 1), are traceable back to a small group of brilliant data scientists. Tech companies depend on recruiting and retaining the best computing talent emerging from graduate schools. Indeed, in his recent work on Chinese and American geopolitical competition for AI, Paul Scharre identifies talent as one of the battlegrounds on which AI development will be fought.25 The companies, and the countries, which can attract the best talent will prevail. The armed forces, and even the defence industries, are outcompeted here. The leading computer scientists emerging from the top universities in the US, Asia, and Europe want to work in the best tech companies, not for defence ministries or the armed forces. The tech sector is where the work is the most interesting and rewarding; there, young
postgraduates know they are at the cutting edge of AI. They are operating in an exciting milieu, surrounded by their peers. In addition, for political and cultural reasons, graduate students are often reluctant to work for the armed forces. Yet any reluctance on political grounds is magnified by the raw financial facts—the salaries and working conditions are incomparable. Programmers in the tech sector might routinely start on $200,000–$300,000 a year. They work in chic, postmodern, open-plan campuses, rather than stuffy Pentagon offices. It used to be that the financial sector was the most attractive to graduates. One young tech-company employee described the attraction of working in the tech sector over working in the financial sector, saying: ‘There is more money in tech and more creative freedom. In Google or Anduril, you can be the head of a department at [the age of] twenty-seven; tech is pulling a lot of talent’.26 Tech companies have become vast, powerful organisations. The tech primes are unique, unmatchable concentrations of technical, human, and financial capital. As such, they are indispensable to the development and application of AI. Without tech companies, their massive capital, and their highly skilled workforce, AI as we know it would not exist. Consequently, when we examine how the armed forces will harness AI, it is not just a technological question but also a profoundly organisational one. Since tech companies own the expertise, the computing power, and often the data on which the armed forces rely, the application of AI is not simply a matter of applying a pristine, fully functional machine-learning program to a military problem. On the contrary, we must understand how the tech sector and the armed forces have begun to form a partnership that has been indispensable to any application of AI. It is also necessary that we consider how the armed forces have collaborated with Silicon Valley and tech companies. In the West, this means that it is essential to analyse how Western militaries and their defence ministries partner with the tech sector in order to operationalise AI for their purposes. Obviously, the US armed forces have a natural alliance with Silicon Valley, since those companies are indigenous American ones.
However, other Western armed forces, such as those of the UK and Australia, are having to develop a relationship with Silicon Valley too. The Realignment of Silicon Valley Silicon Valley has always constructed itself as—and been understood to be—part of the Californian Bohemian counterculture, in contrast to the stiff East Coast. Its long-haired executives wear jeans and flipflops, not suits and ties. There is some truth to the image; tech companies in Silicon Valley have tended to be managed by younger executives, who are often the technological innovators themselves, perhaps more than has been the case in East Coast industries. The entrepreneurialism of Silicon Valley has been one of its principal competitive advantages. Operating in a ruthless competitive market, without the protection of routine government subsidiaries, Silicon Valley tech companies have had to prioritise a ruthless drive to innovate. They are profoundly entrepreneurial. Venture-capitalist backers, who have underwritten the tech sector since the 1950s, are oriented to profit. They want to make money.27 By the 1990s, Silicon Valley and the tech sector had adopted a distinctive political outlook. These companies, and the venture capitalists who backed them, incarnated a spirit of radical, neoliberal globalisation. They rejected state intervention and advocated the elimination of national economic barriers to free global trade. Of course, the prime goal here was economic growth. However, globalists believed that, as the world economy became more integrated, globalisation might encourage non-Western states to align their social values and political principles more closely with liberal and democratic values. The culture of Silicon Valley, drawing on its leftist past, was radically libertarian, then. The tech primes and their employees typically understood themselves to be resistant to the government and the state. Digital technology and AI represented a liberation of the individual from state regulation and control. The tech primes were deeply suspicious of, even hostile to, the US state and especially to its security apparatus: the National Security Agency, the CIA, the FBI, and the Pentagon.
As a result, in the last decade, there have been several disputes between the tech primes and the US government. On 2 December 2015, in San Bernardino, California, a pair of Islamicists attacked civilians who were attending a Department of Public Health training event; they killed fourteen people and injured twenty-two more. Following the attack, the FBI wanted Apple to unlock the iPhone 5c owned by one of the perpetrators. Apple refused. The FBI petitioned Apple to create an operating system on its phones which the US security services might de-encrypt if necessary. Apple again refused. At that time, Silicon Valley companies saw themselves as private enterprises whose sole responsibility was to their individual customers, not to government, much less to the United States. So Apple’s position was not unusual. In 2017, Google won the contract for a major new Pentagon programme called Project Maven (see chapter 5), the aim of which was to use AI to process the huge amounts of full-motion video generated by uncrewed aerial systems.28 However, in March 2018, four thousand Google employees protested the contract in a letter to management, declaring, ‘We believe that Google should not be in the business of war’.29 Faced with that level of internal resistance, Google did not renew its contract with Maven once the original contract period ended. In the last decade, the US Department of Defense thus confronted an intricate problem. Precisely because Silicon Valley was the undisputed leader in AI, the Pentagon needed Silicon Valley and the tech companies. Yet Silicon Valley, with its ethos of freedom, radicalism, and rebellion, was sceptical of, and often resistant to, the US government and its security needs. The Google workers’ protest of Project Maven in 2018 was the most extreme expression of this divide between American libertarian capitalism and the state. It was a disturbing moment for the Department of Defense. If the tech sector refused to cooperate, the US might be unable to compete with China in an AI arms race. The fear was that the resistance of the young employees at Google would spread to other companies. The Google protest did denote an important shift in the political and cultural orientation of Silicon Valley—not the beginning of a
conflict between the Pentagon and the tech sector but the end of one. In 2019, when Microsoft employees made similar complaints about a $480 million contract with the US Army to supply it with HoloLens headsets, Microsoft management refused to withdraw. Satya Nadella, Microsoft’s CEO, pointedly remarked, ‘We made a principled decision that we’re not going to withhold technology from institutions that we have elected in democracies to protect the freedoms we enjoy’.30 After the Google protest, a new settlement between tech, the US government, and the Pentagon began to develop. In fact, 2018 might be taken as the moment when the tech primes realigned themselves from a radical libertarian, globalised ideology, in which they felt no allegiance to the United States, to supporting US national interests. There are several ways to document the nationalisation of Silicon Valley. However, the biographies of leading tech-sector entrepreneurs illustrate the recent political transition of Silicon Valley with particular clarity. There are many entrepreneurs to consider here, including Sergey Brin and Larry Page, Steve Jobs, Bill Gates, and Jeff Bezos. However, in relation to US defence policy and the military application of AI by the US armed forces, five figures have played a particularly prominent role: Peter Thiel, Elon Musk, Eric Schmidt, Alex Karp, and Palmer Luckey. These individuals are not uncontroversial. Peter Thiel donated to Donald Trump’s 2016 presidential campaign, and he bankrupted the media blog Gawker, allegedly because it outed him as gay against his wishes. Elon Musk has frequently expressed radical or offensive opinions, and, with no political experience, has recently been appointed as a senior adviser to President Trump.31 Nevertheless, their own ideological trajectories exemplify the reorientation of the Silicon Valley tech sector from libertarian to nationalist. Let us focus here on Thiel and Schmidt, both of whom have played a crucial role in fostering the rise of a military-tech complex. In chapter 10, I consider Elon Musk. Peter Thiel, born in 1967, studied philosophy and then law at the University of Stanford. He did not conform to the image of a typical Silicon Valley entrepreneur. He was socially conservative—he rejected radical theories of sex and race which were popular among
students in the 1980s and 1990s, and he helped found a student journal, The Stanford Review, in explicit opposition to the leftist consensus on campus. However, Thiel was also a libertarian. As an undergraduate, he was a committed Republican and believed strongly in the power of the free market. He had read and been captivated by Ayn Rand’s work and was a follower of Ronald Reagan. Thiel graduated from Stanford Law School with the intention of beginning a career in law. His rejection from a Supreme Court clerkship in 1994 was a devastating blow which precipitated a ‘quarter life crisis’, forcing him to change direction.32 He worked in New York for Credit Suisse for a year; but, noting the rise of the internet and dot-com firms, he began to be attracted by the potential of the digital economy. In 1996, with Max Levchin, he formed PayPal, a company that aimed to facilitate digital financial transactions. In 1999, Elon Musk became a partner in the venture, and PayPal eventually merged with Musk’s company X. PayPal is currently worth $300 billion. The philosophy which underpinned PayPal is immediately pertinent to the question of the realignment of Silicon Valley. In the 1990s, Thiel conceived PayPal as a company that embodied the free market, liberal principles which Thiel had espoused since he was a young man at Stanford. The company subverted financial regulation, placing consumers in direct contact with business, nationally and internationally. PayPal promoted the interests of the individual consumer over those of regulators and the state. In a 1999 speech, Thiel articulated these economic principles clearly: In the future, when we make our service available outside the U.S. and as Internet penetration continues to expand to all economic tiers of people, PayPal will give citizens worldwide more direct control over their currencies than they ever had before. It will be nearly impossible for corrupt governments to steal wealth from their people through their old means because if they try the people will switch to dollars or Pounds or Yen, in effect dumping the worthless local currency for something more secure.33
PayPal was, then, ‘a libertarian company’. The biographer Max Chafkin later wrote of PayPal, ‘In Thiel’s most extreme imaginings it had been a way to unilaterally strip governments of the power to control their own money supplies’.34 In the late 1990s, Thiel was an arch-globalist. He represented the ethos of Silicon Valley. Things changed quickly, though. The terrorist attacks on 11 September 2001 were damascene for Thiel. He became ‘consumed by the threat posed by Islamic terrorism’ and concerned about the security of the US: ‘He was growing skeptical of democracy, of immigration, and of all other forms of globalization’.35 Consequently, he began to reorient his politics. Rather than desiring globalisation, he wanted business to serve the US and to promote its defence and security. He began to re-interpret the mission of Silicon Valley. From 2001, he committed himself to technological development which would ensure US prosperity and, therefore, security; he sought ‘to bring the military-industrial complex back to Silicon Valley’.36 The primary mission of Silicon Valley was not to free consumers, as he had once believed, but to defend the United States. Following his political re-education, in 2002 he founded Palantir, a tech defence prime dedicated to the US Global War on Terror. Later, he helped to raise the capital for Anduril, which has become another important tech defence company. Eric Schmidt may be less notorious than Thiel, but he has played an important—perhaps decisive—role in reorientating Silicon Valley towards the Pentagon. He served as Google’s CEO from 2001 to 2011 and then as its chair from 2011 to 2015. His executive roles at Google ensured his prominence in AI debates in the US. For instance, in 2008, after campaigning for Barack Obama, he was appointed to the President’s Council of Advisors on Science and Technology (PCAST). Schmidt was an advocate of the free market, and he declared that he was ‘very proud’ that he had helped Google avoid paying billions of pounds of tax in the UK. In 2013, he wrote The Digital Age with Jared Cohen, and in 2014 he published How Google Works with Jonathan Rosenberg. These works argued for the deregulation of the tech sector in order to liberate innovation. Nevertheless, in the
last decade, the defence and security of the US have become central to Schmidt’s political outlook. Having left Google, Schmidt occupied a series of important posts in Washington in relation to AI and to national security. He has retained an advisory role on the board of Google, but from 2016 to 2020, he was the chair of the Defense Innovation Advisory Board, a Pentagon initiative to accelerate the development of military technology. From 2019 to 2021, he served as chair of the National Security Commission on AI (NSCAI), overseeing its final report. In those roles, he became an influential power broker in defence policymaking, connecting Silicon Valley, Congress, and the Pentagon. Following his experiences in Washington on the NSCAI, in 2020 Schmidt wrote an opinion piece for The New York Times in which he expressed deep concern about the threat posed by China’s technological advancements, saying ‘America’s lead in artificial intelligence, for example, is precarious’. For Schmidt, the danger was not just economic, though that was not irrelevant. Chinese supremacy in AI was a clear and present danger to US security. Schmidt urgently recommended a suite of actions: ‘The government should begin by setting out national priorities across emerging technologies, with a special focus on research areas that could enhance our defense and security’. Congress, he said, should double its funding of quantum computing, AI, and biotechnology research. That was not all: ‘At the same time, Congress should meet the president’s request for the highest level of defense R&D funding in over 70 years, and the Defense Department should capitalize on that resource surge to build breakthrough capabilities in A.I., quantum, hypersonics and other priority technology areas’.37 In order to exploit the increased research and development funding, Schmidt wrote, a new relationship between the tech sector and Washington was required: ‘We need unprecedented partnerships between government and industry. For example, a partnership should expand affordable access to cloud computing for university researchers and students’.38 The Age of AI and Our Human Future, by Schmidt, Henry Kissinger, and Daniel Huttenlocher, was a major political statement
as well (see chapter 1). In October 2021, using his own capital, Schmidt founded the Special Competitive Studies Project (SCSP), an issue-specific think tank with a three-year contract. SCSP has slick offices in Crystal City, Virginia, just across the street from Amazon’s Washington, DC, headquarters. It has a remit to analyse the future trajectory of warfare and to highlight the capabilities—especially in AI, autonomy, and robotics—which the US will require in order to sustain its primacy. Its objective is to facilitate legislative change around AI by engaging with policymakers in the administration, Congress, and the Pentagon. To that end, it has published several influential reports and many shorter essays on diverse topics. Yet the themes of regulatory reform and the construction of an alliance between tech companies and the military are a common thread among them. In a recent report, for instance, SCSP proposed a similar argument to Schmidt: because ‘tech platforms are powerful assets in Strategic Competition’, it was imperative to understand and to foster the relationship between Silicon Valley and the armed forces: ‘The actions of many major tech firms have advanced or coincided with the interests of the United States and its allies—even if this power is not closely coordinated with the government. We need a methodical analysis of technology companies’ role in the expanding definition of national security’.39 In summary, the tech sector, emerging in Silicon Valley in the 1980s and 1990s, was initially sceptical about working with the Pentagon. These companies and their owners understood themselves to be in the vanguard of global, free-market capitalism. However, in the last two decades the tech sector has recognised the threat from China and other authoritarian states and has increasingly sought to support US national interests. Thiel and Schmidt illustrate this political realignment of Silicon Valley. Without that reorientation, which has enabled critical regulatory reforms, the US armed forces would have had no chance of adopting AI.
Regulatory Reform Between 2001 and 2020, the tech sector realigned itself from a sceptical, anti-government, libertarian stance to a nationalist position. The sector had become concerned with competition from China and with the security and prosperity of the US. This was a vital shift. If major tech companies such as Google had continued to refuse to work with government, the potential for application of AI to military operations would have been severely limited. Necessary though the political realignment of Silicon Valley has been, it is not in itself sufficient. In order to ally with the tech sector, the Pentagon has had to try and reform its own acquisitions and procurement processes. During the Cold War, the Pentagon developed an effective, if inefficient, system of procurement. It employed a ‘waterfall’ method of procurement. The armed forces specified the technical functions they required from new equipment. The defence industries then manufactured platforms and weapons to those specifications, and the equipment, once developed, was procured centrally and distributed out to the forces. This system of procurement has been much derided, but it aimed to control spending and to stabilise the process of acquisition over the long term. This regulatory environment has not been amenable to encouraging collaboration and partnerships between tech companies and the armed forces, which politicians such as Robert Work, entrepreneurs such as Thiel and Schmidt, and some military commanders have advocated. On the contrary, established regulatory frameworks and norms in defence have often actively obstructed cooperation across the sectors. As Colonel Drew Cukor, who led Project Maven under General Shanahan, observed, ‘You don’t buy AI like you buy ammo’.40 Tech companies have, for the most part, developed not hardware, such as exquisite military platforms and weapons, but software. They have sold new forms of digital communication, data, computer programs, and, of course, AI. Unlike the traditional arms industries, the tech primes have been primarily oriented to a civilian market. That civilian market is
potentially global; almost everyone can buy a smartphone. Moreover, tech companies have principally made their profit not by making a single sale of a device, such as a phone, but rather by locking consumers into services—in the form of software updates and expanded functions—for that device over the long term. Competition in these markets has been furious. Consequently, the pace of change has been bewildering. In the twentieth century, it took years, sometimes decades, to develop military technology. With little or no competition, arms companies developed new platforms slowly. The market in which tech companies operate has been entirely different. Without question, hardware remains important, but the main developments have been in software. With software, operating systems have been updated regularly. As a result, the civilian tech market has been dynamic; new software is being developed all the time. The capability of that software has changed so rapidly that it has often been difficult for the programmers themselves to predict what functionality might be possible in eighteen months. Thus, tech companies have operated in a different way from twentieth-century commercial companies. Their relations to their customers are different. Instead of a one-off sale of a discrete product, tech companies offer an ongoing service to their clients. They contract with their customers to provide a service over months or years. The contracts are not expensive, yet tech companies have tied their customers in over the long term and, exploiting global markets, looked to develop truly massive markets. Precisely because the tech sector is developing software, the old Cold War procurement model is increasingly archaic and obsolescent. Traditional waterfall methods of procurement, which tied arms producers into long, highly specified contracts, are obstructive to tech companies. Tech companies want to form a client relationship with the military, developing AI and software together. Consequently, in order to harness the tech sector, defence ministries have been forced to revise their procurement processes. This has been difficult for the US and even more difficult for other Western states, where the relations between industry and defence ministries are heavily
legislated and formalised, compounded by institutional interests and established norms. The system has become ponderous and inflexible. In the last five years, Silicon Valley has reoriented itself politically. At the same time, the Pentagon has tried to reform its procurement system. It has tried to become more agile and flexible. The US government has tried to reform defence regulation to stimulate and encourage these partnerships. In 2014, the US announced its Third Offset Strategy. This was a watershed for the defence sector. For the first time, the US government began to reform the procurement and acquisition system in order to encourage tech companies to work with the armed forces. Following the announcement of the Third Offset Strategy, the Pentagon established several initiatives to accelerate the partnership between the armed forces and the tech sector: the Defense Innovation Unit, Project Maven, the Joint Artificial Intelligence Center (redesignated in 2021 as the Chief Digital and AI Office), and the Joint Enterprise Defense Infrastructure. In each case, these initiatives aimed to link the armed forces to the tech sector to accelerate the adoption of AI. A new procurement system began to appear. The Defense Innovation Unit Experimental (DIUx) was founded in 2015 as part of the Defense Innovation Initiative; in 2017 it was redesignated as the DIU, dropping the ‘x’. In order to foster its relationship with the tech sector, the DIU has its headquarters in Silicon Valley. It has additional offices in Austin, Texas; in Boston; and inside the Pentagon. It aims to identify new technologies and companies in order to catalyse relations between those enterprises and the armed forces: ‘The Defense Innovation Unit (DIU) strengthens national security by accelerating the adoption of commercial technology throughout the military and bolstering our allied and national security innovation bases. DIU partners with organizations across the Department of Defense (DoD) to rapidly prototype and field dual-use capabilities that solve operational challenges at speed and scale’.41 The aim of the DIU was, according to Ashton Carter, the secretary of defence from 2015 to 2017, ‘to get serious about AI’. The DIU was, therefore, designed to short-circuit traditional procurement models. Instead of specifying a function and
a set of technical requirements for contracts, the DIU has sought to identify a problem, allowing tech contractors to work with their military clients to develop a solution. Tech contracts define not a specific platform, but rather a service which a tech company will deliver to the armed forces. Instead of the laborious process of specifying requirements and offering long-term contracts, the DIU has developed a method of ‘prototyping’. It has selected promising tech projects, invested in them, and assessed rapidly whether they had any potential. If they showed promise, further contracts followed. If not, they have been cut: ‘DIU is the only DoD organization focused exclusively on fielding and scaling commercial technology across the U.S. military at commercial speeds. Our expert team, working in seven critical technology sectors, engages directly within the venture capital and commercial technology innovation ecosystem, many of which are working with the DoD for the first time. Our streamlined process delivers prototypes to our DoD partners, along with scalable revenue opportunities for our commercial vendors, within 12 to 24 months’.42 Between June 2016 and September 2022, the DIU initiated 157 prototype projects. Of these, it ‘transitioned’ 52 commercial solutions to Defense Department users. The total value of the production contracts awarded to commercial companies was $4.9 billion.43 Significantly, the number of proposals received increased from 275 in 2016 to 1,636 in 2022.44 In 2017, Robert Work established the Algorithmic Warfare CrossFunctional Team, Project Maven, which aimed to use AI to process the vast archive of full-motion video footage from drone feeds.45 Despite the Google protest, that project was successful; on the basis of that success, in June 2018 the Department of Defense established the Joint Artificial Intelligence Center (JAIC), dedicated to deepening the relationship between the armed forces and the tech sector. The JAIC’s mission is not dissimilar to the DIU’s, but its remit is broader. The JAIC was established to accelerate and scale AI integration across the Department of Defense. It was intended to identify military projects which might be amenable to AI solutions and facilitate the necessary commercial engagement. Lieutenant
General Jack Shanahan was appointed the first director of the JAIC. Shanahan had had a long and distinguished career in the air force, with extensive operational experience. Initially, Shanahan did not want the directorship. He had intended to retire after running Project Maven. But when Secretary of Defense Mattis asked him to reconsider, he took the position. Based on his experiences establishing Project Maven, Shanahan was aware of the challenges he faced as the founder of an ‘AI start-up’ that was tasked to bring AI to the entire Defense Department: ‘Everyone thought it would fail. At the start, I had four people, no office, no budget’.46 AI was a bipartisan issue in the US Congress, and congressional leaders wanted the Defense Department to move much faster in its adoption. Mattis recognised that the JAIC required traditional military legitimacy if it was to convince sceptics in Washington and in the services themselves. As a former fighter weapon systems officer who had held multiple commands, Shanahan had the credibility to fulfil the role. He was also technically qualified, since his career in the air force had involved substantial work with command and control, cyber, targeting, and intelligence. Shanahan proved a capable leader. He relied on his experiences with Project Maven to build the JAIC. One of his first priorities was to find operational missions that would most benefit from AI integration. He began with low-risk and low-consequence projects; based on lessons learned from those initial projects, he worked to expand the JAIC’s portfolio. Equally important, the JAIC concentrated on projects that were ‘scalable’, projects that could expand from an initial successful demonstration of success to, in the best case, department-wide adoption. Shanahan was impressed by some ‘hack the bureaucracy organisations’, such as DIU, Kessel Run, AFWERX, and SOFWERX, but they had limitations.47 They could not be expanded across the force: ‘They don’t scale very well across the Department of Defense. They’re not leveraged business models. They take individual problems, they solve them, and they move on to the next one’. The JAIC needed ‘solutions that can be scaled, not one-offs’.48
Shanahan held the JAIC directorship until the summer of 2020. By that time, the JAIC had expanded to 150 people and a five-year budget of over $1 billion. It had initiated and developed a series of successful AI projects. For instance, in 2018 the JAIC developed a prototype for a fire perimeter model that could detect and map wildfires. In the autumn of 2019, the California National Guard began field testing this model, the Fire Perimeter AI system, and found that it shortened a process that could take hours to a few minutes. Another project conducted post-disaster assessment, including flood mapping, damage assessment, and route analysis.49 It is not difficult to infer that unclassified AI programs developed to map fires or disaster damage might easily be utilised for classified military applications. These AI tools might presumably just as easily assess battle damage or the location of enemy artillery as assess wildfires. Whether or not that is true, these examples show that the JAIC has successfully sponsored some effective new programmes. The JAIC has also worked closely with the NSCAI, supporting the NSCAI to draft legislation. Since its founding in 2018, the NSCAI has proposed 120 new laws, of which about 80 were written and passed, leading to new defence contracts with the tech sector. That is an unusually high rate of legislative success.50 The success of the DIU, Project Maven, and the JAIC might imply that regulatory change has been easy. That is not the case. Although the Pentagon has tried to catalyse relations with the tech sector, obstacles remain, both in the public and in the tech sector. For instance, in 2018, as part of his programme of reform, Defense Secretary James Mattis created the Joint Enterprise Defense Infrastructure (JEDI). Having met with Amazon’s Jeff Bezos and other tech executives in 2017, Mattis launched a programme to develop cloud computing for the Pentagon. The idea was to construct a framework in which all the Pentagon’s data could be held securely but easily accessed by personnel. JEDI offered a contract of $10 billion. The Pentagon was trying to attract a single tech prime to create one unified, coherent data system. However, the programme failed. Amazon, along with the tech company Oracle, protested the terms of the contract. They believed that the tender had been
written to favour one contractor on whom the Defense Department had already decided: Microsoft. There were extensive legal debates until the Pentagon cancelled JEDI in June 2021. JEDI was eventually replaced by the Joint Warfighting Cloud Capability, which has made significant progress. Nevertheless, JEDI shows that while the US had made significant progress in introducing new regulation, major obstacles still obstruct cooperation between the tech sector and the military. It is a fraught and complex relationship. The Pentagon and the armed forces are not used to working with the tech sector. The tech sector’s prime market is the civilian one, not the Pentagon, and when defence contracts do come up, tech companies aggressively compete with one another. That competition complicates the tendering process for the armed forces. The emergence of the DIU and of the JAIC have been important moments in US defence contracting. These organisations began to integrate the tech sector and the armed forces. Illustrating their success, tech firms have begun to establish offices in Washington, DC, near the Pentagon. The renovated Crystal City is an important location here. Some of the major tech companies, including Amazon, have offices in the area in order to facilitate relations with Defense Department officials. The US has reformed its acquisition and procurement model so that it can work with the tech sector to develop AI-enabled technology more easily. Yet, even there, organisational norms, vested interests, and the structure of the tech market have impeded cooperation between the Pentagon and tech companies. The UK A new relationship is developing between the armed forces and tech companies in the US. Other states certainly aspire to apply AI to military capability too. However, the trajectory and the speed of such developments have differed in those countries. For instance, although they are major European powers, the UK, France, and Germany lag a long way behind the US. The UK exemplifies the special problems which confront a medium-sized power as it
attempts to harness AI by integrating the tech sector into defence. The US enjoys a massive advantage in developing relations between the tech sector and the Pentagon: all the tech primes—Google, Facebook, Microsoft, and Amazon—are indigenous US companies. Consequently, in order to encourage a relationship with the tech sector, the British Ministry of Defence (MOD) has to engage substantially with foreign, US companies. Given the close security and intelligence relations between the two countries and the deep penetration of US tech companies into the UK economy, the national barrier has by no means been insurmountable. As one leading civil servant involved in developing the UK’s AI capability emphasised, ‘In the UK, it [barriers] is not an issue’.51 The far greater challenge for the UK—and its European peers—is budget. The UK defence budget is about 11.5 per cent that of the Pentagon’s, and the UK spends far less on research and development. As policy documents like its Data Strategy for Defence show, the UK is committed to exploiting the potential of data and AI. It urgently wants to digitise its armed forces. It has allocated funds for defence AI projects. The regulatory environment is also a constraint. In the UK, defence contracts have been mediated through the MOD, which sets the requirements from the outset. This contractual system is designed to ensure transparency and fairness. As in the US, the system was designed principally for the procurement of exquisite platforms, such as aircraft, tanks, and ships. However, the British procurement system is a disadvantageous model for developing software, data, and AI, especially since the associated budget is so small. It is often not worth it for tech companies to bid for defence contracts. For example, the US tech company, Rebellion Defence, pulled out of the UK market entirely in 2022 because it judged that the market was not large enough. One executive I interviewed, who had worked in the UK for the Israeli defence manufacturer Elbit Systems, was critical of the regulatory environment. The executive believed that the MOD’s rules were not conducive to developing relations between the armed forces and the tech sector. The informant contrasted experiences in the UK with those in Israel:
In the UK, I have to fight to have a discussion with the warfighter. We [the Israelis] are more flexible. They set the requirement, then they change. Elbit Systems provide demonstrations; the warfighters decide that they liked or did not like that product. We, as industry, and the MOD are flexible. We don’t stick to the requirement, but we have the flexibility to change to better meet the operational requirement. In the UK, the MOD sets the requirement. It is not agile enough. In the UK, they use a waterfall system. It takes time to deliver, but they stick with the requirement from years ago.52 In Israel, Elbit did not just sign contracts with the Israeli Ministry of Defence but had integrated with operational units in the IDF themselves. Elbit had worked together with those ‘warfighters’ to design, develop, and improve software; it was a long-term collaborative relationship. Although the regulatory environment has been obstructive, the UK has nevertheless attempted to improve the process of acquisition and procurement to encourage cooperation with the tech sector. It has attempted to introduce more agile systems of procurement. To this end, it has developed several new institutions designed to foster military-tech integration. According to The Digital Strategy for Defence, the UK aims to achieve three strategic outcomes by 2025: (1) a ‘Digital Backbone’ enabling the cross-departmental sharing of data; (2) a ‘Digital Foundry’, a software and data-analytics factory; and (3) a skilled community of specialists who can support the MOD’s digital transformation.53 Following the Integrated Review of 2021, the UK established the Defence Artificial Intelligence Centre (DAIC) in 2022. The DAIC’s mission is very similar to the JAIC’s. The DAIC aims to identify defence functions and services which might be made more efficient by the application of AI. It then seeks to locate tech companies which might deliver these services and to facilitate their collaboration with the armed forces. According to the organisation’s website, ‘We are enabling military use cases by working collaboratively with international partners across government, academia, and industry’.54 In its strategy, the DAIC identifies three roles for itself: champion,
enabler, and innovator. The DAIC aims to act ‘as a visionary hub, accelerating the coherent understanding, development and use of AI capabilities’. It will provide ‘common AI services, good practice and a critical mass of expertise to support local adoption’ in order to ‘rapidly develop, deliver and scale AI projects that generate breakthroughs in strategic advantage in support of Defence priorities’.55 In addition, each of the services has tried to adapt its procurement systems to facilitate the uptake of AI. For instance, the UK Strategic Command’s jHub is developing and testing a ‘Sustainable Tech Adoption Model’ to explore new ways of working with industry to procure new capabilities. The Army AI Centre, established in 2023, aims to accelerate the army’s AI procurement cycle. Similarly, the Royal Navy has actively embraced a new prototyping acquisition model with the introduction of NavyX in 2022.56 These regulatory reforms have generated some results. In December 2022, the MOD signed a three-year contract with Amazon Web Services to provide ‘cloud-based skills development and training to thousands of personnel across UK defence’. Doubtless, training will be important, but it is probable that access to the Amazon cloud, its datasets, and its computing facilities will be critical to the British military. In the same month, the MOD signed a three-year, £75 million contract with Palantir. Reuters reported: ‘Palantir will aid military operations and intelligence, further widening access to its software across the defense ministry after a narrower pilot with the Royal Navy that started years ago. Palantir’s software will prompt the armed forces about possible real-time actions and provide predictions on how various choices might play out’.57 The UK has, therefore, begun a concerted effort to forge a partnership between the defence and the tech sectors. There is some clear evidence of successful beginnings. As in Washington, tech firms are beginning to cluster around Whitehall in an effort to cultivate relations with policymakers and budget holders. In the last five years, the MOD has been working with Adarga AI, Rebellion Defence, Palantir, Helsing, and Anduril. Each of these tech companies has located its personnel close to the centres of power—
three of them have offices within one kilometre of the MOD and Whitehall; the other two, within two kilometres. A new geography of military-tech cooperation is appearing in the UK. There have been important developments in the UK, then. However, the UK defence sector is severely impeded by a limited budget, old regulatory mechanisms, and entrenched interests in the MOD and the industrial base. Thus, ‘more time is needed to assess whether this transition is a significant step away from waterfall capability-development processes towards more agile, iterative and user-driven ones’.58 The UK is certainly not alone in Europe in this. For instance, a recent report has highlighted many of the shortfalls in the procurement systems. Defence ministries across Europe have struggled to create an environment which facilitates partnerships with tech. The digitalisation of defence has accelerated over the past three decades, albeit at different speeds, in China, France, Germany, the UK and the US through the exponential increase in the volume and complexity of defence software. However, governments have not adopted competitive business practices for defence software development and procurement. Defence software and AI/ML continue to be developed within waterfall and incremental capabilitydevelopment frameworks which are not optimised for software’s rapid progress.59 It is important not to overstate the scale of regulatory change, yet a new relationship is developing between the armed forces and the tech sector. Defence is trying to forge a partnership with the tech sector because it knows it cannot compete without the tech sector’s support. Alliance Capitalism In order to highlight the impediments which obstruct defence-tech cooperation in Western states, let us consider the example of Israel. Israel is a distinctive, if sometimes controversial, case; there is a close integration between Israel’s defence ministry, its armed forces,
and its tech sector. This nexus has been critical for the military development of AI. Israel enjoys some obvious strategic, organisational, and cultural advantages over the US and the UK. Unlike the US and the UK—and, indeed, most Western states—Israel confronts a clear and immediate strategic threat; this small country tries to defend the integrity of its 20,000-square-kilometre territory. Moreover, the Israel Defense Forces have confronted—and some might say precipitated—a continuous security threat in the West Bank and Gaza since occupying these lands in 1967. Consequently, the armed forces have been gathering data on their Palestinian opponents and on Palestinian civilians for years. In addition, because of Israel’s often precarious strategic position, there have always been close relationships between the armed forces, the state, and military industrial companies in Israel. The armed forces have required the most advanced weaponry to ensure their superiority over their opponents, and Israeli industries have of course wanted to provide this weaponry themselves. Elbit Systems, Rafael Advanced Defense Systems, and Israel Aerospace Industries are dedicated to the immediate defence of Israel. The Israeli state has been keen to foster this relationship, not only because national security is a prime political issue but also because Israel has been keen to promote its thriving weapons-export industry. The military-industrial complex in Israel has been reinforced by intimate, organic ties between the civilian and military sectors. Because Israel is such a small country, unified by the ethnic and religious bonds of Judaism, kinship links have traversed every sector in Israeli society. Personal networks in Israel are constitutive. They have been crucial to the development of AI and its application to defence. In the defence sector, these kinship ties are further accentuated by the fact that Israel operates a system of conscription. Currently, every young Israeli must serve for two years in the military and thereafter serve in the reserves.60 The result is a uniquely close bond between civilian and military sectors and especially between Israel’s three defence industrial primes, Israel
Aerospace Industries, Rafael, and Elbit Systems. The defence industries are partners to the IDF, rather than just contractors. Israeli military officers, policymakers, and defence executives are well aware of the special conditions which they enjoy. An Elbit executive described the relationship as follows: ‘I have the capacity to talk directly with the warfighter. The IDF does not hold the contract. The Israeli MOD holds the contract, but there is a direct line between us and the end user—the warfighter. And the Israeli MOD supports that. If the IDF need a system—an operational requirement—we have a dialogue not with the customer, but with the end user, the warfighter’.61 The executive continued: ‘In the light of an operational requirement, we form a joint team: the MOD, the warfighter, and industry. We work together till we deliver’. One of Elbit’s most important products is the AI-enabled battlemanagement system Torch. Constant interaction with operational IDF units has been essential in Torch’s development. The intimate relationships between the tech sector, the armed forces, and the ministry of defence have been crucial for the military application of AI in Israel. In the West, organisational conditions have been less conducive to an alliance between the armed forces and the tech sector. However, a new defence architecture is appearing. This defence-tech ecology is different in each country, depending on the size and character of the armed forces and the tech sector. In the West, it is the most advanced in the US, which has the budget and the tech industrial base to have fused civil and military sectors most closely. Yet, across the West, a new political economy is appearing. The armed forces have begun to develop long-term relations with companies such as Google, Amazon Web Services, Starlink, Palantir, and Anduril. It seems probable that these relations will deepen in the next five years. A structural transformation of the defence sector is starting to be observable. The tech companies are providing the armed forces an indispensable service which will become only more important. This arrangement contrasts markedly with the twentieth century. Then, and especially throughout the Cold War, an ‘iron triangle’ consisting of defence ministries, governments, and the arms
industries crystallised. A ‘military-industrial complex’, as President Dwight Eisenhower called it in 1961, dominated the sector; that complex contracted, designed, and manufactured military technology for the armed forces. Today, with AI, a new alliance is observable between the tech sector, defence ministries, and the armed forces. Defence ministries are contracting tech companies to develop new capabilities; those companies are working closely with operational forces themselves to deliver those services. A new triangle is appearing—a ‘digital triangle’. It may be possible to call this new alignment an emergent military-tech complex. It has profound political, strategic, and military implications for how—and even whether—states will fight their wars.
5 The Special Relationship In the last decade, defence ministries have begun to work ever more closely with tech companies. Governments and defence ministries have revised the regulations and procedures around procurement and acquisition in order to facilitate a partnership between tech companies and the armed forces. New organisations, such as the Joint Artificial Intelligence Center, the Defense Innovation Unit, and the Defence Artificial Intelligence Centre, have been created to accelerate the cooperation between the tech and defence sectors. The emergent alliance between the tech sector and the armed forces is very important and must be a starting point of any discussion of the military application of AI. Military-Tech Lifeworld In order to understand the reality of an emergent military-tech complex, it is essential that we move from the level of policy and regulation to actual working relations between the two sectors. We must understand how particular tech companies have collaborated with the armed forces. In short, we must move from the more abstract level of regulatory reform to the lifeworld of military-tech cooperation itself. Here, the new regulation that has facilitated the appearance of a military-tech complex does not adequately capture the concrete, living interface between the armed forces and tech companies, much less that between military personnel—the people running operations—and the data scientists, engineers, and programmers contracted in to support them. Yet social interactions,
interpersonal relationships, and emergent collaborations between professionals from two entirely different sectors have been essential to the application of AI. In order to utilise AI, military personnel have collaborated with tech experts on specific operational problems. In this chapter, I begin to illustrate these emergent webs of interconnection. Palantir In order to understand this relationship between the tech sector and the armed forces, we might wish to look at a number of tech companies. Over the last two decades, the armed forces have worked closely with many tech companies, including Microsoft, Google, Amazon Web Services, Palantir, Anduril, Rebellion Defence, Adarga, Rhombus, and Helsing. Any one of these companies exemplifies this germinating relationship between tech and the military. However, for clarity, it is useful to focus on one company: Palantir Technologies. Palantir is certainly not uncontroversial. Peter Thiel, who founded the company and remains its chair, is an often divisive figure. The company was implicated in the Cambridge Analytica scandal exposed in 2018, and, in the defence sector, it is regarded as an opportunistic, monopolising enterprise. Yet it is one of the leading tech defence companies of the last decade. Consequently, it is a perspicuous example of wider processes. In particular, it usefully reveals the distinctive utility of AI for the armed forces. It is notable that Palantir has not attempted to automate weapon systems, much less military operations. Rather, it has tried to develop AI programs capable of processing data at speed and scale. It has helped the armed forces to organise and curate their data in order to improve their situational awareness and intelligence, expediting planning and targeting. In this way, it has fulfilled the strategic visions repeatedly articulated by Robert Work, governments, and the armed forces as discussed in chapter 4. Thiel founded Palantir in 2003. As noted in the previous chapter, following the 11 September 2001 attacks, Thiel wanted to realign a
traditionally anti-authoritarian, leftist Silicon Valley with US national security interests. Palantir was designed to contribute directly to US security by providing software support to the Department of Defense. The concept of Palantir arose from Thiel’s experiences at PayPal, which he founded with Max Levchin in 1996. From its inception, PayPal faced a serious challenge: fraud. In 2000, the company was losing more than $10 million a year to fraud. Since it processed hundreds of thousands of transactions every minute, it could not review each one. Criminals took advantage of this. Consequently, Thiel sought to use AI to help security-check customers, confirming thousands of transactions instantly. Levchin assembled an expert group of mathematicians to study fraudulent transfers in detail. Eventually, they wrote an algorithm to identify and automatically cancel bogus transactions. However, the thieves quickly recognised how the algorithm worked, and they changed their tactics. While the fraudsters’ adaptive evasions fooled automatic detection algorithms, they did not deceive humans.1 Consequently, Levchin developed a hybrid model. Levchin’s engineers rewrote the algorithm to flag up suspicious transactions; human operators made the final judgement of whether fraud was involved. The system was dubbed ‘Igor’, after a Russian fraudster who was one of PayPal’s most consistent abusers. Partly thanks to the reduction in fraud, PayPal soon turned its first quarterly profit. Thiel subsequently sold PayPal to eBay for $1.5 billion in 2002. Following its success, the FBI requested that Igor software be used to assist the agency in identifying financial crime. Palantir evolved from the anti-fraud software which Thiel commissioned at PayPal. In 2003, Thiel proposed Palantir to Alex Karp, who was an old friend of his from Stanford, and Stephen Cohen, a software engineer. They would create a new company which would use the software pioneered at PayPal to identify terrorist networks and financial fraudsters.2 Palantir aimed to help the CIA, the National Security Agency, and the Department of Defense because ‘its software analyzes the data the government feeds it—phone records of radical clerics in Yemen or bank accounts linked to terror cell activity, for instance—and flags suspicious activity
for a trained analyst to review’.3 The potential security applications of Palantir software were broad. The software might help predict where IEDs were planted in Afghanistan, take down child pornography, support the US Centers for Disease Control and Prevention, or detect fraud for banks and governments. After a relatively slow start, Palantir began to develop software for US military forces which could analyse data to identify terrorist and insurgent networks. In the early 2000s, the US Army was developing a digital intelligence system, based on an existing US Air Force system called the Distributed Common Ground System (DCGS). The DCGS was intended to collate and unify all the intelligence data being collected by an army formation so that all the units in that formation could access the most recent information. The DCGS attempted to develop a rich, accurate, and accessible intelligence picture of an operating area. The system was plagued with difficulties, however; here, Thiel and Karp saw an opportunity. To launch PayPal, Thiel had developed a novel marketing technique. In order to create and then monopolise the market for digital transactions, Thiel had to find customers; the initial user base for PayPal was twenty-four people, all of whom worked at PayPal. Thiel refused to use banner advertising, believing it too expensive. Instead, he paid users to use PayPal and then paid them more to recommend the service to friends. After five months of use, there were hundreds of thousands of users. Thiel had cultivated an interpersonal client network of customers; as he wrote, ‘We needed a smaller niche market segment with a higher velocity of money’. Like eBay, Thiel leveraged ‘PowerSellers’ who were able to influence other consumers.4 From that niche, he was able to access a wider market. From the Second World War, the defence industries normalised a strategy for winning contracts; arms manufacturers hired three- and four-star generals and admirals as they retired. The individual leadership qualities of these officers were not so relevant; they were not being employed to run the defence companies. They were employed for their contacts inside the Pentagon or other defence ministries. In the course of their long careers, these senior generals
consolidated a unique network of contacts in critical offices and roles in the armed forces and ministries of defence. They had access to the civil servants, military officers, and budget-holders who contracted equipment programmes. Moreover, although they had retired, they often still exerted some influence on defence policy and military doctrine. In many cases, these senior generals had operated or commanded the platforms which the company which hired them had manufactured. These generals tended to be drawn from the heart of the conventional forces; they were intimately familiar with mainstream military hardware such as tanks, fighter planes, and aircraft carriers. Consequently, retired senior officers were a precious resource for the defence industries. They connected industries with the decisive influencers and policymakers inside the ministries, and they could persuade the forces to bid for the major platform programmes which a company was developing. Retired generals and admirals were decisive agents in the military-industrial complex. From 2003, Palantir adopted a radically different model. The company adopted the sales strategy that Thiel had developed at PayPal; Palantir leveraged personal contacts to form an individualised network in the defence sector. As it started to market its software, the company had no dedicated sales team; it employed only technical experts who understood the software. Every single employee was responsible for sales; Palantir ‘doesn’t employ anyone separately tasked with selling its product’, said Thiel.5 The company then identified high-status, influential military clients who might act as super-sellers for Palantir in the defence sector. It curated an interpersonal network with these clients. Building an interpersonal network was not easy. For instance, Alex Karp, the CEO of Palantir, spent ‘25 days a month on the road meeting with potential clients’.6 Palantir employed interpersonal networks not merely to identify and recruit clients but also to leverage the regulatory authorities and to facilitate the company’s operations in the US and overseas. Karp’s interpersonal networking was critical in advancing Palantir. In 2009, despite US Army opposition, Palantir began to sell software directly to army units on operations in Iraq and Afghanistan: ‘Palantir adopted a strategy of targeting mid-level Army
commanders who might be open to trying out something new, giving them free versions of the software to try’.7 Colonel Harry Tunnell was an important agent here. Intelligent, confident, and aggressive, Tunnell was a rising military star. He would later become a deeply controversial figure.8 Tunnell was a paratrooper who had jumped into Kirkuk as a battalion commander in the 173rd Airborne Brigade in 2003. He had been wounded there, shot in the leg by insurgents. When he deployed to Iraq in 2007 as the commander of the 5th (Stryker) Brigade, 2nd Infantry Division, he asked Palantir whether it might modify its software to help find insurgents. While intelligence analysts enjoyed large-bandwidth connections to large government servers, field commanders like Tunnell operated from a laptop. Over the course of the brigade’s deployment, Palantir wrote software which allowed Tunnell’s soldiers to access databases offline and upload the modified data. Squad leaders could build on one another’s work. Palantir provided this service and software to train with for free. In 2009, when Tunnell deployed to Afghanistan with his Stryker Brigade, he tried to persuade the army to purchase a version of Palantir’s software. The army resisted. Soldiers in the brigade claimed that the delay resulted in thirty casualties. In 2010, the Special Operations Forces bought Palantir software for their operations in Afghanistan.9 This was crucial because, as a strategic group that employs the best-trained and best-equipped fighters to conduct difficult covert missions, Special Operations Forces enjoy a unique status among the US armed forces. Not coincidentally, they enjoy far higher levels of funding than conventional forces. Because they are almost universally regarded as composed of the finest operators, a mystique has developed around them, and thus conventional forces often imitate them—sometimes slavishly. Regular units presume that whatever equipment the Special Operations Forces employ has to be the best. At the same time that the SOF were starting to use Palantir software, Major General Michael Flynn, the International Security Assistance Force Director of Intelligence in General Stanley McChrystal’s headquarters in Kabul, became interested in Palantir. Flynn was an influential and
well-connected intelligence officer who had played an important part in the Surge in Iraq; he had served as director of intelligence in the Joint Special Operations Command in Baghdad, commanded by McChrystal, with whom he shared a close personal relationship. He recognised the primacy of good intelligence to successful counterinsurgency operations. Indeed, in January 2010, he had published a famously angry paper on the complete failure of International Security Assistance Force (ISAF) intelligence in Afghanistan, titled ‘Fixing Intel: A Blueprint for Making Intelligence Relevant in Afghanistan’. In that paper, he complained that US-led intelligence in Afghanistan focused too much on the insurgents and not enough on understanding local politics. In early 2010, Palantir engineers visited Flynn to give him a demonstration of their software. As Flynn stated, these Palantir representatives were exactly the ‘mercurial people—a little bit eccentric, a little bit bombastic—who we could ally with’.10 Consequently, he submitted an urgent operational request to the Department of Defense to buy enough Palantir licences for the whole force in Afghanistan. He continued to lobby the army for Palantir when he was appointed the director of the Defense Intelligence Agency. By 2011, more than three dozen Special Operations Forces and marine units, and several army units, were using Palantir software.11 The army denied at least seventeen brigades’ requests for Palantir software in the next three years.12 The soldiers who used it found the Palantir software better at disaggregating a huge quantity of data accurately and far easier to use than the army’s DCGS. Major General John Toolan, the US Marine commander of the ISAF’s Regional Command South-West in Afghanistan in 2011, noted, ‘Palantir reduced the time required for countless analytical functions’.13 By 2011, Palantir had a small foothold in the US forces, based on a series of personal connections. However, Palantir had forged relations with important individuals and institutions at key locations in the defence sector. It had a strategic bridgehead. Following the assassination of Osama bin Laden in May 2011, Peter Thiel claimed that Palantir had played an essential role in
locating the elusive al Qaeda leader. In a book about the operation, The Finish, Mark Bowden claimed that two technical breakthroughs had been critical: Predator drones and the Total Information Awareness programme.14 Palantir supposedly produced the program that accomplished the aims of Total Information Awareness: ‘The software produced from this very unlikely source would help turn America’s special forces into deadly effective hunters’.15 Business Week described Palantir as ‘the War on Terror’s secret weapon’.16 Palantir’s software was popularly known as ‘the Killer App’.17 It was not true. Palantir had played a small role at best. Indeed, the journalist Sharon Weinberger observed, ‘No one I spoke with in either national security or intelligence believes Palantir played any significant role in finding Bin Laden’.18 Indeed, some intelligence officers dismissed Palantir’s capability as grossly exaggerated. Nevertheless, Thiel’s propaganda about the company’s role helped to promote its operations yet further. Between 2011 and 2013, the company’s turnover expanded from $2.5 billion to $9 billion. On the basis of its initial programming work between 2009 and 2013, and the interpersonal networks which the company had built within leading elements of the US armed forces, Palantir was eventually able to build a highly effective software program, called Gotham, during the campaign against ISIS between 2014 and 2018. Palantir simultaneously developed a program called Metaconstellation, which integrated data from satellites and could feed Gotham. In the course of the campaign against ISIS, Palantir became skilled at curating data for US forces.19 As General Petraeus observed, the company emerged because it was ‘a better mousetrap when a better mousetrap was needed’.20 During the campaign, Palantir concentrated on ‘operational AI’. It was not interested in abstract questions of general autonomy. Palantir software did not make command decisions; much less did it direct autonomous weapons systems. Gotham, and Metaconstellation, were intended to help commanders make better and faster decisions. As programmers in the company stated, Palantir focused on a simpler question: ‘What are the discrete, specific, limited circumstances in which AI makes sense?’21 Palantir, therefore, concentrated on bounded, specific
problems. It tried to develop software that could process a mountain of data to generate better situational awareness. According to the programmers, ‘Palantir brings data together to make a usable ontology and reduces it to a nonspecialist user’.22 Palantir employees are discreet about the details of Gotham and Metaconstellation. However, the central functions of the programs are clear. Today, the US armed forces rely on multiple sensors across the battlespace, generating huge quantities of data. Palantir has tried to bring together the big datasets gathered from all these sensors and across all organisations. Gotham processes all that data. It codes that data and identifies patterns or signatures the software has been taught to see. Palantir has developed bespoke applications which process specific bits of data.23 These applications sift the data and automatically send warnings of key indicators. For instance, in the fight against ISIS, Gotham helped to build up a picture of the ISIS leadership network and ISIS leaders’ possible locations: ‘It focused on a wire-diagram of the bad guys’. Gotham seems to be able to ingest data from a variety of sources to identify targets in real time. One interviewee, who had used Palantir during the fight against ISIS, described how Gotham worked: ‘Palantir helps organisations convert their myriad data sources, including video streams, into a single coherent whole that can be exploited rapidly and precisely to achieve potentially huge strategic impact’.24 By the end of the campaign against ISIS, Palantir had developed AI software capable of collating and analysing massive datasets. Palantir also resorted to the law to assert itself whenever necessary. The US Army had always been sceptical about Palantir and had resisted its approaches, excluding it from tendering competition. In 2016, Palantir sued the US Army for preventing the company from submitting a tender to develop a software database. The case consolidated Palantir’s position in the defence market. In 2019, the US Army announced that Palantir had won the $800 million contract for the DCGS outright.25 Although Palantir’s software has become very effective, in his description of the success of Palantir, Thiel himself has never overemphasised the company’s technical capabilities. The AI software
programs which the company has developed have been essential to its operations. Its algorithms have been vital. Nevertheless, Thiel located the critical strength of Palantir elsewhere: ‘Advanced software made this possible, but even more important were the human analysts, prosecutors, scientists and financial professionals without whose active engagement the software model would have been useless’.26 Human expertise was crucial. Indeed, Thiel recognised the importance of ‘complementarity’ at PayPal, when Levchin developed the Igor software. That software was effective not because it operated completely autonomously but because, on the contrary, it was used and refined by skilled human users. As Thiel said, ‘Computers are complements for humans, not substitutes’.27 Thiel has emphasised the importance of humans to the application of AI: ‘People have intentionality—we form plans and make decisions in complicated situations. We’re less good at making sense of enormous amounts of data. Computers are exactly the opposite: they excel at efficient data processing, but they struggle to make basic judgments’.28 Operational AI is not simply about the algorithms. Building an operational AI system requires not mere software but human experts who can build skilled and cohesive teams capable of identifying and solving problems. As Thiel has pronounced: ‘The most valuable companies in the future won’t ask what problems can be solved with computers alone. Instead, they’ll ask: how can computers help humans solve hard problems?’29 Special Operations Forces Palantir has adopted a novel marketing method. It eschewed the traditional procurement processes organised by defence bureaucracies and sold its software directly to military users. It has tried to find ‘supersellers’ in the armed forces who will encourage others to purchase its software. Other tech companies have adopted similarly unconventional sales methods, using a more subtle, informal approach which seeks to create relations with smaller, more agile, specialist military milieus. As a result, those milieus have begun to employ AI, or to
see the potential of AI. In the defence sector, the Special Operations Forces are, of course, among the most important actors. They have become a decisive partner for tech companies. They—not conventional forces or ministries of defence—are the privileged, high-status actors with which tech companies have begun to develop alliances in the last decade. From the first decade of the twenty-first century, Special Operations Forces have acted as a strategic partner for the tech sector. Many tech companies have generated a relation with the military sector through collaboration with Special Operations Forces. This connection has only deepened in the last ten years as AI has become more important. There are manifest reasons why the Special Operations Forces have acted as partners and agents for the tech sector. In the 1950s and 1960s, the Special Operations Forces were a marginal capability. They played a role in conflicts of the postcolonial era, including in Malaya, Borneo, and Vietnam, but they were of little utility to conventional warfare. In the 1970s, the Special Operations Forces took on a counterterrorist role. This proved crucial to them, repositioning them in national defence and security architecture and raising their profile. However, from the end of the Cold War, the Special Operations Forces have become an increasingly central element in American, British, and, indeed, Western military capability. They played a pivotal role in the destruction of the Taliban regime in 2001 and in the Iraq War in destroying al Qaeda between 2004 and 2008. They have subsequently played an important role in Libya, the campaign against ISIS, Yemen, the Syrian Civil War, and the Russo-Ukraine War. They have become a strategic capability. Many commentators have focused on the considerable individual and collective skills of the members of the Special Operations Forces.30 It is true that the Special Operations Forces recruit the best military talent, which they train intensively. However, in terms of the tech sector, their other institutional capacities are more significant. The Special Operations Forces have access to many more resources than other forces do, and they enjoy large budgets.31 The US Special Operations Command (SOCOM) exemplifies this advantage. Before
the attacks of 11 September 2001, the SOCOM budget was about $4 billion a year, but during the war against terrorism, and specifically the conflicts in Afghanistan and Iraq, the budget nearly doubled; by 2006, it stood at $7.4 billion. In the following years, the budget gradually rose even more, from $9.3 billion in 2008 to $10.8 billion in 2012. Notably, SOCOM did not suffer the overall defence expenditure budgetary cuts demanded of the four services at the time. In addition, the SOCOM budget was augmented by Overseas Contingency Operations funding, which in 2015 amounted to $2.3 billion.32 In its 2023 annual budget, SOCOM recorded that it consisted of sixty-seven thousand military personnel. Its official enacted budget was $9.8 billion—less than it had been at the height of operations in Afghanistan, yet still very high relative to the conventional forces.33 For instance, in the same fiscal year, 2023, the US Army reported a budget of $177 billion. The US Army budget was nearly nineteen times the budget of the Special Operations Forces. However, in terms of budget per person, the Special Operations Forces were far better endowed. Even without including additional funding such as the Overseas Contingency Operations funding, SOCOM spent $140,000 per service member in 2023. The US Army, by contrast, spent $37,000 per soldier. It is not just that the Special Operations Forces have more money than any other force. Above all, they have a licence to procure equipment and technology quickly and independently. Unlike the rest of the armed forces, the Special Operations Forces have been routinely able to acquire new weaponry, equipment, and technology on the basis of urgent operational requirements. They have their own acquisition and procurement system. In this way, they have been able to sidestep much of the bureaucracy which so hampers standard military procurement.34 Already advantaged by their small size, they have become more flexible, agile, and entrepreneurial than conventional forces. As a result, they have become organisational entrepreneurs. They are experienced and adept at collaborating with other agencies. As Eyal Ben-Ari and Eitan Shamir have described in their analysis of the Special Operations Forces: ‘SOF take up a variety of boundary-spanning roles as part of highly
adaptable mixtures of alliances, coalitions, ad-hoc formations, and temporary organizational structures’.35 As one tech executive observed: ‘Special Forces across Europe tend to be early adopters of technology. In theory, you’re also getting from an MOD talent pool a more strategic thinker, used to operating with minimal direction against a broader intent, and there’s also some cachet when interacting with clients’.36 The Special Operations Forces are a talented, privileged military milieu. Their freedom from heavy bureaucratic oversight gives them a major competitive advantage in forging a relationship with the tech sector and, therefore, in harnessing AI. In addition to their intrinsic capacities and these organisational advantages, one of the most important features of the Special Operations Forces is their unique position in the politico-military hierarchy.37 As a result of their role as counterterrorist specialists from the 1970s, the Special Operations Forces, especially in the US, the UK, Australia, Canada, and New Zealand, have assumed an unusual position in the national security architecture. They are no longer under direct military command; Delta Force and the SEAL Teams are not subordinated to the army or navy, for instance. In the US and the UK, the Special Operations Forces have their own autonomous commands, such as SOCOM in Tampa, Florida, and the Director of Special Forces in London. These commands are themselves uniquely positioned, with close relations to intelligence agencies and to the government, to which they have immediate access. The Special Operations Forces are in almost direct communication with the White House and Downing Street. At the same time, they have forged a close relationship with national intelligence agencies—the CIA and NSA in the US, and MI5, MI6, and GCHQ in the UK. The Special Operations Forces have also developed close transnational relations with their peer Special Operations Forces units in other countries—and indeed, in the case of the anglophone ‘Five Eyes’ community (the US, the UK, Canada, Australia, and New Zealand), with their allied intelligence agencies. These relations have been critical to the Special Operations Forces, providing them with
access to top-secret information for use in planning and conducting their operations. They have subsequently been exposed to alternative, non-military methods used by the intelligence community and other security agencies. They are free to innovate with their partners in other security agencies. The Special Operations Forces are special not so much because of their intrinsic military capabilities, but because of their unique location in a transnational defence-intelligence nexus. They operate at the very centre of the defence, security, and foreign policy establishment. As a result of their prodigious resourcing and their position in the security apparatus, the Special Operations Forces have a latitude—and a budget—to innovate and adapt that is not accorded to conventional forces, making them the ideal partners for the tech sector. Finally, if all these advantages were not enough, the Special Operations Forces are almost continually committed to operations. Indeed, in the last two decades, Special Operations Forces have been constantly deployed on combat operations around the world, many of them involving intense warfighting. In each case, they have been given clear missions. They have targeted specific adversaries, such as ISIS and al Qaeda, and have supported identified allies, such as the Iraqi, Afghan, and Kurdish forces. On these operations, they have recurrently needed to overcome a series of specific, concrete operational problems. Experts working in AI have stressed the importance of the problems of this type: ‘The key to successful software is identifying a specific, boundable problem [and] then developing a software platform that solves the problem end to end’.38 Conventional forces have often been significantly disadvantaged here. Since leaving Afghanistan, they have mainly been oriented to a general ‘warfighting’ function, not to a specific mission with discrete problems to solve. By contrast, the Special Operations Forces have been able to resolve the specific conundrums they face creatively. They have been able to adopt unusual solutions, procuring the equipment and technology suitable for each task or contracting tech companies which can help them. By the 1990s, the Special Operations Forces were organisationally well-adapted to forge an alliance with the tech sector, and from the
early 2000s a relationship began to develop between the two, especially in Iraq. It is possible to trace the origins of this partnership. From 2003, the US military started to employ data systematically to target opponents during the Iraq conflict. In 2004, a Joint Special Operations Command (JSOC) was formally created in Baghdad, with General Stanley McChrystal as its commander. JSOC’s mission between 2004 and 2008 was to destroy al Qaeda in Iraq and, in particular, to hunt down Abu Musab al Zarqawi, the leader of the organisation. JSOC instituted an industrial-level counterterrorist operation in which troops, mainly from the US Delta Force and the British SAS, conducted missions against al Qaeda networks every night, mainly in Baghdad, Ramadi, and Fallujah. They raided houses and bases to kill or capture al Qaeda members and to acquire intelligence on the networks. It was a remarkable operation, and JSOC became a uniquely networked global organisation, eventually playing a key role in the elimination of al Zarqawi in 2006. To prosecute its campaign, JSOC employed every available intelligence feed. It received intelligence from the CIA, the NSA, MI6, and other national intelligence agencies. It drew on satellite imagery, signals intelligence, phone intercepts, open-source intelligence, and human intelligence. It was deluged with information. Many traditional techniques of collation and analysis applied. JSOC was dealing with complicated information and evidence. Early in the campaign, there were many mistakes, which underscored the importance of improving intelligence collection and fusion. For instance, on 16 September 2004, the British civil engineer Kenneth Bigley was captured by al Qaeda in Baghdad. For three weeks after his capture, JSOC searched for him. Bigley was beheaded by al Zarqawi himself on 7 October. Sadly, Bigley’s al Qaeda handler had been identified in JSOC’s data, but human analysts had missed this detail. As a former Special Operations Forces officer observed of the oversight: ‘It was unacceptable then, now even more so’.39 If JSOC had had access to effective AI that could comb its data, it would have found this evidence and may have saved Bigley. Eventually, JSOC brought in data experts to help; they wanted to apply machine-learning AI to the problem. A team from Rhombus
Power, led by Dr Anshu Roy, played an important role. Soon after earning his PhD in computing from the University of Michigan, Roy had patented a platform for solid-state subatomic particle detection. He had also applied his programming expertise to security problems, setting up the company Rhombus Power and building Guardian, its AI platform. Rhombus Power had assisted the Department of Defense by developing software which could identify patterns in data as an aid to finding terrorists and other enemies. As Roy observed: ‘There is an order in turbulence. It is possible to discern that order, so that you can intervene in a complex problem’.40 Roy joined McChrystal’s headquarters in Baghdad and played an important role in the hunt for al Zarqawi. Roy provided an interesting account of how his team helped JSOC to use AI and data analysis in this process. Roy’s team took all the data from tactical units and developed an automated system for fusing data from various intelligence sources and analysing it at speed. As Roy noted: ‘We aggregated and geolocated that data’.41 JSOC constructed the intelligence framework, determining the mission and, on that basis, identifying its critical information requirements. Roy summarised the process: ‘We mathematized it. We were able to quickly and reiteratively put it into a maths construct that could be encoded and put into an AI system’. Using machine-learning algorithms, Roy’s team was able to identify anomalies and signatures and to make predictions. Roy summarised what he and the other team members had done: ‘Capture everything you can, mathematize it, encode it so that the next set of people have a far easier time’.42 JSOC’s use of data, data analysis, algorithms, and machine learning was not a panacea. JSOC did not succeed because of AI or an algorithm—it succeeded because it had a clear mission and gained a deep understanding of the insurgency in Iraq and al Qaeda. However, by processing big data at a scale quite impossible for humans, AI accelerated and improved the decision cycle. JSOC was able to plan better and target the enemy more effectively. As McChrystal said: ‘As a commander, you check data, then trust it. You develop leaders who know about AI. This information being
connected, it makes sense to them’.43 AI was employed to channel the deluge of information which was in danger of swamping JSOC. JSOC provides perhaps one of the first examples of exploiting data and AI to conduct military operations. It also provides one of the earliest cases of close collaboration between the civilian tech sector and an operational military headquarters executing a lethal campaign. Civilian technicians, such as Roy, were not so much contractors to whom services were outsourced as they were active partners in the operation; they were integrated inside JSOC, working alongside McChrystal and his staff. Since the JSOC operation in Baghdad, the relationship between the tech sector and the Special Operations Forces has deepened. Between 2003 and 2010, Palantir fostered this relationship, and in the last decade, many other tech companies have actively cultivated it. To do this, they have adopted an alternative business strategy to traditional defence industries. Instead of hiring fifty-to-sixty-year-old senior officers, they have preferred to recruit thirty-to-forty-year-old veterans from the Special Operations Forces as marketing executives. For instance, Palantir has hired a prominent member of the US Special Operations Forces in a senior role, while the UK branches of Palantir have employed several young ex–Special Operations Forces personnel. The CEO of Adarga is a decorated former SAS soldier with a distinguished career. Twenty per cent of Anduril’s staff are veterans; of those, many have a Special Operations Forces background. This is no accident. As an Anduril executive noted: ‘Our Mission Operations team (account management/system support) tends to reflect the users much more, though; for Royal Marines, we have a former Royal Marine. For Special Forces, former Special Forces’.44 Seeing as they constitute less than 10 per cent of the armed forces as a whole, the Special Operations Forces are vastly overrepresented in defence tech companies. These individuals are primarily tasked with promoting services and acquiring defence contracts, especially with the Special Operations Forces. They provide credibility, but they also facilitate access to the Special Operations Forces on active duty. They have just left the Special
Operations Forces as non-commissioned officers or field officers with immediate operational experience. They are closely connected with their former colleagues and comrades still active on operations, rather than with senior budget holders and programme contractors in defence ministries. Precisely because their world is a secretive and exclusive one, Special Operations personnel prefer to work with veterans whom they can trust. These veterans understand what problems the Special Operations Forces are trying to solve and are able to communicate to Special Operations Forces personnel how tech might help. They have become the critical interlocutors between two spheres. The architecture of commercial-defence relations has altered. Where formerly the defence industry was locked into the armed forces at the apex of the defence hierarchy—in the defence ministries—tech companies now integrate directly into a privileged milieu, the Special Operations Forces. It is possible to explore this Special Operations Forces–tech partnership more closely by looking at specific examples from the last decade. Operationally, the situations which Special Operations Forces face have become more and more complex, with adversaries located in cluttered, contested urban areas. These adversaries have become hard to identify, as their tactics and tradecraft have improved. At the same time, the proliferation of sensors has created an exponential rise in data. The Special Operations Forces recognise the importance of data and have increasingly sought to use data to target their opponents. As one Special Operations Forces officer put it: ‘Data is the engine. It is the locomotive of strategy’.45 This officer wanted to draw on as many data sources as possible, including open-source information, in order to increase the effectiveness, efficiency, and lethality of the Special Operations Forces. He aspired to ‘turn this organisation [the Special Operations Forces] into Bellingcat with guns’.46 By this, he meant that he wanted the Special Operations Forces to be a data-enabled organisation, enhanced by AI. The problem is that the Special Operations Forces have been in danger of drowning in data. They have too much intelligence to
analyse and not enough intelligence operators. And they have been trying to pursue harder targets without an increase in personnel. The result is that they have missed opportunities. They have missed missions and targets because they have not been able to exploit their datasets.47 They have, therefore, tried to increase the amount of time dedicated to analysis in order to improve targeting and to make the organisation more effective and efficient. Consequently, some Special Operations Forces have tried to construct in-house tech teams, aiming to identify solutions not in months or years—as is the norm with traditional arms manufacturers—but in days. They have then created software solutions to process data within three to four weeks.48 They have iterated at speed, perfecting their software through trial and error.49 For instance, one tech company developed software which allowed the Special Operations Forces to search interrogation reports not only for the phone numbers of a target, but even for phone numbers which might be used by that target. The Special Operations Forces have also been interested in facial recognition and natural language processing systems which can detect in many languages. To exploit data and reduce the gap between opportunities and actual operations, the Special Operations Forces have created multidisciplinary teams. They are mostly military; they typically consist of Special Operations Forces personnel, signals-intelligence specialists, and military intelligence operatives. These multidisciplinary teams have included civilians too: ‘The Special Operations Forces have deployed small numbers of data scientists and technicians forward. It is not an easy relation always, because the civilian technicians have no experience of the armed forces, while the armed forces personnel are ignorant of technology’.50 For instance, in 2021 and 2022, one tech company provided the Special Operations Forces with programmers and data scientists who could run its software. The Special Operations Forces have also recruited computer science graduates from the best universities. Those graduates have been put on contracts and sometimes have deployed forward with units on operations.51 The relations which have developed between experienced Special Operations Forces operators
and young computer scientists have been rather unusual. For example, ‘these kids [the young scientists] have never seen a corporal or a captain but they get a radio message with the coordinates for an air strike’.52 The model has worked successfully in the short term, though. Civilian programmers have found it interesting and exciting to work with the Special Operations Forces on operations. However, retention is an issue. The Special Operations Forces cannot pay civilians engineers the salaries available in the private sector. Consequently, in a few years the Special Operations Forces may lose these technicians.53 Nevertheless, American and British Special Operations Forces have begun to form close relations with tech companies, so that they have been able to apply AI to military operations. Civil-Military Relations The US and UK Special Operations Forces have formed the deepest relationships with the tech sector. Indeed, they have become critical partners and agents for the tech sector. The Special Operations Forces–tech axis has become a milieu of military innovation. However, Special Operations Forces in the West are still hampered by regulatory, labour, and cultural obstructions. It is worth contrasting their experiences with those of other Special Operations Forces. For instance, specialist units in the IDF have often avoided some of the problems which have been evident in the West. Because Israel maintains full conscription, all young Israelis serve in the military. They are then held in the reserves for many years. This pool of specialist labour has contributed substantially to the IDF’s advances in the use of data and AI. The Israeli tech sector is young and vibrant; because every citizen has served in the IDF, there is an organic connection between the military and civilian sectors. Indeed, in the light of Israel’s strategic situation, there is no absolute divide between the two spheres. Israeli industry and its tech sector contribute to the defence of the nation, while the IDF protect civil society. As one Israeli interviewee stated: ‘Some civilians are not
really civilians. And some military industries are not really industries. Some companies are public, and there is close cooperation. They go together. There is close military-industrial cooperation and trilateral cooperation between government, the military, and the defence industries’.54 The IDF have very active Special Operations Forces, including many intelligence and cyber units. Unit 8200, one of the most prominent of these, was founded in 1952. Originally a signalsintelligence outfit, the unit has expanded to several thousand personnel. It is reputed to have been instrumental in the 2007 Stuxnet attack on Iranian nuclear facilities.55 Its role in collecting and analysing communications is now widely acknowledged: ‘Unit 8200 is probably the foremost technical intelligence agency in the world and stands on a par with the NSA in everything except scale’.56 In the past decade, digital communications, social media, email, the dark web, and mobile phone intercepts and analysis have become central to Unit 8200’s work. Data—information in the digital sphere—has become a prime source of intelligence about Israel’s enemies; opensource intelligence is an important speciality of the unit. Unit 8200 is dedicated to signals intelligence in cyberspace; it monitors open sources but also infiltrates and decrypts digital messages of opposition groups, such as Hamas and Hezbollah. It has been involved in digital surveillance of the private communications of Palestinians. Indeed, in 2014, forty-three former members of Unit 8200 signed a letter of protest about this surveillance. The unit’s intelligence role involves the analysis of massive datasets; it is monitoring communication traffic at scale, which requires software and, when possible, algorithms. There seems little doubt Unit 8200 was heavily involved in the operation against Hamas in Gaza in 2021 and in the war in Gaza that began in October 2023. The question is, how does Unit 8200 conduct its operations? The unit is highly expert. Its personnel profile is illuminating. Unit 8200 is substantially comprised of one-year conscripts, who have technical expertise, supported by civilian reserves in the tech sector. It consists of civilian data scientists, programmers, and engineers, some of whom are temporarily in military uniform. As one IDF officer
noted: ‘The expertise comes from conscripts. How do you develop fast the young expertise of Unit 8200? You do it with conscripts. The core of the human resource is conscripts’.57 In developing teams of civilian technical experts and military personnel, the IDF have enjoyed obvious organisational advantages over other militaries. The IDF have been able to establish and evolve Unit 8200 and other specialist tech organisations which combine civilian technicians with military personnel in professionalised teams, committed to a military task but highly adept in AI and data manipulation. In the past decade, when security scholars have addressed the question of AI, they have typically been drawn to the question of military automation. They have suggested that AI is about to automate war itself, superseding human commanders, making strategic decisions and controlling entire weapon systems. They have focused on the technology—to which they impute often unfeasible powers—ignoring the organisational transformations which have made and will make the military application of AI possible. Blinded by the remarkable technical powers of AI, many have overlooked this human collaboration between the tech sector and Special Operations Forces. The collaboration is on a small scale, and it is often mundane. It is always discreet, often covert, and sometimes classified. The Special Operations Forces are but one small node in the armed forces. Yet the emergent connection between the tech sector and the Special Operations Forces is a significant development. In the defence sector, the Special Operations Forces have become the supersellers. They are the market leaders in defence, the ones whom other forces tend to follow and imitate. Through their advocacy for and application of AI, it is likely that the Special Operations Forces will accelerate the take-up of AI in the conventional forces. The link between the tech sector and Special Operations Forces is still nascent. Yet it may be as important politically, economically, and militarily as the birth of the military-industrial complex in the 1950s. It seems to signify the beginnings of a new—military-tech—complex.
6 AI and Planning In order to harness AI, the armed forces have formed everdeepening partnerships with the tech sector. A new defence architecture is appearing. However, in order to understand militarytech collaboration, it is necessary for us to examine how the armed forces are actually using AI. Against much of the contemporary literature, it is very unlikely that AI will replace human commanders. As discussed in chapter 2, second-generation AI has almost infinite powers to process data, but it remains greatly limited; it only calculates probabilities. Command decisions, by contrast, are not based on calculation alone; they require judgement too. For instance, commanders have to define a mission; they have to decide what their forces will do. To do this, they have to judge the political goals that they have been set and then assign military forces in pursuit of those goals; they have to negotiate with political and military superiors, with allies, and with subordinates to determine what is at stake. They have to understand the intentions of their political masters, their peers, their subordinates, and their enemies— and how they interact. They have to assess the political and military effects of their decisions; they have to consider cultural and moral factors which are difficult to digitise or mathematise. In addition, many of the subordinate elements of planning refer to the orchestration of military units; that synchronisation involves negotiation, not calculation. Commanders must recruit the support of military, political, and civilian allies. Commanders rarely order their subordinates to execute a plan as if they were machines. They have
to judge what might be acceptable and possible for their subordinates to do and then instruct them to contribute to the plan in line with their willingness and capacity to comply. Finally, commanders have to motivate their troops to fight; they have to lead, not just manage. The process is interpretive, intersubjective, and iterative. It is not submissible to inductive reason alone; it involves much more than plotting correlations between data points. It requires imagination, interpretation, inference, and abduction. It is improbable that second-generation AI is about to automate command, then. Nevertheless, precisely because AI has become so capable, it can perform a number of valuable functions in support of human commanders. Indeed, as discussed in chapter 3, governments, defence ministries, and the armed forces themselves have identified more concrete functions for AI. Robotics and lethal autonomy are not irrelevant here. As a result of the emergence of AI, autonomous weapons are likely to proliferate more quickly. Yet, as national defence strategies expound, AI has been and will primarily be employed to process data, at a scale and speed which exceed human capacity, leading to improved situational awareness, understanding, and military intelligence, broadly conceived. More specifically, AI will be used to improve three major command functions: planning, targeting, and cyber operations. AI has already proved extremely useful in planning; it has expanded, accelerated, and refined the targeting process; and it has been indispensable for cyber operations. It has helped commanders to conduct cyberattacks, to provide cyberdefence, and to promote information campaigns in cyberspace. In the next three chapters I examine each of these functions—planning, targeting, and cyber operations—in turn to show how AI has enhanced military effectiveness while changing the way in which operations have been conducted. Let us begin with planning. The supreme act of command is mission definition. Without an identified mission, no coherent military action can take place. However, defining the mission is only a start; one must then develop a plan. Military campaigns almost always involve thousands of
military personnel, organised into many subordinate formations and units, dispersed over a large area and engaged in complex tasks. Commanders—and their staffs—need to identify all the tasks to be completed in order to accomplish the mission; they must then prioritise, sequence, and assign those tasks—and coordinate them when the operation begins. It is very difficult to organise all this activity.1 Consequently, for a military operation of any size, planning is essential. Planning is an often onerous and complex process.2 Some aspects of planning are not obviously susceptible to machine learning. Planning involves identifying a series of military tasks which will contribute to the accomplishment of the overarching mission. Planners need to make judgements about what the enemy might do, as well as about what friendly or allied forces can do, and to have an appreciation of the political acceptability of certain courses of action. Military planners need to exploit surprise; they need to develop plans which are not strictly rational or statistically the most efficient. Planning remains an intricate political process, involving human judgement, imagination, experience, and guile. It would therefore be very difficult to automate planning as a whole. Yet, while planning might remain a human endeavour, AI has already substantially facilitated this function. By processing massive amounts of data, AI has enhanced and, in some cases, automated elements of the planning process. Planning is not reducible to calculation, but many important aspects of planning involve numbers: data. A major part of a staff officer’s unenviable duties is purely quantitative. Staff officers often have to account for many factors by calculating the answers to questions such as ‘How many soldiers and guns are in a unit?’, ‘How many troops can march on a particular route?’, ‘How long will that march take?’, ‘What supplies are needed to complete it?’, and ‘How much ammunition is required?’. Consequently, while the creative, interpretative, and imaginative elements of planning will be difficult to automate, there are important sub-processes which are eminently open to AI, because all they involve is data and data processing. It is
vital that we have an understanding of how AI has helped with planning and how it might further help. The main part of the present chapter focuses on AI planning tools which the armed forces have already employed: the US’s Major Combat Operations Statistical Model and several other programs; the UK’s Enhanced Command and Control Spearhead; Elbit’s Torch system; Ukraine’s Krypova and Delta systems; Anduril’s Lattice Operating System; and the Royal Navy’s StormCloud. There are many similar systems which other military forces are currently employing, but these six examples provide a good insight into current capabilities and illustrate the great potential, but also the realistic limitations, of AI and planning. The potential for planning demonstrated by generative AI— exemplified by large language models, such as ChatGPT—seems very significant as well. I will therefore conclude this chapter with a discussion of the application of large language models to planning. The Major Combat Operations Statistical Model (MCOSM) There is little doubt that the US armed forces are in the lead among Western powers in applying AI to planning. The Pentagon has sponsored many interesting projects designed to assist in military planning. It is currently committed to building the Joint All-Domain Command and Control system (JADC2).3 The aspiration is that this AI-enabled battle-management system will allow all US forces to share a common operating picture, fusing all the data from the sensors of every service. JADC2 is a long way from fruition. However, the US has already implemented several AI programs to assist with planning. For instance, engineers at the US Naval Postgraduate School in Monterey, California, recently developed the Major Combat Operations Statistical Model (MCOSM). This program is based on data from ninety-six battles from 1918 to the present. On the basis of its algorithms, it is capable of calculating the likely outcome of future campaigns. For instance, when MCOSM was fed data on the
Russian attack on Kyiv in February 2022, the model predicted the likely operational outcomes; it gave the Russians a 2 in 7 chance of success and the Ukrainian defenders a 5 in 7 chance of success. The predictions were correct; the Russians withdrew from Kyiv on 25 March and by early April had re-deployed their forces to the east and south.4 In July 2022, MCOSM was programmed to calculate the likely outcome of the Russo-Ukraine War. With Ukraine having superior artillery but being outgunned, MCOSM awarded performance scores of 5 out of 7 for both Russia and Ukraine: a stalemate. That is more or less what has happened. MCOSM’s success should not be overstated. Many observers of the Russian invasion of Ukraine would have predicted that the Russian army would, with a force of about fifteen thousand soldiers, be unable to take Kyiv, a city of three million, against fierce military and civil resistance. Only a few key variables—the size of the Russian force, the size of the city, and the size and determination of the Ukrainian military—were enough to generate a sound prediction. Similarly, many commentators predicted a long, brutal war. Yet predictive planning programs like MCOSM are proliferating and, unlike humans, the more powerful ones are able to process multiple variables. For instance, ManTech, a defence firm in Herndon, Virginia, has developed a probabilistic combat predictor called BRAWLER. This system processes data on the performance of aircraft, their various subsystems, ground radar, and missile batteries. On the basis of this data, BRAWLER runs simulations to generate probable outcomes for particular scenarios. For instance, it calculated how evasive manoeuvres might ‘increase an F-16’s chance of dodging a Russian S-400 missile’.5 BRAWLER has typically run simulations with twenty aircraft, but it is able to process three times that number. BRAWLER does not solve the problems of planning, much less executing, aerial operations. Yet it is able to inform commanders about the viability of any plan they are contemplating. Of course, predictive calculations of this type have long been a duty of staff officers. Indeed, the practice of war-gaming was invented in the eighteenth century to calculate these probabilities. War games allowed commanders to test the coherence of their plans in an
adversarial situation. Yet war-gaming consumes staff time and effort, and war-gamers can consider only a few variables. Sometimes, they have already framed the answer they want. AI programs, such as BRAWLER, can accelerate and refine the planning process. The US has also been experimenting with AI for urban operations, as ‘the urban environment […] is one of the most challenging for military operations’.6 In recent campaigns, improvised explosive devices (IEDs) have been a major threat to US forces; they are especially deadly in towns and cities, where troops are channelled along streets, roads, and alleyways. Consequently, the US has begun to explore using AI to identify blind spots where IEDs have not been identified. One AI model developed for this purpose employed reinforcement learning; programmers specified the outcome, while the program trained itself through trial and error. The AI program created heat maps identifying the probable locations of IEDs; researchers concluded that ‘the approach to use competing AI systems in order to assist the commander is feasible and can result in a relatively quick and effective source of intelligence for the planning of military operations’.7 It is possible to create similar programs to help plan urban operations. Urban warfare is challenging because the environment is so dynamic. Cities change radically as buildings are damaged and destroyed by the fighting. Commanders have to be aware of the current topography of the urban area they are fighting over, not what their old maps claim. Yet it is difficult to maintain accurate situational awareness. AI can help here. For instance, one group of researchers explored whether it was possible to estimate the level of building destruction in a war zone more accurately.8 The researchers trained a program called a convolutional neural network (CNN), a machine-learning algorithm, to identify structural destruction in satellite images. The CNN was programmed on high-resolution images from Aleppo and other Syrian cities. The researchers explained, ‘Our method of identifying building destruction combines the existing state-of-the-art computer-vision methods with an additional postprocessing step and exploits the time dimension of the destruction data to expand the training dataset’.9 The
researchers concluded, ‘These results were encouraging and allow applicability for automated destruction classification and even close to real-time tracking for policy purposes’. The Pentagon’s Joint All-Domain Command and Control system (JADC2) may appear in the coming decade. The US has already made considerable progress in applying AI to planning. AI has not begun to automate planning in its entirety. That would be improbable, even if the US does develop JADC2. However, AI has relieved staff officers of some of the burden of planning. It has automated some supporting planning functions, where it has often proved more effective and efficient than humans. The UK’s Enhanced Command and Control Spearhead After attempting to develop an AI capacity for several years, the UK finally published its AI strategy in 2021. In 2019, the vice chief of the defence staff, General Sir Gordon Messenger, supported by the chief scientific adviser, had initiated a programme to accelerate the employment of technology on the battlefield. As part of this programme, there was an effort to examine whether AI might support command and control. The project was consistent with the concept of ‘prototype warfare’ which the Ministry of Defence was exploring at the time. The aim was ‘to go after new ways of working’10 and to test numerous emergent technologies simultaneously, working out quickly which ones might have potential: ‘It was a state up/scale up ethos: move fast, break things. Fail but learn’.11 A number of projects emerged from this process, one of which was the Enhanced Command and Control Spearhead, set up in 2019; it was one of the ‘key transformation programmes under the Integrated Review’.12 The Enhanced Command and Control Spearhead tried to apply AI to support command decisions. Specifically, it was an experiment to determine whether it was possible to use AI to accelerate and improve planning processes, increasing the tempo of decision-making in military headquarters. The Spearhead was eventually established at the Royal Signals Corps base in Blandford Forum, Dorset, in south-west England. The Royal
Signals Corps, which specialises in communications and command support, is one of the most technically advanced services in the British Army. It was believed that the Spearhead would benefit from close integration with the Royal Signals. A British Army lieutenant colonel was appointed director of the Spearhead. Ordered to build a team, he selected a small group, including three lead technicians, an administrator, and additional staff. The working conditions were unusual for the British armed forces, he said: ‘The risk appetite was high. It’s the only time in the Army I’ve ever been allowed to get on with it’.13 The idea of Spearhead was to use AI to improve working practices in military headquarters: ‘Command and control was a vehicle to go after AI. It was to support decisions—across all the joint planning functions. The aim was to get an idea where AI could and could not go’.14 Specifically, the Spearhead tried to use AI to process data, since ‘humans do not have the cognitive capacity to fully benefit from the available data from across multiple domains at the speed of relevance’.15 The team quickly realised some important principles. The key to applying AI to command and control was identifying, curating, and organising the relevant data. The Spearhead operated on the principle of a ‘fire triangle’, consisting of data, algorithms, and architecture; ‘If you can’t define each one, you’re nowhere’. Of the three elements, data was perhaps the most critical. Echoing precisely the NATO general quoted in chapter 3, the Spearhead team declared that ‘data is king’. Gradually, the Spearhead built a database stored on a hard drive, the Microworld, which they instrumented to help in military decision-making. The Microworld’s hard drive is itself housed in a robust and partially weather-proof box about 70 centimetres long, 50 centimetres wide, and 20 centimetres tall, transportable on operations. The software was the key element of the Microworld. It consisted of a virtual model of the operating environment: ‘a Multi-Domain Area of Interest comprised of deliberately selected, structured data, directly relevant to the operational theatre, generating a curated data set that can be exploited for a variety of focused AI technology
applications’.16 Commercial off-the-shelf services and AI agents supplied the basic framework for the Microworld. Commercial data and intelligence services played an important role in the curation of the database itself; commercial intelligence services, in particular, were crucial. Vendors specialising in global military intelligence were selected to supply information on force postures, equipment, operations, and geospatial information.17 With offices worldwide, thousands of staff, and sizeable budgets, those vendors had more capacity than the UK’s Ministry of Defence did to collect intelligence; their information was also very accurate. In addition, the Staff Officers’ Handbook, a British Army planning manual, and other military sources were digitised to create an encyclopaedic dataset. As a result, the Spearhead was able to create an actionable operational dataset which included geospatial information, friendly forces’ locations, the enemy’s locations, tactics, and battle management tools. The Microworld is impressive. However, the Spearhead’s aims were modest. It did not want to automate ‘command’, which would have been an unrealistic and undesirable goal. It wanted the Microworld to process data to help the commander make decisions: ‘The purpose is not to use AI to complete all of the process of decision-making, but only to remove the repetitive tasks and enable informed decision-making for human users’.18 The most successful application of the Spearhead’s Microworld occurred on a multinational exercise. Since 2016, the British Army has deployed a mechanised battlegroup, under a brigade, into a European ally’s territory to reassure local allies and to deter potential opponents. The tour lasts six months, during which the deployed battlegroup normally performs one major exercise. Between 21 and 28 May 2021, a British armoured brigade conducted an exercise as part of their reassurance and deterrence rotation. The Microworld was employed in the headquarters of this armoured brigade. Route planning and geographical surveying are important staff functions during the planning cycle. Commanders have to be aware of the best routes so that they can develop good plans. In the past, staff officers have conducted this work, by studying maps and
assessing the viability of particular routes. Staff officers have identified a variety of factors—such as road widths, choke points, bridges, inclines, and exposure to enemy observation and fire—in order to determine the quality of a route. Route-planning is timeconsuming and tiring work. It normally takes brigade staff officers three to four hours to do this analysis for any operation. Eventually, the staff typically produce a coloured map of routes to highlight various factors. On the military exercise in 2021, the Microworld was used to do route-planning. It conducted the same operation in mere minutes. As one officer stated, ‘It was a prophetic moment’.19 The Microworld not only did complete route-analysis in a fraction of the usual time but was also more comprehensive and detailed. It was able to highlight a route’s vulnerabilities by illustrating the precise points on a route where the enemy could see friendly forces, directly or indirectly. The Spearhead had created a militarised version of Google Maps. The system provided a commander with route choices automatically, documenting all the details and hazards on each route. In the course of the exercise, a very senior British officer observed the Microworld at work. When he saw the program working, ‘he fell off his chair’.20 He enthused that this program would have saved him hours if it had been available when he was a staff officer in the British Army’s Third Division Headquarters. Overall, in the course of this exercise, the Microworld was effective, saving 11.5 hours of staff time and producing 42 products. The products contained 81 per cent more information than traditional staff work and were 91 per cent higher quality.21 The Microworld is a modest innovation. It has not transformed command, and it has not revolutionised military decision-making. Its achievements are much more limited. It has automated one specific staff function: route planning. The Spearhead’s Microworld does not make any decisions for the commander at all. Nevertheless, because it saves hours of labour, it liberates the staff to concentrate on more difficult analytical problems for the commander, where careful human judgement is required. Consequently, indirectly, the Microworld enables the staff to support the commander and to
improve military decision-making. It would be silly to exaggerate the capabilities of the Microworld or the successes of the Enhanced Command and Control Spearhead more generally. Indeed, one senior British Army officer was rather disappointed by the work precisely because it seemed so mundane. He believed that AI could and, perhaps, should produce much more exciting results. However, the Microworld evinces two important facts about AI in relation to military decision-making. First, AI is eminently capable of supporting decision-making, but it cannot assume the role of a human commander. Second, AI can process data to generate useful results, but it cannot really make a decision. It can help in the planning process, but it cannot write an actional plan entirely on its own. Torch and Delta MCOSM, the Microworld, IED heat maps, building-destruction algorithms, and BRAWLER perform very specific and narrowly defined planning functions. However, there are other AI-enabled planning software programs which provide more general command support: namely, digitised ‘battlespace management’ systems. These programs do not simply process specific data to help commanders make a decision; they supply a digital framework which collates and shares all the data in a recognised situational awareness picture, denoting the location of friendly forces and enemy forces. They facilitate the execution of operations and the coordination of forces in real time. Digitised command and control systems have existed since the 1990s. As part of the Revolution in Military Affairs, the US armed forces introduced Blue Force Tracker and SIPRNet, which enabled commanders to see a digital image of the battlefield in real time and to communicate with all their forces securely via digital satellite communication. With the introduction of AI, these digitised command systems have become more capable, and some functions have been automated. Elbit’s AI-enabled battle-management system Torch is a prominent example here. Elbit is one of the three Israeli defence
primes. It has manufactured a number of defence technologies, including some remotely controlled and automated weapons systems. Torch, one of its most innovative technologies, is founded on a digital architecture called the E-CIX, consisting of a large database and a software system. Torch integrates all the data collected by Israeli forces and their constellation of sensors across an operating area to generate a shared picture of the situation. For instance, Torch’s Dismounted-Joint Fires Integrator is specialist subsoftware which can be downloaded to any sensor which infantry soldiers might use, such as binoculars or a laser designator on a rifle. When Israeli soldiers use the Dismounted-Joint Fires Integrator, they can access all the data on Torch, while the Dismounted-Joint Fires Integrator can upload any data collected by the soldiers on the ground automatically to Torch. Torch identifies targets, confirms them automatically, and relays them to all the other users in a platoon or company. Torch also automatically identifies what types of munitions or which platforms might be employed against a given target. Torch also digitises planning. Once the staff input a mission, Torch can help design the battlespace-management methods, such as phase lines, unit boundaries, and known areas of interest. For instance, if a unit is moved to a different command, Torch instructs the communications infrastructure to change automatically; the unit receives only those messages from Torch which are relevant to its new mission. Torch gathers data from sensors, identifies targets, and suggests optimal weapon systems so that a commander can make decisions more quickly and accurately. The commander still decides whether and how to prosecute a target. Torch does not obviate the requirement for military or political judgement. Nevertheless, Torch’s software represents a significant improvement over the digital battlemanagement systems of the early 2000s. Torch is able to collate a vast amount of data from a host of sensors in a single operational picture to identify numerous targets quickly over a wide area and to recommend the assignment of certain weapon systems in response to them.
In 2022, members of the Israel Defense Forces visited the UK to engage with an airborne artillery regiment in the British Army’s leading 16 Air Assault Brigade. Because of its role as a global reaction force, this brigade and its subordinate units often enjoy a higher level of resourcing than other army units. So the Israeli visitors were shocked that their hosts were still calculating targets with paper, map, compass, and binoculars. They wondered: ‘How do they do it? They’ve got no equipment’.22 For the Israel Defense Forces, Torch does not make soldiers redundant; rather, it enables them to operate more effectively. In 2014 and 2015, in the face of the Russian annexation of the Donbas, the Ukrainians developed a battle-management system called Kropyva, designed to counter Russian artillery. Kropyva, which runs on Android, enables a user to mark enemy artillery positions and to transmit that data to a Ukrainian artillery unit, which is able to share the information so that it can synchronise targets in several places. Kropyva reduced the counter-battery response time to thirty seconds, one-tenth the time that was achievable using the previous communications system.23 Since the Russian invasion of Ukraine in February 2022, the Ukrainian armed forces have developed a more comprehensive battle-management system, Delta, which is functionally very similar to Torch. As the Ukrainian vice prime minister and minister of digital transformation, Mykhailo Fedorov, declared: ‘The enemy has been preparing for full-fledged war for 20 years. We made a technological leap in 10 months’.24 In the face of a national crisis, the Ukrainian armed forces mobilised civilian expertise to build a working, if shabby, system. Ukraine was advantaged here in that its Westernleaning younger generation are extremely competent with digital technologies; it has a thriving economy of software engineers, data scientists, and hackers. Delta is an ad hoc system which collates Ukrainian military data to produce a common operating picture that identifies the location of Russian troops. It has been the basis of command and control in Ukraine. Via Delta, Ukrainian forces have been able to share data about the locations of forces and to identify and communicate the locations of Russian targets. Delta software
has been used for tasks such as controlling artillery fire. Delta has also proved extremely hard to jam. Although Delta is not as advanced as Torch, it is a remarkable achievement that within a few months of the start of the war, and even while fighting a potent enemy, the Ukrainians constructed an advanced battle-management system. ‘Ukraine has achieved a cut-price version of what the Pentagon has spent decades and billions of dollars striving to accomplish: digitally-networked fighters, intelligence, and 25 weapons’. There are other examples of when AI has been employed to enhance a battle-management system. Anduril’s Lattice system is pertinent here. Anduril Technologies was launched by Trae Stephens and Palmer Luckey in 2014 with $17.5 million raised from Peter Thiel’s venture-capital firm. The company was initially tasked to create a multi-sensor software system capable of monitoring the Mexican border. To this end, Anduril has developed a suite of hardware products, including automated surveillance sensors, such as the Sentry Tower, the Ghost surveillance drone, and the Dust camera. In each case, these sensors are automated or remote systems. They are programmed to identify objects of interest automatically and to share that data with all the other sensors in the system. Anduril’s central product is the Lattice operating system. Lattice facilitates the fusion of data from all sensors to generate faithful and immediate situational awareness of the battlespace. According to the company’s website, ‘Lattice cuts through the noise and creates a shared real-time understanding of the battlespace. It autonomously parses data from thousands of sensors & data sources into an intelligent common operating picture in a single pane of glass. Lattice uses technologies like sensor fusion, computer vision, edge computing, and machine learning and artificial intelligence to detect, track, and classify every object of interest in an operator’s vicinity’.26 The promotional description continues: ‘Lattice persistently analyzes the environment and artificial intelligence algorithms detect, track and classify objects of interest in any domain. Only relevant points are surfaced to operators through customized alerts allowing them to focus on the mission and not the monitors’.27
Lattice automatically identifies possible threats from its network of sensors and is able to then list which assets are best able to respond to that threat, either by further surveillance or by engaging with it. Lattice presents human commanders and their staff with early warning and options. It aids planning and decision-making. For some years, the Royal Navy has also been exploring the possibility of creating an AI-enabled battle-management system, especially for amphibious and littoral (near shore) operations by the Royal Marines. In 2021, the Royal Navy contracted Microsoft and Amazon Web Services to develop such a system. In collaboration with BAE Systems, in only twelve weeks the consortium developed a ‘mesh’ network called StormCloud. StormCloud connected marines on the ground, drones, and a suite of sensors, collecting and sharing all their data. That data was processed at the edge, on computers strapped to vehicles but connected to cloud servers: ‘Command-andcontrol software monitored a designated area, decided which drones should fly where, identified objects on the ground and suggested which weapon to strike which target’.28 One officer described StormCloud as ‘the world’s most advanced kill chain’.29 StormCloud ‘allowed the marines to run circles around much larger forces in previous exercises’.30 The examples of Torch, Delta, Lattice, and StormCloud are instructive. Commanders and their staff still plan military operations. AI systems have not automated planning, to say nothing of military decision-making or command itself. However, AI-enabled software programs such as Delta, Torch, and StormCloud have helped military commanders and their forces by collating, analysing, and displaying a huge quantity of data. The result is that while many planning functions remain manual, headquarters that are AI-enabled have a better understanding of the operating environment. They can see and respond to threats more quickly and more coherently. Large Language Models The armed forces have already employed AI to conduct specific planning functions, such as route analysis. Up to now, AI has
refined, improved, and assisted traditional planning techniques. The question is whether large language models or generative AI, like ChatGPT and Copilot, might supersede military planning, staff-work, and headquarters themselves. As discussed in chapter 2, generative AI is able to respond to a prompt to scour the whole of the internet to answer a diversity of questions creatively. Although generative AI is still founded on a probabilistic and inductive method, it has increased the functionality of AI considerably. The defence sector has recognised the potential of large language models for operational planning. For instance, military planners have to draw up information from existing datasets —including emails, reports, and orders—to write intelligence summaries and to prepare presentations. On many occasions, staff officers forget, or cannot find, crucial pieces of information; they have to repeat already completed work. Generative AI may eliminate these inefficiencies. Might a large language model, like ChatGPT, automate planning to such an extent that staff officers—even commanders—become redundant? Eric Schmidt’s Special Competitive Studies Project (SCSP) has claimed that generative AI has huge potential: ‘GenAI has the potential to revolutionize military capabilities, intelligence gathering and analysis, and cyber warfare, all of which means the United States and its allies and partners must maintain a competitive edge’.31 SCSP identified ten functional areas where generative AI might be applied: (1) simulations and planning, (2) optimization and productivity, (3) science and technology, (4) data analytics, (5) a unique user interface, (6) new automation, (7) code creation, (8) synthetic data production, (9) grant writing, and (10) content creation. SCSP identified planning as one of the prime functions to which generative AI might be applied. In order to test the utility of large language models, the US Marine Corps School of Advanced Warfighting recently conducted a series of planning experiments with a large language model. The school was assisted by a volunteer team from Scale AI, a commercial AI company which works with the Department of Defense. The planning exercises involved a scenario against China below the level
of war. Extrapolating from the exercise scenario to an actual warfighting operation might require care. However, the experiments were illuminating. Scale AI worked closely with students and faculty at the School of Advanced Warfighting to develop Hermes, a large language model which reflected some of the special challenges of military operations. In the course of the exercises, Hermes proved very helpful. It answered many questions, saving time: ‘Students often sought to use Hermes to understand the economic dimensions of statecraft shaping lines of communication and theater strategy’.32 It helped the students to understand the adversary. They were able to ask various questions about China’s capacities and intentions: ‘Student teams used the model to move between macro understandings of regional economic linkages to country-specific looks at political timelines (e.g., elections) and major infrastructure investments like China’s Belt and Road Initiative’. It was particularly useful in helping to generate and refine courses of action; ‘Students used Hermes to help generate hypotheses about temporal and positional advantage in competition’.33 Finally, and perhaps most significant, Hermes enhanced the students’ operation art: ‘The dialogic format of asking and refining questions with the assistance of a large-language model helped military planners gain a better appreciation of the operational environment and identify how best to understand concepts in terms of time, space, and forces’.34 In particular, ‘the large-language model helped military planners see battlefield geometry in multiple dimensions’.35 Hermes and other large language models are plainly important. They have the potential to improve the efficiency and effectiveness of headquarters and their staff. However, they have obvious constraints. For one, they are entirely dependent upon their data. Any gaps or faults in the data will undermine their validity. In addition, staff officers have to learn how to get the best out of a large language model. In the School of Advanced Warfighting exercise, there was an art to asking Hermes mundane questions which the program could operationalise. Without a trained user who understood how Hermes’s algorithms operated, there was a constant
risk of confirmation bias: ‘Modern warriors have to learn how to translate their doctrine, concept of warfighting, modern capabilities, and historical reference points—their craft—into questions based on core assumptions and hypotheses they can falsify and augment in an ongoing dialogue with large-language models’.36 Staff need to invest in critical thinking and in basic research methods in order to exploit large language models like Hermes safely. Even then, large language models are prone to hallucinations. Because they ‘understand’ nothing and are only retrieving and processing raw data, it is common for them to generate quite spurious correlations. They frequently produce findings which are nonsensical; thus, ‘the military ought to ensure planners understand the limitations of algorithmic methods’. To reduce the risk of hallucination, users need to become competent at dialogical questioning which refines the search and reduces the chances of stochastic nonsense. Although, as mentioned, Hermes proved to be extremely useful in the exercise, the conveners of the experiment at the School of Advanced Warfighting concluded that ‘the model augments—but does not replace—the warrior’.37 In their work on generative AI, SCSP drew the same conclusion: ‘We should not exaggerate the capabilities of Generative AI’.38 One of their lead researchers has highlighted the problem. Generative AI is extremely effective for simple tasks. For instance, it can easily produce a recipe for a meal. In response to a simple question, it trawls the internet for an answer which, on purely statistical grounds, has a good chance of being correct. However, military operations are more complex. They require a level of understanding which generative AI lacks. As one interviewee commented, ‘Specific military tasks [require] a much greater degree of specificity than creating a great chicken parmesan’.39 Military plans involve a diversity of factors—military, political, diplomatic, economic, climatic, geographical, social, and cultural. Plans are complex, delicate constructs. Generative AI is not well suited to that sensitive, interpretive endeavour because it does not comprehend anything in the real world. As Owen Daniels has noted, making AI work requires focus, resources, and effort. Like all previous military innovations, AI
will require experimentation, bureaucratic coherence, and cultural adaptation.40 While generative AI is not about to eliminate the need for human staff officers and commanders, large language models do have real potential for planning. They could usefully summarise intelligence reports (in foreign languages) or search through diverse datasets so that no vital pieces of information are missed. A large language model might have saved Kenneth Bigley. It seems entirely plausible that a large language model could generate and test provisional courses of action. It could make headquarters quicker, more effective, and perhaps more efficient. Digital Cartography If this survey of how AI has recently been used by the armed forces is accurate, AI will not replace human commanders or their staffs. AI is not about to automate military planning as a whole. However, because planning involves calculating and processing a large amount of data, AI has already helped headquarters to plan. As the armed forces begin to employ large language models, it is likely that AI will become an even more useful and integral part of the planning process. There is an irony here, though. As they become more efficient, headquarters are likely to be able to plan more ambitious, complex operations which orchestrate diverse forces across larger areas ever more harmoniously. After all, the armed forces are in an adversarial environment in which the enemy will always react. There can, therefore, be no limit to the pursuit of military effectiveness. Commanders may be able to and may have to prosecute operations which would have been impossible in the past. Precisely because AI may facilitate more complex operations, it will generate new and increasingly difficult questions for commanders and their headquarters. Even as AI automates simple planning functions formerly performed by staff officers, it may multiply coordination challenges and raise difficult questions. It is most probable that only human commanders and staff will be able to resolve the proliferating
and intricate organisational problems which AI-enabled operations will themselves generate. AI, no matter how creative, will be a tool for the headquarters, performing mundane labour for staff or augmenting staff expertise. Human judgement will remain crucial. Human commanders and staff are likely to become more essential, not less. If an AI commander is not about to materialize, how are we to understand the likely role of AI in military headquarters today? It is perhaps useful to contemplate a historical analogue to illustrate the likely role of AI in planning military operations. From the sixteenth century, European armies began to expand. The Spanish Army grew from twenty thousand regular troops in 1470 to over two hundred thousand in 1630. In order to move such large forces, commanders required more and more accurate information about roads and terrain. At the same time, because the weight of the artillery on which their forces increasingly relied made it difficult to transport, commanders needed a precise understanding of the topography. Armies needed good routes that avoided steep terrain. Thus, from the sixteenth century, maps became indispensable. Military cartography began with simple reconnaissance sketches. Troops, often disguised as civilians, rode out in front of armies to create rough sketch maps, which they brought back to their commanders. However, by the seventeenth century, map-making had become an established expertise, funded by European monarchs. The British Ordnance Survey was founded by the Crown and was, in the first instance, conducted by military engineers for purely military purposes. By the eighteenth century, military cartography had become a highly developed and specialist industry.41 Grand military operations could not be conducted without maps. In 1805, Napoleon conducted his famous Ulm campaign. He marched seven separate army corps, consisting of 235,000 troops, about a hundred miles from the French border to converge on the Imperial Army in southern Germany. He could not have executed the manoeuvre without advanced mapping. AI may be seen as a new form of military cartography. AI processes and tabulates an enormous volume of data so that
commanders can identify their enemies’ locations more precisely and plan against them more efficiently and quickly. AI may facilitate ever more complex coordination over time and space, predicting where the enemy and one’s own forces will be and when. AI may be the basis for a revolution in military cartography. Just like their early modern counterparts, the militaries of the twenty-first century may find that without AI to map the data landscape for them, they will be lost. As a dynamic digital map, AI cannot be said to automate war or even decision-making. However, in a quiet way, AI may be as significant to military capability as mapping was in the early modern period.42 In summary, AI will not operate as HAL did in the famous Stanley Kubrick film 2001: A Space Odyssey. AI will not take over the headquarters. An AI agent is not about to supersede commanders or their staff, independently planning operations and making all the decisions itself. Yet AI may enable commanders to coordinate operations over longer periods of time and over larger areas with ever-greater precision. Such a role for this technology may be less exciting to imagine, but it is no less significant.
7 AI and Targeting For decades now, Amazon and other companies have used data processed by algorithms to target customers. They have mined their data to map the market and to profile individual consumers, recommending goods and services to them on the basis of their digital signatures. The national AI strategies discussed in chapter 3 identify targeting as a major area for AI exploitation. Just like commercial companies, the armed forces might use AI to mine massive amounts of data to identify the signatures of their adversaries. In the preceding chapter, we explored how AI has enhanced military planning. Once operations are planned, AI might help the armed forces to execute them by identifying targets more quickly, accurately, and extensively than was previously possible. In fact, AIenabled targeting has already become more common among the world’s advanced militaries; the US, China, Israel, Iran, the UK, and France have all sought to exploit data to increase their effectiveness —and their lethality. Data Mining In a recent monograph on the use of big data and counterinsurgency, Eli Berman has rightly noted that ‘information and how it is leveraged, we will argue, […] plays a key role in the government’s efforts to defeat or contain insurgencies’.1 Big data is increasingly important here because ‘big data allows us to measure things we never could before’.2 In short, big data has allowed
military forces to target the enemy more accurately. Consequently, there is a growing number of examples when the armed forces have employed AI-enabled targeting. Some important cases from the Russo-Ukraine War will be discussed later in this book. However, in order to highlight how data can help the armed forces to target their opponents, this chapter focuses on three examples: the Pentagon’s Project Maven, launched in 2017; the British Army’s Covid testing operations in Liverpool in 2020; and the IDF’s eleven-day campaign against Hamas in May 2021, called Operation Guardian of the Walls, and the current war in Gaza, known as Operation Swords of Iron. Project Maven has, of course, attracted significant scholarly attention—and concern. Owing to the 2018 Google protest (see chapter 4), it has been the object of significant public criticism and has been treated with deep suspicion in much of the security-studies literature. Scholars may be justified in their concerns about its potential applications. Nevertheless, it remains one of the best, and most successful, examples of how the armed forces have employed AI to help them target opponents. It may be a paradigmatic example of the nascent application of AI to military operations. The British Army’s response to Covid is less well known. Indeed, even in the UK, it is a little-recognised case, but the operation exemplifies how data and AI might be used to target opponents— even though, in this case, the adversary was a virus. The Israel Defense Forces (IDF) have become leaders in the military application of AI to operations. It is impossible to ignore their innovations. In the previous chapter, I examined the case of Unit 8200 as a specialist military organisation which has been involved in cyber operations. The IDF, with Unit 8200 playing a major role, have also increasingly employed AI for targeting. Operation Guardian of the Walls and Operation Swords of Iron—in which two new programs, the Gospel and Lavender, were employed —exemplify the application of AI to targeting with especial force.
Project Maven In the late 2000s, the Pentagon developed various AI initiatives. For instance, as discussed in the previous chapter, the US Army tried to develop a digital battle-management system, the Distributed Common Ground System (DCGS). Committed to the Third Offset Strategy, Assistant Secretary of Defense Robert Work was convinced that the US military had to exploit the potential of AI. He believed the technology was now mature enough to create a viable system. Consequently, on 26 April 2017, Work established the ‘Algorithmic Warfare Cross-Functional Team’, Project Maven.3 The aim of Project Maven was to use AI to help collate, curate, and fuse defence data so that the US could pre-empt its enemies and adversaries. Project Maven sought to apply AI to the problem of military targeting. In the first instance, Project Maven was narrowly conceived. It attempted to process the massive archive of full-motion video which US forces were collecting by means of uncrewed aerial systems (i.e. drones). The project aimed ‘to field AI capabilities to augment, accelerate and automate exploitation and the analysis of full motion videos (FMV) from unmanned aerial systems (UAS)’.4 In Afghanistan, the US Air Force had developed a system of extensive drone surveillance called Gorgon Stare. It involved a comprehensive surveillance of entire towns and cities. Using Gorgon Stare, an intelligence analyst was able to track back from car bombs to their points of origin. As one officer explained, ‘Once you have that location of origin […] then you could start going back in time and mapping out where people came from’.5 However, it took human analysts an inordinate amount of time to analyse all the footage. General Jack Shanahan, appointed to direct the project, recognised the problem: ‘We were experiencing a catastrophic success story. While there was more intelligence collection from more sources, at every classification level, than at any previous point in history, it was impossible for analysts to process and analyze so much information. They were drowning in data. We needed to find an entirely new way, even revolutionary way, of improving effectiveness and efficiency across every phase of the intelligence cycle’.6 Consequently, the air
force started to experiment with computer-vision algorithms to sift through full-motion videos.7 Project Maven sought to scale up and accelerate the process. As Shanahan has noted, Project Maven ‘did not start with an AI solution in a search of a problem. Instead, we started with a problem which could not be solved except by AI’.8 Project Maven, therefore, started with a single task—processing massive quantities of data from full-motion video: ‘The aim was to create an AI system that would allow analysts to select a target and then see every existing clip of drone footage that featured the same person or vehicle. Ultimately, the Defense Department wanted an automated search engine of drone videos to detect and track enemy combatants’.9 However, the project became more ambitious. Under Project Maven, the Department of Defense eventually aimed to identify anomalies in areas of interest and to identify signatures by collating information from open sources, human intelligence, satellite imagery, and electromagnetic sources. There was initially a great deal of scepticism about whether Project Maven was even viable. Will Roper, the head of the Defense Department’s Strategic Capability Office, described the opposition: ‘The Pentagon had been burned by decades of automatic target recognition programs that did not work’.10 Shanahan confirmed the scepticism: ‘People did not think Maven could work. But in the end, it provided a lot of intelligence which then people like [Christopher] Donahue in Afghanistan could tailor to their particular needs’.11 However, Roper understood the remarkable recent advances in machine learning.12 He believed Maven could work. As discussed in chapter 4, Project Maven was established in April 2017. Shanahan was the director, and Marine Colonel Drew Cukor ran the day-to-day operations. In 2017, Shanahan, Cukor, and Colonel Jason Brown, an air force intelligence officer supporting both Project Maven and later the JAIC, attended the 2017 Computer Vision and Pattern Recognition Conference in Honolulu to create connections with the leading AI experts. There, they met representatives from Google, to whom they described their requirement for one of their most challenging problems, wide-area motion imagery. Following the conference, several contracts for
Project Maven were awarded to start-up AI companies, followed later by the contract award to Google, specifically for Maven’s Wide Area Motion Imagery project. Other companies involved in Maven’s work with Google included DigitalGlobe, a geospatial-imaging company, and CrowdFlower (now called Figure Eight), a datalabelling company. DigitalGlobe and CrowdFlower could provide global surveillance data from satellites and individual hand-held devices which Google’s algorithms could fuse and analyse to give the Pentagon a high-fidelity intelligence picture from anywhere in the world on request. The Defense Innovation Unit (DIU) was already working with two open-source computer-vision projects, one of which was ‘building the largest dataset for object detection in overhead [satellite] imagery’; the other had set ‘the world speed record for training deep neural nets on public cloud infrastructure’.13 As Brendan McCord, a Pentagon defence official responsible for Maven, noted, the DIU identified tech companies which might partner with Maven: ‘It was pinpointing who those folks are, big and small, and then helping [Maven] go out and figure out […] how to assess these companies. It was a combination of my experience in venture capital, my experience in the startup, and my time at DIU, with seeing a lot of different companies and knowing what to ask for’.14 Maven emerged out of the collaboration of Shanahan, his subordinates Cukor and Brown, and the DIU. From its inception in 2017, Maven developed rapidly. At that time, the campaign against ISIS was reaching its peak. In 2016, the Global Coalition to Defeat ISIS had begun to retake some ISIS territory, including Fallujah. The decisive Battle of Mosul started in October 2016. The US actively wanted to employ Project Maven in this campaign. In July 2017, Cukor announced that Project Maven intended to deploy an operational system by the end of the year. The aim was to develop an AI-enabled image classifier that could detect thirty-eight classes of objects, including people and vehicles. The tool was not complicated, but it was able to identify and track ISIS targets from ScanEagle drones in Syria and Iraq. Within eight months, the project had delivered an operational output. It demonstrated the great potential of AI—but also its limitations. The
algorithms were reliable; they eventually recognised signatures automatically. However, the program was not magical. Its capability was quite a long way from the all-seeing, infallible supercomputer envisaged in contemporary discussions of AI. In order for it to assist in targeting, many conditions had to be met. First, Project Maven was successful because it addressed a ‘narrowly defined problem’. The challenge was to identify a small number of signatures in a specific theatre. It was possible to program software to do this, since the US had enough data on which to train the algorithms. Data was absolutely crucial here. As Shanahan emphasised: ‘AI is the tip of the iceberg […] Project Maven, high-performant AI is critically dependent on the underlying information architecture, the AI stack, and a modern data management pipeline [in order to function]’.15 Second, once this architecture was in place, the project also depended on continued expertise, effort, and refinements for it to work properly. For the programs to be operational, they required constant monitoring and adjustment. As Shanahan explained, ‘Unless the model is trained on the same types of data they will ingest under operational conditions, they will not perform as well as expected’.16 Shanahan has emphasised the importance of this maintenance and tuning: ‘Once you deploy it to a real location, it [the AI] is flying against a different environment than it was trained on. Still works, of course […] but it’s just different enough in this location—say there [is] more scrub brush, or there’s fewer buildings, or there are animals running around that we hadn’t seen in certain videos. That’s why it’s so important in the first five days of real-world deployment to optimise or refine the algorithm’.17 When the program was first deployed against ISIS, the Project Maven team updated it six times in eight days. Indeed, Shanahan believed that this labour was both an important discovery and Maven’s first major achievement: ‘Maybe one of our most impressive achievements is the idea of refinement to the algorithm’.18 Even then, AI is fragile. As Shanahan himself has observed, it is susceptible to corruption and to adversarial attack. Project Maven is probably the most famous application of AI to military operations. Against resounding scepticism from the
Pentagon and Congress, Project Maven was able to develop an AI program which could identify a suite of signatures in a sea of data. It had allowed the US armed forces to analyse the vast amounts of video footage which it has collected. In short, Maven has improved US intelligence analysis and, thereby, enabled the US military to target its opponents more accurately. Google employees were appalled by this capability; many other observers are critical of Project Maven too. The ethical questions are certainly pertinent, especially if Maven were to be applied to non-combatants. Yet Maven usefully exemplifies what AI can do for the armed forces— and what it cannot. Maven automates the processing of huge datasets, but it is far from omniscient. It is limited to its data and is brittle. It requires constant intervention by experts to sustain and refine its programs. Covid Testing in Liverpool While Project Maven is the most famous example of using AI to automate some parts of the targeting process, the British Army’s contributions to mass testing for Covid in Liverpool in 2020 is an interesting example of a less well-known case. On 23 March 2020, the UK went into its first national Covid lockdown. The lockdown was lifted in May, though some restrictions stayed in place throughout the summer. By autumn, Covid cases began to rise again. Liverpool, a city in the north-west of England with a regional population of 1.6 million, had been hit hard by both Covid and lockdowns. High levels of poverty increased the population’s susceptibility to severe disease and death. In addition, half of Liverpool’s economy depended on visitors, who because of Covid restrictions were no longer able to go to the shops, bars, restaurants, hotels, and theatres. In the autumn of 2020, the city was nearing a financial and health crisis as a result of Covid. On 30 October 2020, Liverpool’s Covid Gold Command—composed of leaders from local government, the NHS, public health authorities, police and other emergency services, and academia—agreed to pilot mass testing for people with or without symptoms of Covid who lived
or worked in Liverpool. However, it was immediately apparent that none of the organisations represented in the Covid Gold Command had the personnel or resources to conduct mass testing on their own. Consequently, the British Army was ordered to support the process, under the Military Aid to the Civil Authorities Act. The army deployed 8 Engineer Brigade, under Brigadier Joe Fossey, to the city in the same month. On 6 November 2020, the mass-testing pilot began, with the assistance of 8 Engineer Brigade. Over the next two months, a third of the city’s population was tested.19 Liverpool, plainly, was not a military operation. 8 Engineer Brigade deployed to provide assistance to a civil power. Nevertheless, it is an interesting case in which the army exploited data. Although AI was not used extensively in this case, it is pertinent to understanding how AI might help the armed forces to target opponents. Liverpool was manifestly not an urban insurgency; the army was not fighting guerrillas in Merseyside. Yet the British Army faced a threat to the civilian population in an urban environment. It was asked to help identify concentrations of the Covid virus in order to suppress those viral outbreaks. The challenge to the armed forces was not entirely different to the kinds of insurgencies which militaries have often faced and which, indeed, British forces did face in Iraq from 2003 to 2008 and in Helmand between 2006 and 2013. Liverpool might be taken as an analogue for a real military operation. Instructively, civil and military participants in Liverpool saw the operation in military terms. For instance, Professor Iain Buchan, chair of public health and civic informatics at Liverpool University, was an important civilian leader throughout Covid, working closely with 8 Engineer Brigade. He described the situation in Liverpool in October 2020 as a ‘battlefield situation’.20 Intensive-care units in hospitals were full, people were dying, and families already living in poverty were suffering job losses as a result of lockdowns; the city and the NHS were under enormous stress. At various points in the crisis in Liverpool, Buchan himself received threats from those opposed to testing and to the possible introduction of ‘Covid passports’. ‘It was emotionally hard work and deeply motivating’, he
later recalled, adding, ‘We got little sleep’.21 Indeed, Buchan drew an explicit parallel between this health crisis and a military operation. In both cases, there was a lack of sufficient information. Yet leaders had to make decisions quickly; they had to act and then react to the results of their actions.22 Less surprisingly, members of 8 Engineer Brigade—Brigadier Fossey, for instance—also defined the operation in military terms. Although he fully recognised his role was to support the civil authorities, Fossey, a veteran of Iraq and Afghanistan, found it useful to conceive of the Covid virus as an enemy to be identified, targeted, and destroyed. At the start of the operation, he asked himself: I recognised we knew very little about the virus. My first question was, where is the bug? As we started to look for Covid-19, I was thinking in physical terms. I needed to find ‘it’. I wanted to find the ‘Taliban’. Or I wanted to find ‘Taliban-associated people’. This quickly evolved into asking where I could find Covid—the infected people. As we started to gather the data, I was able to start asking clearer questions. And as we started to answer those questions, it made our response much more targeted and I was able to use the resources at my disposal much more effectively.23 Consequently, the Liverpool case may be at least indicative of more military applications of data. The members of 8 Engineer Brigade deployed into a mature institutional environment. They were subordinated to the civil powers: the Liverpool City Council, the Liverpool City Region Combined Authority, the NHS, and the Merseyside Police. Nevertheless, bringing a powerful headquarters of more than fifty officers, over two thousand troops, and many vehicles and resources, they were able to exercise considerable agency in the support of the Covid testing. Consequently, they partnered closely with health authorities in the city to draw on their information. They were assisted in that regard by improvements in health-data management over the preceding years. In 2019, Liverpool City Region Combined Authority had supported a plan to develop the first
UK Civic Data Cooperative. The aim was to collate and fuse health data from various sources to improve the delivery of health care and social care services, public health services, and health research for the population. The initiative was accelerated and, in 2020, the University of Liverpool and a leading public health physician and data scientist designed a data linkage and AI-automated intelligence system called Combined Intelligence for Population Health Action (CIPHA). The CIPHA Task Force was led by the University of Liverpool and Mersey Care NHS Foundation Trust and worked with local doctors and civic leaders on data-sharing with the Civic Data Cooperative. Within ninety days during the summer of 2020, CIPHA deployed a cloud-based care record for 2.7 million residents with a data-analytic engine. The system involved a series of dashboards providing nearreal-time Covid and wider health intelligence. CIPHA was fully operational by October 2020.24 Initially the NHS did not want to share its data with the army. However, because of the close relations which had developed between 8 Brigade and the civil authorities in Liverpool and, in particular, between Professor Iain Buchan and Brigadier Fossey, the testing team was able to break down these data silos. CIPHA proved to be vital for the leaders of 8 Engineer Brigade as they developed their plan for Liverpool. They were able to share and pool intelligence. The collaboration was crucial to the operation. Once the members of 8 Brigade had access to the CIPHA dashboards, they could themselves develop a better appreciation of the problem they faced. In order to exploit the health data at hand, Brigadier Fossey identified the three key questions which his brigade had to answer: Where is Covid? Where are we? Where do we need to be?25 The questions seemed simple, even mundane. Yet they were crucial. To test most effectively, it was imperative to work out precisely
where the spikes in infection might be concentrated and, therefore, to locate testing centres there. Intelligence was clearly critical here. There were two thousand soldiers in 8 Brigade. However, that was still a small number of personnel for the problem which they faced; Liverpool is a large city, and soldiers cannot work twenty-four hours a day. The brigade had to try and be accurate in its deployments and be efficient in its use of personnel. It could not erect testing sites everywhere. The brigade had to consider where to locate testing sites carefully in order to maximise footfall. In this way, its situation was similar to that of Project Maven. The aim was to maximise the efficiency of the use of military forces through the collation and fusion of health data, targeting Covid concentrations more accurately. At first, the brigade had little choice in the placement of testing sites; it had to target the infected population. The initial sites were easy and obvious places to which access was already agreed by the council. Consequently, one officer noted: ‘It was a bit of a knee-jerk reaction. We put the sites in random spots, with access’.26 The brigade soon realised that this approach was inadequate. The question was ‘how to carve up the city and allocate resources’; ‘It required proper military planning’.27 In developing a more refined plan for testing, the brigade’s GEO [Geographic] cell was critical. The GEO cell consisted of two relatively low-ranking—but key—specialists in mapping, surveying, and terrain analysis: Captain Tom de Silva and Staff Sergeant Arran Burton. It was tasked with generating a better intelligence picture of the city, in collaboration with CIPHA. Initially, 8 Brigade’s GEO cell used the city’s ward boundaries to organise the operation. It then divided these wards into sectors according to population density: so-called modifiable areal unit problems. These areas were colour-coded, with purple assigned to the most densely populated areas. Using this analysis of the urban demography, de Silva and Burton were able to determine mathematically the optimal locations for testing sites. The brigade had a good idea of where testing sites were necessary simply on the basis of the population density: ‘We needed so many sites to test so many people. This is where the data started to help. We used data
to work out where the people were’.28 With more data, the brigade might have identified even better sites.29 The brigade eventually procured better data. Above all, the brigade was looking for additional, proxy data which might help it pinpoint concentrations of Covid. To that end, it reached a datasharing agreement with the chemists Superdrug and Boots for records of sales of pharmaceutical products which were associated with patients ‘not fully presenting with Covid but with presymptomatic conditions’; paracetamol, energy drinks, and vitamin C were the most prominent here.30 However, this technique was limited. The army did not have access to all the pharmaceutical stores, and the data was not entirely reliable, since consumers might buy these products for reasons other than having Covid. Shopping data was, therefore, not optimal in targeting outbreaks of Covid. Consequently, 8 Engineer Brigade searched elsewhere. The UK Health Security Agency suggested that the brigade examine wastewater, since it is possible to plot infections in wastewater. Felicitously, the system for collecting and employing this data was already in place. In particular, the process had already been trialled in Birmingham: ‘The nation uses wastewater to monitor polio. We just bootstrapped onto that for Covid. For instance, we first experimented by putting a sampling station on an outlet in a Jaguar manufacturing plant in Solihull. That kept the manufacturing open because you could say how many people were asymptomatic. It was a fantastic use of testing. Previous to that, we could only tell who was pre-symptomatic by their shopping.’31 Therefore, 8 Engineer Brigade used wastewater data. The staff took data provided by the UK Joint Biosecurity Centre, which recorded the wastewater viral load. The idea was to detect Covid in human waste flushed down the toilet in specific neighbourhoods: ‘We used wastewater outlets. You can analyse water samples and it gives you hits in parts per million. With that data, we could get to the hotter spots. There were loads of asymptomatic people. The wastewater told you where Covid-19 people were or might be. It allowed us to target better’.32 This data was not comprehensive, since large parts of the city did not record wastewater: ‘The data did
not line up. It did not cover all the wards. For instance, Everton, which was in Merseyside Water, was not covered.’33 However, the data proved very useful: ‘In the wards for which there was evidence, it was possible to map the spread of infections more accurately. Where is the virus concentrated? I can see it. I know where the infections are. That is useful. It confirmed assumptions’.34 In particular, this method of looking at wastewater answered the brigadier’s question ‘Where is the Covid?’ It revealed where infected people were actually going to the toilet, whereas testing recorded only an individual’s postcode. As a result of combining the data from wastewater with the evidence from NHS records of Covid testing/infections in CIPHA,35 8 Brigade began to identify concentrations of infection in Liverpool. The brigade found a close connection between infection rates and poverty. Infection tended to be concentrated in the poorest areas. By correlating health data with social data, the brigade developed a good understanding of population densities in Liverpool. This data indicated a more profound challenge the brigade faced in testing the population. In urban areas, very high population density has always been associated with poverty. It was the same in Liverpool in 2020. The most densely populated neighbourhoods were also the poorest. Indeed, the brigade developed a ‘neglect map’ on the basis of its population data.36 The brigade noted that, as a result of lower population density and easier access to testing sites, the testing operation had initially ‘favoured the worried well and the affluent’.37 The neglect map, combined with the viral load data, showed that the real problem was in the poorest and most densely populated parts of Liverpool, where the general health of the population was already lower. The leaders of 8 Brigade decided to prioritise testing in these areas. However, the challenges of putting testing sites in these neighbourhoods and encouraging citizens to use them were considerable. Residents in these areas did not generally own vehicles. In addition, they tended to mistrust the authorities. They avoided getting tested for Covid, for fear of finding they were positive, which would mean they would have to self-isolate and
could not go to work—for low-income families and individuals, the attendant drop in income could be disastrous. Consequently, 8 Brigade had to locate its testing sites in places which maximised the chances that locals would actually use them. They chose gyms and community centres, reasoning that ‘people look for places they regularly use within their world’.38 The data also revealed that stationary testing sites were of limited effect—after ten days, everyone in that area who was going to come for a test had already done so. Consequently, the brigade used a few stationary hubs, orbited by mobile ones. The mobile test centres were crucial in the areas which were not testing. The brigade calculated that, to be effective, a mobile testing site had to be within 800 meters of a person’s home. The GEO cell modelled the locations of sites and found that it was misleading to locate sites by lineal Euclidian distance from housing. In a city, people usually cannot walk a straight line to their destination; the actual route they must take might be much longer than the straightline distance. Consequently, the cell inserted a Manhattan system algorithm into the mapping data to work out the average walk for inhabitants of a particular neighbourhood.39 The cell then looked at the data on footfall in the sites which the brigade had established. It could determine which sites were most effective, as well as observe the traffic flow during the day. It could see surges at certain times, such as in the morning, or oscillations between the morning and the evening. The brigade also tried to identify places with which locals were familiar and at which they felt at ease. Here, the brigade acknowledged the work of Mark Green, a health geographer embedded in the Office for National Statistics: ‘Mark worked with fellow geodata scientists in 8 Brigade GEO cell and produced maps of optimal placement of testing sites taking these factors into account. And all those relations drove decision-making’.40 Once the brigade had located optimal sites for the testing centres, it engaged in a process common in Afghanistan: ‘it conducted key leader engagement. It held “shuras” to persuade leaders to get [their] population to a testing site’.41 Iain Buchan
played a crucial role in these information campaigns. The brigade never used fear to encourage testing; fear disincentivised the population. Rather, every Tuesday and Thursday, the brigade drove through key neighbourhoods to tell inhabitants where mobile testing sites were located. Faith leaders and gyms provided further conduits for information. The deployment of 8 Brigade to Liverpool in support of civil authorities was a notable success, in spite of the initial scepticism and occasional resistance of the local populace. By mid-December 2020, detection had increased by 20 per cent, case rates were down by 20 per cent, and the number of Covid-19 hospitalisations in the city had been reduced by 43 per cent.42 In addition, hospitality was re-opened in December for Christmas, while other cities remained closed—this was very important for a city which was so dependent upon its hospitality sector and visitor economy.43 The ninety-day operation that 8 Engineer Brigade began in Liverpool in late 2020 was a minor military operation in a critical civil situation—not a military one. However, it shows how the armed forces might employ AI to process data to target adversaries in future operations.44 The Gospel and Lavender In 1967, after the Six-Day War, the IDF occupied the West Bank and Gaza. Since the Lebanon War of 1982, the IDF’s main mission has been not interstate warfighting but, increasingly, the control of the Palestinian population and the suppression of Palestinian terrorist cells in Gaza and the West Bank. For decades Israel has been monitoring the West Bank and Gaza, gathering intelligence and information on terrorist organisations, political groups, and civilians. As one Israeli informant noted: ‘This is the thing about the IDF. They don’t need information on the entire world. It is very specific to the West Bank and Gaza. The datasets for these areas, the IDF is all very advanced in. It is specialised in its expertise, but it is geographically specialised. The UK or US are worldwide. We focus on fifteen cities’.45
Many deplore the political situation in Israel; they describe the West Bank as a de facto apartheid state. However, whatever the political situation, the IDF provide a perspicuous example of how AI has been employed for targeting in urbanised operations. In the last fifteen years, the IDF have sought to exploit data as a way of tracing and targeting Palestinian militants and terrorists. No matter how many countermeasures they employ, Palestinian operatives leave a digital signature in cyberspace. In addition to traditional methods of intelligence collection, the West Bank and the Gaza Strip are now saturated with Israeli sensors, including satellites, radar, drones, and cameras. In addition, open sources have become a means of collecting rich intelligence, but the amounts of data they may involve are vast. IDF officers emphasise the problem: the data is ‘endless, reaching into petabytes (one million gigabytes) in some areas’.46 ‘The data revolution has led to massive amounts of operational data being collected from cameras, microphones, networks, information systems, and devices. The human factor is no longer sufficient, as soldiers cannot physically keep up with the amount of incoming data’.47 The IDF have a data problem. Consequently, in the last decade, the IDF have created a number of specialist units to process data; AI has become crucial. I have already mentioned Unit 8200, but Unit 9900, J6/C4I Directorate’s Lotem Unit, and the Sigma Branch IDF have also been very important in the application of AI. As the commander of the Sigma Branch has noted, ‘the goal is to improve the effectiveness of the IDF’. The IDF have trained AI applications to sift through massive amounts of data in order to recognise the important information. The IDF’s AI programs can analyse hundreds of videos at a time and automatically flag suspicious activity. For instance, after Operation Protective Edge in 2014, an operation against Hamas in Gaza following rocket strikes, the Lotem Unit developed an app that learned from field sensors and other data: ‘We collected what are the most likely areas launchers will be set up and at what hours. That enables us to know in advance what will happen and what areas should be attacked in order to fight [Hamas] more effectively’.48 By 2017, the IDF had developed sophisticated AI
programs whose algorithms could automatically identify objects of interest. The aim was to go further and to create predictive AI algorithms. AI programs of this type would not just recognise targets which were already operating but also make predictions about the movements of Palestinian operatives and recommend courses of action against them. In 2021, in response to Hamas rocket strikes, the IDF mounted another major operation in Gaza, Operation Guardian of the Walls. The Israeli armed forces described this action as ‘the first AI war’. Building on the work of its specialist units from 2014, the IDF incorporated AI into the targeting process. AI was useful because the Israelis had employed a network of electronic sensors on drones, F-35s, seismological monitors, and other systems over the course of several years. The IDF had collected billions of pieces of signal, satellite, and open-source data on Hamas and the Palestinian Islamic Jihad. The IDF fused these diverse datasets: ‘By employing AI algorithms and machine learning, paired with intelligence analysts in “man-machine teams” to flag and review potential targets, the IDF synthesized extensive amounts of data into pre-conflict target folders that were significantly more detailed, accurate, and timely than in 2014’.49 AI facilitated dynamic targeting, which was also more accurate than targeting had been in the past. With the help of AI to process the data, the IDF developed a system of ‘intelligence-driven combat’ which disseminated intelligence to combat units digitally in real time and matched the targets with precision-guided munitions. In this way, ‘the IDF could conduct highly accurate airstrikes, substantially mitigating risks to civilians’.50 During Operation Guardian of the Walls, the IDF employed a new AI system, called the Gospel (Habsora), for the first time. The Gospel processes Israel’s massive databases on Hamas militants and Gaza citizens to identify military targets. The Gospel was developed by the IDF’s cyber specialists, Unit 8200. It is a machine-learning program, trained to analyse massive amounts of data from numerous and diverse sources to generate accurate and dynamic targeting. The Gospel has accelerated the IDF’s targeting process. Before its institution, twenty Israeli staff officers typically produced
50–100 targets a year. Gospel produces 100 targets a day. Aviv Kochavi, former chief of staff of the IDF, has himself emphasised these enhanced capacities: ‘This is a machine that, with the help of AI, processes a lot of data better and faster than any human, and translates it into targets for attack. The result was that in Operation Guardian of the Walls [in 2021], from the moment this machine was activated, it generated 100 new targets every day. You see, in the past there were times in Gaza when we would create 50 targets per year. And here the machine produced 100 targets in one day’.51 If Kochavi’s figures are accurate, the Gospel has increased the IDF’s targeting capacity by over 30,000 per cent. In the course of Operation Guardian of the Walls, the Israelis targeted Muhammed Bawab, the leader of Hamas’s East Rafah Brigade, who was responsible for abducting two IDF soldiers in 2014. The IDF wanted to strike his house, which was acting as a command post. However, a variety of feeds—some of them processed by AI—showed that there were civilians sheltering under a palm tree just outside the house. Shin Bet, the Israeli secret service, rang Bawab’s neighbours, warning them in Arabic: ‘You are under the palm tree, near the house. Go away, there’s a one-ton bomb coming and you are going to get hurt’.52 AI helped the IDF to prosecute a very precise lethal campaign. Israel’s strikes were certainly more accurate than in the past; 99 confirmed enemy were killed, while 40 more deaths were believed to be enemy. Civilian casualties were still high, though; 120 were killed. The IDF claimed a 1:1 civilian-to-belligerent casualty rate; a historical low.53 Whether this ratio is accepted as a justification, AI had allowed the IDF to target more accurately in a complex, dense urban environment. Following the Hamas attacks on 7 October 2023 (see chapter 10), the IDF were engaged in a long and intense war in Gaza, Operation Swords of Iron. The IDF have committed to a prolonged siege of Gaza, into which sixty thousand Israeli troops were deployed, with many more in support. The fighting was brutal. Many buildings have been destroyed, and more than twenty thousand civilians have been killed; perhaps 90 per cent of Gaza’s 2.2 million residents have been displaced.
Clearly, the Israelis’ strategic goal is the complete destruction of Hamas, and the IDF have been less constrained by concerns about collateral damage and civilian casualties than in the past. Their targeting regime has been liberal. Yet, despite the apparent indiscriminateness, the IDF have tried to be precise in their strikes. For instance, on 31 October 2023, they bombed the Jabalia Refugee Camp in northern Gaza. The Israelis plausibly claimed that they eliminated several Hamas leaders who were hiding in a tunnel system. However, the strike also brought down an apartment block and damaged others; it was reported that 120 civilians were killed. The IDF’s strike on two World Central Kitchen vehicles in Gaza in April 2024 caused outrage; seven civilian charity workers and their security detail inside the cars were all killed. Yet the strike was very precise indeed. Rockets penetrated the roofs of the vehicles. However, the IDF had mis-identified the vehicles; the intelligence was inaccurate. Collateral-damage considerations have been loose, but AI has, in fact, played an important role in identifying Hamas targets accurately and quickly. The Gospel has remained important, but it has been augmented by a second AI-enabled targeting system, called Lavender. While Gospel has identified the locations of Hamas facilities and headquarters, Lavender has tracked individual Hamas operatives by collating data from satellite imagery, drone footage, and mobile phone messages. Lavender has been particularly effective in identifying middle- and lower-ranking Hamas commanders; in all, it has identified thirty-seven thousand militants. Indeed, Lavender has overwhelmed Israeli headquarters with so many targets that staff officers cannot check them all properly. In the early months of the war, Israeli officers had only twenty seconds to determine whether a targeted person was a man or a woman. If it were a man, the strike was confirmed.54 In the first thirty-five days of the conflict, in October and November 2023, the IDF struck 15,000 targets—or 428 targets, rather than 100, a day.55 As a result of Lavender, the IDF has quadrupled its targeting capacity, which Gospel had already increased exponentially in 2021.
The IDF have refuted many of the claims circulating about Lavender in the media. In a formal statement published by the newspaper The Guardian on 3 April 2024, the IDF denied that they ‘use an artificial intelligence system that identifies terrorist operatives or predicts whether a person is a terrorist’. The IDF claimed that Lavender was ‘not a system’ but ‘simply a database, whose purpose is to cross-reference intelligence sources’ and that ‘the IDF reviews targets before strikes and chooses the proper munition in accordance with operational and humanitarian considerations’. Lavender is not, then, a completely automated targeting system, but an information system which supports Israeli targeting: ‘Information systems are merely tools for analysts in the target identification process’.56 Nevertheless, as an AI-enabled database or information system, Lavender has expanded the IDF’s targeting process dramatically. Teamwork The armed forces are now inundated with data from multiple sources. It is not possible for humans to process this data on their own. The datasets are too vast. Consequently, it has become necessary for the armed forces to employ software, algorithms, and AI. AI opens up new possibilities for military targeting. With the help of AI, it is now possible for the armed forces to train AI models to recognise signatures in vast data archives to identify, locate, and track targets of interest. It is possible to link together apparently disparate data from satellite imagery, mobile-phone records, and open-source information to develop an accurate, dynamic, and contemporary picture of the location and activities of opponents. AI is not alchemy, though. The functions of AI are specific and limited; AI needs major technical investment and constant expert supervision for it to operate properly. Even then, its capabilities are fragile. Digitised targeting does not eliminate traditional methods of targeting. In no way does AI eliminate the requirement for human operators to decide the theatres or regions on which to focus or to determine which signatures are significant. They program the AI on
the basis of those decisions. AI only processes large datasets that humans cannot. That work helps the final analysis. Yet human commanders and their staffs frame the targeting. Only they can interpret the data, situating it in a wider operational context; only they can calculate whether it is worth prosecuting targets and in what way. AI has facilitated remarkable advancements in targeting. However, rather than automating decision-making—superseding humans, as many scholars worry that machines soon will—AIenabled targeting involves a densely populated process. The crucial enabler in the Liverpool case study, for instance, was not data or AI but partnerships between individuals and teams. In Liverpool, Brigadier Fossey and his headquarters formed a very close relationship with Professor Buchan. There was close cooperation between the health authorities, 8 Brigade’s GEO cell, Mark Green at the Office for National Statistics, and many others. As Fossey noted, ‘It was a really important fusion of academic, public sector, and commerce’.57 Indeed, it was with some emotion that Buchan recalled the comradeship of that period: ‘There was a strong relationship of trust between our military colleagues and public health workers on the ground. There was a lot of respect. It felt like one team’. The support from 8 Brigade in this situation—and the civil-military team’s solidarity—became crucial in sustaining the effort: ‘It was very moving. The discipline was great […] I enjoyed working with the military. They were disciplined, respectful, kind, and dependable. I was working with people who were honourable’.58 The Israeli case also illustrates the importance of human teamwork. The Gospel and Lavender are artificial intelligence systems. Some of their functions operate independently of human intervention. They are automatic; they process data without immediate human intervention to generate new results. Yet the Gospel and Lavender have not automated military decision-making. On the contrary, they rely on a large number of human experts to support, maintain, manage, and update them. The Gospel and Lavender do not work on their own; they are employed by the IDF’s Target Administration Division. That division, as Kochavi has
stressed, ‘includes hundreds of officers and soldiers’.59 Without those operators, the Gospel and Lavender could not function. It would be quite wrong to suggest that the Gospel and Lavender have displaced Israeli commanders, much less their legal and ethical responsibilities. Israeli commanders have to determine the parameters within which the Gospel and Lavender will operate; they have to direct the technicians to write and refine the software. They still make the final decision of whether to prosecute Hamas targets—even if they have ruled that, because the operation is urgent, subordinates will have only twenty seconds to confirm a strike. AI has allowed the IDF to target more quickly, more accurately, and more widely. Despite the fears of many commentators, AI has not superseded humans in this area; on the contrary, precisely because AI has required the support of data scientists, engineers, and programmers, the number of humans involved in the targeting process over the last decade has increased. Ironically, rather than eliminating people, AI-enabled targeting may, therefore, multiply organisational problems in military headquarters, as a broader range of human operators have to develop new ways of collaborating with each other if they are to exploit AI’s potential.
8 AI and Cyber Operations In their AI strategies, discussed in chapter 3, the US, the UK, and NATO described how they wanted to apply AI to enhance situational awareness and intelligence analysis. Specifically, AI would help the armed forces to plan and target by processing massive amounts of data. In this way, it would help them to conduct physical operations in the real world. AI has also been equally important for virtual operations in cyberspace. Indeed, in the last decade, policymakers and military professionals have become increasingly worried that AI will be used to prosecute cyberwar; enemies will use AI to prosecute digital attacks. Instead of attacking physically with bombs and guns, an enemy might destroy civilian infrastructure with a keystroke; banks, hospitals, powerplants, government administration, and news agencies might be interdicted with ease. Viruses and malware might be more destructive than bombs and rockets. In a 2012 speech, Leon Panetta, the US secretary of defence, warned of a ‘cyber Pearl Harbour’. He feared that one of the US’s enemies—China, Russia, Iran, or perhaps a non-state actor—might mount a cyberattack that would result in catastrophic damage to or the destruction of a critical piece of national infrastructure, such as a power grid, a nuclear power plant, or a dam. He said: ‘An aggressor nation or extremist group could use these kinds of cyber tools to gain control of critical switches. They could derail passenger trains, or even more dangerous, freight trains loaded with lethal chemicals. They could contaminate the water supply in major cities, or shut down the power grid across large parts of the country.’1 In 2011,
Admiral Mike Mullen, the chair of the US Joint Chiefs of Staff, declared that ‘the single biggest existential threat that’s out there, I think, is cyber’.2 In their book on AI, Kissinger, Schmidt, and Huttenlocher were also deeply concerned about the potential of AI as a cyberweapon. Automating Cyberwar The concept of AI-enabled cyberwar—that AI might be able to attack computer networks, clouds, and codes autonomously—is troubling. Yet, once again, it is easy to fall into hyperbole, exaggerating the powers of AI and the actual effects of cyber operations. The proponents of cyberwar have misrepresented its character. They therefore overstate the risks of AI automation. In On War, Clausewitz famously claimed that ‘war is […] an act of force to compel our enemy to do our will’.3 Thomas Rid has rightly argued that cyber operations cannot independently meet the criteria for war: ‘Any act of war has to have the potential to be lethal; it has to be instrumental; and it has to be political’.4 War must be an act of violence, perpetrated by a recognisable actor, with political goals which opponents understand. On these criteria, ‘not one single past cyber offense, neither a minor nor a major one, constitutes an act of war on its own’.5 Indeed, ‘no cyber offense has ever caused the loss of human life’.6 Cyberwar on its own cannot be war. In June 2022, the British Army’s Chief of the General Staff, General Sir Patrick Sanders, in his annual address at the Land Warfare Conference, summarised the situation bluntly: ‘You can’t cyber your way across a river’.7 War is purposive, physical, and violent. Even as cyber operations become more important to conflict, and as AI plays a greater role in them, it would be wrong to presume that war itself will be automated as a result. Cyber operations are a preliminary or an adjunct to the actual physical fighting, not a substitute for it. AI has played an increasing role in cyber operations. It is vital that we understand this role by looking more closely at the contribution of AI to cyber operations.
Cyber operations have taken three forms: sabotage, espionage, and subversion. An actor can employ a cyber operation to damage an opponent’s computer hardware, data, and software programs; it is possible to corrupt, poison, and degrade the enemy’s computer systems with worms, viruses, and malware. It is also eminently possible to hack into computer systems to steal information and data. Computer systems are very vulnerable to infiltration. Spying has, therefore, been one of the most common forms of cyber operation over the last three decades; China, in particular, has perpetrated cyberespionage against rival states on an industrial scale. The Snowden revelations showed that the National Security Agency also monitored cyber traffic, even that of US allies. Finally, state and non-state actors have employed the cyber domain, and especially the internet and social media, to subvert their opponents; the cyber domain is a fertile space for information and psychological operations. Actors promulgate their messages, targeting specific communities and even individuals for recruitment or dissuasion. Operations in cyberspace have not always involved AI. Nevertheless, precisely because cyber operations take place in the digital world of clouds, computer networks, and data, AI is eminently well-adapted to them. AI can be employed to develop the software which actors use to corrupt, to hack, and to subvert digitally. Indeed, in many cases, successful cyber operations rely on AI. Offensive cyber operations have therefore tended to depend on AI. Defensive cyber operations are even more reliant on AI. It is impossible for humans to monitor digital systems manually. For instance, in 2016, as debates about AI and lethal autonomy were intensifying in Washington, Robert Work, the deputy secretary of defence, ruled out abrogating control to AI, saying ‘The US will not delegate legal authority for a machine to make a decision’. There was one exception, though: ‘The only time we’ll delegate authority is in things that go faster than human reaction times, like cyber or electronic warfare’.8 He stressed that point strongly in his address to the North Atlantic Council, discussed in chapter 3. Malign code and viruses operating at computer speed have to be countered at the same tempo. Only AI-enabled programs trained by machine learning can
even begin to accomplish that task. AI has, therefore, become essential to computer defence. AI is able to identify deviant, malignant code automatically. States and non-state actors, such as commercial companies, depend on AI programs to identify hostile infiltrations, warning and, if possible, deterring and defeating the incursion before it can penetrate the system. AI-enabled programs automate this process so that they can operate at computer speed. AI has therefore become a major part of military operations in the cyber domain. As Jon R Lindsay, the security studies scholar, has noted, ‘Most advanced militaries are actively experimenting with cyber attacks for command and control surveillance, deception and disruption’.9 AI has played a major role in their experimentation. To this end, many military forces have instituted cyber units which employ AI to develop code for offensive or defensive actions. In Russia, the GRU’s Unit 26165 and Unit 74455 are cyber specialists; China’s People’s Liberation Army’s Units 61398 and 61486, North Korea’s Bureau 121 and Unit 180, and the Iranian Government Islamic Revolutionary Guard Corps (IRGC) all conduct cyber operations. The US and the UK have Cyber Commands at the strategic level to coordinate against national threats. Cyber capabilities have also been developed at lower levels in the armed forces themselves. Israel’s Unit 8200 is one of the leading exponents of cyber operations; the British Army’s 77 Brigade have operated in this space too. AI is an integral part of these units’ capabilities; it allows them to operate more effectively in cyberspace. NotPetya and Stuxnet In 2012, Leon Panetta claimed that a cyber Pearl Harbour was imminent. No strategic cyberattack has yet occurred. Nevertheless, several cases have foreshadowed the potential destructiveness of a cyberattack. In 2009, Russia suffered a disaster at the SayanoShushenskaya dam. Software on a malfunctioning computer, located some five hundred miles from the dam, sent a signal to start a turbine, which released a flood, killing seventy-five people and ruining the other turbines.10 It seemed possible that a malign actor
could easily replicate this event by infecting a computer system with a virus. Israel has been at the forefront of cyberattacks of this type, using them against the nuclear facilities of hostile neighbouring states Syria and Iran. For instance, on 6 September 2007, in an action called Operation Orchard, Israeli aircraft bombed the Dayr ez-Zor nuclear reactor in northern Syria. Syrian air defence was blinded for the duration of the attack. It seems likely that the IDF’s Unit 8200 had successfully infected the Syrian computer systems so that their air defences failed.11 In 2010, further evidence of an Israeli cyberattack became public. Israel had been concerned about the Natanz nuclear facility in northern Iran for some time. The plant was producing enriched uranium as part of Iran’s attempt to acquire a nuclear bomb. Israel considered the possibility of another airstrike. However, the chances of collateral damage were considered too high. Consequently, Israel decided on a cyberattack. The CIA and NSA became involved in the Natanz project and what became known as the Stuxnet attack. The Stuxnet attack was complex and expensive; $300 million was authorised for ‘joint covert projects’ of which it was part. It took the Americans and Israelis many months to develop the malware. The Natanz facility used SIMATIC Step 7 software on Microsoft Windows. The Stuxnet attack exploited a generic code in Windows. It seems to have been introduced into the Natanz system by a human agent who gave the infected media to an unwitting employee at the nuclear power plant. Once that person had uploaded the Stuxnet malware, the attack code altered the cycle in one of the centrifuges; it repeated an irregular cycle until the centrifuges eventually broke. Stuxnet damaged 11.5 per cent of Natanz’s 8,700 centrifuges.12 The Stuxnet attack was a striking episode, though its effects were modest. It was a long way short of a cyber Pearl Harbour. Iran replaced a thousand centrifuges by January 2010, though Natanz had already increased production of enriched uranium from 80 kilograms a month in mid-2009 to 120 kilograms a month by mid2010.13 The Stuxnet virus was inconvenient to Iran, costing time and money to fix; it probably slowed production, but it was by no means
decisive, much less catastrophic. Stuxnet also spread globally, eventually infecting one hundred thousand machines, including those at the US energy giant Chevron. Other states have demonstrated their offensive cyber capabilities too. On 12 May 2017, North Korea launched a malware attack on Microsoft software. The attack eventually affected two hundred thousand computers, particularly severely those of the National Health Service in the UK. All these events were serious breaches. However, Russia has executed many of the most destructive and well-known attacks. In April 2007, the Estonian government relocated the Bronze Soldier, a statue memorialising the Soviet victory in the Second World War, in Tallinn, precipitating public disorder. Russia responded with a distributed denial of service (DDoS) attack on 27 April. Eighty-five thousand computers were infected, with a peak on 9 May, which coincided with Russia’s Victory Day celebrations. The following year, Russia mounted a military action in support of South Ossetian separatist forces against the Georgian Army. The Russian Army quickly defeated the Georgians to ensure that South Ossetia remained within the Russian sphere of influence. The military actions were supported by computer attacks which started on 29 July and culminated on 8 August; the attacks involved defacing websites, denial of services, and malware.14 In each case, the effects of the attacks were relatively minor—what makes these attacks notable is that they may mark the first time Russia employed a cyberattack to pressurise a rival state and the first time that such attacks were coordinated with military action. In 2014, Russia and its proxies seized the Crimea and the Donbas, precipitating a war in eastern provinces as the Ukrainian government sought to regain its territory. Eventually, following the deployment of a large Russian army force to support the Donetsk and Luhansk People’s Republic Armies, the war descended into a stalemate in 2015. However, Russia continued to mount cyberattacks on Ukraine. In December 2015, Russian hackers took down three regional electricity power stations. In 2016, they increased their attacks against Kyiv’s transmission system.15 These attacks
culminated in a massive cyber onslaught the following year. On 27 June 2017, Russia mounted its largest cyberattack: ‘NotPetya’. This attack targeted Ukraine and is regarded as the most destructive ever. Masquerading as ransomware, a malicious data-encryption tool was inserted into Ukrainian governmental and financial systems.16 The UK’s Foreign Office Minister of State with responsibility for cyber, Lord Tariq Ahmad, claimed at the time: ‘The attack showed a continued disregard for Ukrainian sovereignty. Its reckless release disrupted organisations across Europe costing hundreds of millions of pounds’.17 It was estimated that the attack did $10 billion worth of damage: the pharmaceutical giant Merck lost $870 million; FedEx’s European subsidiary, TNT Express, lost $400 million.18 NotPetya was an attempt to pressurise the Ukrainian regime, as Russia sought a favourable settlement for the Donbas and Crimea. AI has also been employed for cyberespionage, hacking into systems to steal secrets. China has been accused of many cyber intrusions of this type, including Titan Rain, Byzantine Hades, Aurora, and Shady RAT. Between 2007 and 2009, a large amount of data about the F-35 was stolen from the Pentagon—presumably by China. Although it was unclear whether it originated in China, Titan Rain accessed hundreds of firewalled US military and governmental computer systems to download 10–20 terabytes of information from non-classified Defense Department networks.19 Ukraine On 24 February 2022, Russia mounted its full-scale invasion of Ukraine. Ukraine was extremely vulnerable to cyberattacks. Only a week before the invasion, Ukraine was still operating on local services inside government buildings. Consequently, US tech companies volunteered their services to defend Ukraine’s network: ‘American firms big and small, including Microsoft, gave early warning of cyber attacks, providing patches, coordinating in real time with governments and other companies on actionable threats. Others have taken private sector organizations under their digital umbrella to shield them from cyber attacks with their superior cyber
defense resources’.20 On 17 February 2022, the Ukrainian government altered its laws so that its data could be transferred to the cloud, located outside the country. The Russo-Ukraine War itself has been characterised by savage attritional fighting; as of October 2024, Russia has lost over 150,000 troops, with over 350,000 more wounded, while perhaps 60,000 Ukrainian soldiers have been killed. Cities have been damaged; many towns in the east have been destroyed. Nevertheless, cyber operations have been an important current in this conflict. Following the NotPetya attack of 2017, the Ukrainian government and its allies were understandably concerned that Russia would mount a massive cyberattack in support of its military operations. Indeed, Lindy Cameron, the head of the UK’s National Cyber Security Centre (NCSC), claimed that Russian cyberattacks were ‘probably the most sustained and intensive cyber campaign on record’.21 Brad Smith, the CEO of Microsoft, described the attack as follows: ‘The Russian invasion relies in part on a cyber strategy that includes at least three distinct and sometimes coordinated efforts—destructive cyberattacks within Ukraine, network penetration and espionage outside Ukraine, and cyber influence operations targeting people around the world’. The sabotage attempts were particularly serious: ‘Russia not surprisingly targeted Ukraine’s governmental data center in an early cruise missile attack, and other “on premise” servers similarly were vulnerable to attacks by conventional weapons. Russia also targeted its destructive “wiper” attacks at on-premises computer networks’.22 The cyberattacks were intended to amplify the shock induced by the invasion, paralysing the Ukrainian government. The attacks were serious. Nearly every major Russian state actor participated, using a range of tools, from routine distributed denial of service attacks to novel data-wiper malware. On 23 February, wiper programs were identified on hundreds of Ukrainian systems. Russia then attacked Viasat, Ukraine’s satellite communications system, on 24 February. In April 2022, hackers working for a company called Sandworm (probably a front for the GRU) used Industroyer2 malware to assault the Ukrainian electricity grid.23
The Russians invested significant effort in their cyberattacks. According to David Cattler, NATO’s leading intelligence official, Russia had used more destructive malware against Ukrainian systems ‘than the rest of the world’s cyber powers combined typically used in a single year’.24 Nevertheless, the Russian attacks were less effective than they might have been. The attacks were poorly coordinated. For instance, Russian air strikes destroyed the same network Russian hackers were trying to infect. In addition, the Russians released their malware prematurely, so it was riddled with errors. The viruses spread beyond their intended targets. As The Economist reported, ‘There were significant operational failings in almost every single attack that [the Russians] have ever carried out in cyberspace’.25 As David Cattler noted, ‘Russia is almost certainly capable of cyberattacks of greater scale and consequence than events in Ukraine would have one believe’. After the attack on Viasat, Russia’s cyberattacks became opportunistic and tactical.26 Russia seems to have chosen to conduct multiple lower-end attacks, rather than a single dedicated infiltration. Russian cyber strategy may have been poor. Yet the failure of the Russian attacks may very substantially be a result of the assistance of Western tech companies. Microsoft had already been involved in Ukraine before the invasion. Just before the invasion, an attack on the Ukrainian government and financial websites with a cyberweapon called Foxblade had failed. Microsoft’s Threat Intelligence Centre had written a program to protect Ukraine’s networks. Once the invasion had started, Microsoft worked hard to move Ukraine’s data to the cloud, and the government worked closely with private companies to back up its critical data securely.27 Microsoft also detected Russian attacks outside of Ukraine, where Russia sought to penetrate and spy on the governments which were supporting Kyiv; it had targeted 128 organisations in forty countries. Ukrainians relied on the intervention of Western, and especially US, private tech companies to defend them from Russian depredations. However, Ukraine has become a highly digitised society with a vibrant and youthful tech sector of its own. This expertise has been harnessed in the course of the war. Indeed, the
transition has been remarkable. In 2020, Harvard’s Belfer Center for Science and International Affairs ranked Ukraine twenty-sixth out of twenty-nine cyber powers. In 2022, Ukraine was ranked twelfth out of thirty. The so-called IT Army of Ukraine has been a major force behind this digital transformation. Soon after the invasion on 24 February 2022, Yegor Aushev met Mykhailo Fedorov, the vice prime minister and minister of digital transformation. Together they decided to assemble 1,000 to 1,500 IT specialists; the IT Army of Ukraine was established on 26 February, two days after the invasion.28 The IT Army of Ukraine also recruited volunteers from across the EU; the Ukrainian International Legion consists of 20,000 people, some of whom have joined the IT Army.29 Ukrainian cyberdefences have improved. However, Ukraine has developed significant offensive capabilities too. The IT Army has been engaged in many of these cyberactivities. The strategy was originally disorganised; it exploited any available avenue for denial of service attacks. Ukraine mounted numerous cyberattacks on Russia, including assaults on Russian banks. It created a website which contained DDoS tools so that Ukrainian operatives—and indeed any supporter of Ukraine—might initiate a cyberattack against Russia. During the early part of the invasion, the IT Army designated DDoS targets by posting URLs on Telegram, a Russian social media site. In October 2022, the IT Army breached the Russian company Loesk’s power plant to steal its customer databases. It also targeted Gazprombank.30 In January 2023, it disabled the accounts of a proRussian hacktivist group, NoName057(16).31 As the war has progressed, the IT Army has developed a more purposeful strategy. It has gradually focused on DDoS attacks, for which it has typically employed private networks. This is advantageous, as private accounts are difficult to locate. For instance, Russia might perceive an attack to be from Finland when the origin of the hack is France. In both attack and defence, AI has been crucial.
Algorithmic Warfare Cyberspace has become a fertile domain for AI-enabled sabotage and espionage. Cyberspace has also become an important sphere for information and psychological operations. States and non-state actors have increasingly sought to address audiences, recruiting supporters and undermining opponents in the digital sphere. It is, of course, possible to conduct digitised information operations manually, without the aid of AI. However, AI has played an increasingly prominent role in digitised information operations. Indeed, AI has become inseparably associated with subversion campaigns. Facebook, Instagram, Twitter/X, Telegram, and other social-media sites employ algorithms to organise their content and disseminate it to the appropriate audiences. It is no longer possible to discuss information operations in cyberspace without considering the role of AI in these activities. AI has become a crucial enabler— identifying audiences, generating content, disseminating it, and amplifying any reaction it stimulates. Algorithms on social-media platforms sift through posts, promoting some material over others, preferring certain audiences while demoting others; they profile authors, their followers, and current trends to calculate the importance of a post. AI has identified fertile audiences to amplify the effect of its messaging. AI has been routinely employed to automate messaging, multiplying or even fabricating content, to increase its exposure. Algorithms influence and often determine what users see and experience. Tech companies have often employed algorithms to influence consumers. Amazon’s algorithms which recommend and advertise commodities on the basis of users’ buying history are at the more benign end of this digital influencing. However, algorithms have been used for more sinister purposes. For instance, Cambridge Analytica used private Facebook data to identify and profile voters in order to influence and, perhaps, subvert the US presidential election of 2016. States and their armed forces have also employed AI and algorithms for information operations. They have tried to rally activists to their
cause while marginalising their opponents. They have employed big data and algorithms to help them pursue this strategy. Russia has, of course, been at the forefront of the use of algorithms for information operations. Since the Soviet era, Russia has been a proponent of ‘active measures’; subversion campaigns, including information and psychological operations, have been a central part of its strategy. Yet, while active measures have been a long-standing technique, ‘the digital revolution fundamentally altered the disinformation game’. In particular, ‘The rise of networked computers gave rise to a wider culture of hacking and leaking’.32 With the rise of social media and AI, subversion became ever easier for Russian intelligence agencies the FSB and GRU. As a result, ‘By the early 2010s, it was easier than ever to test, amplify, sustain, and deny active measures, and harder than ever to counter or suppress rumours, lies, and conspiracy theories’.33 Russian subversion in the twenty-first century has not always required AI. On the contrary, traditional methods have worked on digital platforms. Nevertheless, the Russian intelligence services have developed some sophisticated uses for software programs and AI. In particular, automated ‘bots’ have played an increasingly important role. In Rid’s estimation, ‘Active measures could even be technically amplified, by using semiautomated accounts and fully automated bots’.34 The Russian intelligence services have created several units to conduct digital subversion campaigns, using bots and AI programs to help them. For instance, during the 2014 invasion of Ukraine, the GRU’s Unit 54777 forged emails, putatively from the CIA, instructing far-right Ukrainian commanders to attack an airfield in eastern Ukraine. The Russians have exerted considerable energy on automating these cyber operations. To that end, the Kremlin established the Internet Research Agency, which is, in fact, a troll factory. By mid-2015, the agency employed over eight hundred staff and, by September 2016, had a monthly budget of $1.25 million.35 It has specialised in the use of bots. Bots are fake social-media accounts, generated by software programs, to replicate human actors. Unlike human operatives, a bot can post and repost feeds almost infinitely; bots automate
messaging. Bots are not sophisticated; they consist of scripts which bots disseminate in conjunction with human-operated accounts. They amplify human-created content, rather than autonomously generating their own.36 However, even though they are not always AI-programmed, bots actively exploit the AI algorithms which organise social media. Through sheer force of numbers, bots can trick internet algorithms into promoting their content over others. Consequently, bots have rightly engendered deep concern. The fear is that bots have so colonised certain news stories with misinformation and fake news that they are now influencing human behaviour. The bots are effectively acting autonomously. In 2017 it was believed that 15 per cent of Twitter accounts may have been bots.37 There is, in fact, very good evidence that the Russian state, for instance, has become adept at exploiting bots to influence public debate in the West. There are several examples of how bots have manipulated algorithms. In the run-up to the 2016 US presidential election, the Russian Internet Research Agency posted on Facebook a cartoon of Satan arm-wrestling Jesus, under the caption ‘Satan: If I win Clinton wins. Jesus: Not if I can help it!’ House Democrats used the cartoon as an example of the dangers of Russian disinformation. Many news outlets picked up on the story and, therefore, in complete opposition to the Democrats’ intentions, promulgated the Russian subversion. The Internet Research Agency’s bots had enrolled traditional journalism into their AI-enabled campaigns. Russia and other states have also employed AI to generate and multiply content for social media. In many cases, bots have promulgated footage of actual events or commentary on events. AI has also facilitated more sophisticated deceptions. With the help of AI, it has been possible for state and non-state actors to generate ‘deepfakes’. Generative adversarial AI has been central to the creation of these deepfakes. Adversarial AI operates by setting two machine-learning programs against each other. One generates the fake content; the other detects it. The models learn from each other, iteratively generating fakes until they are more and more convincing. Trained on large datasets, some AI programs of this type can now
manipulate footage so effectively that it seems entirely authentic.38 Criminal gangs have been leading the way here. In 2019, fraudsters used an AI-generated voice to impersonate the CEO of a European company, directing executives to transfer $243,000 for a putatively urgent business deal. In 2023, the Hollywood actor Scarlett Johansson was the victim of a pornographic deepfake; in 2024, Taylor Swift, the pop star, was targeted. Deepfakes potentially have military implications as well. States have increasingly employed deepfakes as part of their disinformation campaigns and as an adjunct to military operations. From the start of the Russo-Ukraine War, Russia has spread disinformation claiming that the US had bioweapon labs in Ukraine. The Russians have also employed some deepfakes. For instance, in March 2022, footage of Ukrainian president Volodymyr Zelensky in which he called for Ukrainians to lay down their weapons appeared online. The video was, of course, fake, and it was quickly exposed. Some deepfakes are easily identified by close observation, but others are very difficult to detect. Indeed, tech companies have developed AI programs for detecting deepfakes. For instance, a company called Sensity has developed deepfake software because, as its director has warned, in the near future ‘we will not be able to certify that something is authentic just with our senses’. In other words, it will become necessary to fight AI with AI.39 In the current war with Ukraine, the Russian state has sought to employ AI-enabled information and psychological operations. The social-media site Telegram, through which the Kremlin, the FSB, and the GRU have promulgated footage, has played an important role here. Telegram, a Russian-owned platform, is one of the most accessible sources of battlefield footage. The pro-Russian Telegram channel Colonelcassad has 800,000 subscribers; the Rybar opensource intelligence channel has 1.1 million.40 Russian bloggers have employed Telegram extensively, especially on the channel ‘Look for your own’, which features material from Ukraine. For instance, on 12 July 2022, the blogger ‘Know Khokla by Chuki’ posted a twenty-twosecond video of the corpses of Ukrainian soldiers killed in battle. The video was accompanied by a strange Wagnerian epithet: ‘Lohengrin
midnight. The swan chivalry of the Russian army breaks the pagan bastions of Nazism’. By October 2022, it had been viewed forty-five thousand times.41 The Ukrainian government, intelligence services, and armed forces have become adept at digitised psychological operations, and it is widely believed that the Ukrainians have been more successful in the informational space than the Russians have. They have learnt from their experiences in the Donbas in 2014 and 2015. In some cases, they have employed Russian psychological techniques against Russian soldiers. They have prosecuted AI-enabled psychological warfare. In addition to carrying out cybersabotage, the IT Army of Ukraine has played an important role in these campaigns. Its Telegram account, itarmyofukraine2022, has frequently posted material in support of the Ukrainian war effort. Using Clearview AI software, Ukrainian officers have conducted thousands of facial recognition searches to identify dead or captured Russian soldiers. The IT Army of Ukraine has then used this imagery to inform Russian families of the deaths of 582 Russian soldiers, sending them photographs of their corpses.42 A Ukrainian disinformation group, with the handle @stop_russian_war_bot, has also employed Telegram.43 Like the Russians, the Ukrainians have experimented with deepfakes. In 2022, they released stories about ‘the Ghost of Kyiv’, a Ukrainian pilot who had shot down six Russian planes. Ukrainian pilots had played a vital role in the defence of Kyiv, and some had destroyed several Russian planes. Yet the Ghost of Kyiv was fictional; the footage had been generated from a computer game.44 The éminence grise behind many of Ukraine’s information operations is General Kyrylo Budanov, a legendary figure. In early 2015, as a lieutenant colonel operating behind enemy lines in the Donbas, Budanov was wounded by an anti-personnel mine; shrapnel penetrated his neck and shoulder. At first, he demanded that his soldiers leave him. When they refused, he walked three kilometres to safety. In 2016, he led a raid to destroy helicopters at Dzankoi air base in the Crimea which resulted in the death of several Russian Spetsnaz soldiers. His subordinates have described him as a snake
hypnotising his enemies ‘before he comes in for the kill’. At the start of the Russian invasion, armed with a machine gun, he personally intercepted saboteurs and commanded operations at the Hostomel airfield in Kyiv. The Russians have consequently targeted him on ‘at least ten’ occasions. Budanov has organised the most daring, and controversial, raids into Russia, including the drone attacks on Moscow from 2022 and the Bryansk Raid in March 2023.45 As an intelligence commander, Budanov has been responsible for subverting Russia’s war effort. Digital communications, social media, and the manipulation of algorithms have been central to his methods. Budanov has exploited the power of social media to hypnotise Russians so that Ukrainian forces might kill them. The drone strikes on Moscow and many of the other raids have had minimal military relevance, but they have been part of Budanov’s information and disinformation campaign. AI has often enabled Ukrainian psychological and information operations. The Russians are by no means alone in conducting offensive cyber operations. On the contrary, although initially the US might have preferred to use cyber defensively, it has developed significant offensive capabilities too. Joint Task Force Ares’ Operation Glowing Symphony in 2017 is the most celebrated example of such an attack, in the public domain at least. In 2014, ISIS captured Mosul, in Iraq, with the intent to establish a caliphate. In addition to its military capabilities, ISIS developed very sophisticated cyber capabilities, having recruited some outstanding programmers. Junaid Hussain, a renowned hacker from London, was probably the most famous. He and other ISIS operatives infiltrated US Defense Department archives—Hussain himself released the names of one hundred US service personnel on an ISIS kill list. The organisation conducted brilliant information operations which terrified opponents and inspired global recruitment. Eventually, the US recognised that ISIS had to be opposed in cyberspace as well as on the ground. Consequently, in 2015, US Cyber Command was integrated into Operation Inherent Resolve, the campaign against ISIS. Cyber Command established a specialist group called Joint Task Force Ares in May 2016, which, by mid-2016, began to plan Operation Glowing
Symphony, the aim of which was to hack into ISIS systems and prevent ISIS cyberwarriors from posting material or communicating online.46 In the summer of 2016, Joint Task Force Ares executed a small attack on ISIS to test the concept. In November of that year, Operation Glowing Symphony was launched at Fort Meade in Maryland. The task force had plotted the ISIS computer network and identified its key points—programmers and hackers who acted as hubs in the system. Eventually, the task force targeted about sixty operatives whose accounts were particularly important to ISIS cyber operations; each one was tabulated and numbered on a large ‘bingo card’ which hung on the wall of the operations room at Fort Meade. Task force personnel were organised into four teams, each consisting of four members: an operator working the keyboard to control the malware, a signals-intelligence development analyst who was an expert in the ISIS network, an intelligence analyst who studied the target organisation and individuals, and the team leader. Each team was tasked to disrupt ten to fifteen targets. It was imperative that the operation be enacted quickly so that ISIS could not react. On the launch day of the operation, the commander of the task force ordered ‘Fire’, and the teams began to hack into ISIS’s systems. Within fifteen minutes, it was clear that the operation was going to be successful, and after six hours, the teams had ‘eliminated’ many of the targets; they had hacked into and disrupted their accounts, deleting files and changing passwords. The preparatory work the teams had done proved vital. For instance, early on in the operation, one of the teams encountered the security question ‘What is the name of your pet?’ The team was momentarily stumped and feared the entire operation would unravel. However, the intelligence analyst on that team had been studying the target for years and knew the answer to the question: ‘1257’, presumably the date of the Mongol siege of Baghdad.47 Operation Glowing Symphony was a major success; ISIS propaganda outlets were delayed, and Task Force Ares was still accessing ISIS systems two years after the attack.48
Audiences AI has helped states and non-state actors promulgate messages as part of their information and psychological operations. Yet civilians have not been merely passive targets of these operations. On the contrary, they have actively participated in these information struggles, often exploiting AI and algorithms to advance their own positions. The Second Nagorno-Karabakh War is a good example of how ethnic diasporas across the world have participated in war through internet activism. Nagorno-Karabakh, a region between Armenia and Azerbaijan, had been disputed since the end of the Cold War. Regional competition for control of this territory led to a series of disputes between Armenia and Azerbaijan between 1988 and 1990. After the collapse of the Soviet Union, in 1992, a full-scale war broke out between the two states which, after two years of intense fighting involving mass displacement and approximately twenty-five thousand deaths, the Armenians won. Armenia established Karabakh as a de facto state, the Republic of Artsakh. Although defeated, Azerbaijan continued to dispute Nagorno-Karabakh. In the next two decades, there were numerous ceasefire violations, culminating in a four-day war in 2016. On 27 September 2020, Azerbaijan launched a major military operation to retake Nagorno-Karabakh. Its forces attacked from the south, defeating Armenian forces and eventually seizing the Lachin Corridor and Shusha. On 10 November, Armenia declared a ceasefire, accepting defeat and ceding Karabakh. The military dimensions of the Nagorno-Karabakh War are profoundly interesting. Yet, during this war, information operations in cyberspace constituted a significant domain of conflict. Algorithms played a role here. In 2020, the ethnic Armenian diaspora sought to support Armenian forces and citizens in Nagorno-Karabakh, with resources, capital, and influence. The internet was an important forum, as digital diasporas extended ‘hostilities, nationalist mobilization and polarization into the international arena’.49 For instance, individual Armenians or Armenian bots verbally attacked celebrities who expressed support for Azerbaijan, and vice versa. As
one young Russian commented: ‘This war is so real that any celebrity who says something pro-Armenian gets bombed with hateful messages, by bots and real Azeris. And a day later they say, “Sorry folks, we are not ready for this, we take our words back”’.50 At other times, activists sought to enhance Armenia’s profile through the use of hashtags and links to prominent figures. Diaspora Armenians also developed sophisticated collective cyber tactics in order to promote their political message. They tried to manipulate AI algorithms. The self-named Armenian Twitter Army on Telegram actively sought ‘to get the algorithm on our side’. Armenian activists customised their tweets, connecting them to other tweets by their group with hashtags that algorithms would note. They learned how to report and ban abusive Azerbaijani comments.51 Diaspora Armenians sought to manipulate social media algorithms to enhance their message. They were conscious that they needed to counter what they took to be Azerbaijani bots. As one Armenian activist described their tactics: If you go on Twitter and type #stop, I want #stopAzerbaijaniAggression, #stopTurkey. The first ones are #stopArmenianAggression, #stopArmenianLies. Why is theirs before [ours]? That’s because [Azerbaijani] bots are doing it. So, our aim is to get on top, so when people on Twitter are looking at US news we’ve got #sanctionTurkey trending. We want people to be curious, we want it to be in front of their face.52 Activist groups endeavoured to become ‘algorithmically recognisable’, exploiting software biases to exaggerate their profile; in this way, they engaged in an ‘algorithmic dance’. Daniel Chernobrov has summarised the dynamics of this campaign: ‘Trending hashtags, likes, reposts and coordinated tactics become aims and metrics of participatory warfare, as they determine the visibility of political causes and groups. Algorithms are productive of culture and politics, as they make certain subject positions more real and available—not unlike the power of other intermediaries and gatekeepers that selectively grant visibility and certify meaning’.53
The audience for this algorithmic activism during the NagornoKarabakh War was not Azerbaijani opponents. It was actually counterproductive in this information war to engage with opponents, since social-media algorithms ensured that it would simply improve Azerbaijani visibility; algorithms promoted tweets and posts which attracted attention. The process was self-reinforcing: the more likes and follows a post attracted, the more the algorithms disseminated it, to encourage even more support. So the targets of these information operations were third parties, host-nation governments, and international organisations. The activism of the Armenian diaspora is a useful example of how AI is playing a role in contemporary information operations. AI has not in any way colonised information operations, as some have feared. Armenian activists were paramount in creating and coordinating a network. They learnt how social-media algorithms operated and what kind of content those algorithms favoured, and they were able to manipulate an automated, AI-enabled system to their advantage. The Human Factor Cyber operations—espionage, sabotage, and subversion—have become an important element in contemporary warfare. States and non-state actors are increasingly exploiting cyberweapons and cyberspace to undermine their opponents and to empower their supporters. They are hacking into computer networks to sabotage their enemies’ systems, to spy on them, and to subvert their populations. AI has become an important supporting element of these operations. AI has been vital in defending against cyberattacks, identifying vulnerabilities, and countering malware. AI, algorithms, and deepfakes have also played an increasing role in information and psychological operations. Yet it is important not to exaggerate AI-enabled cyber operations’ impact. It is true that cyberattacks can be destructive; NotPetya, for example, caused damage and disruption. States and their armed forces will continue to develop their offensive and defensive
cyberweapons. Cyber operations are likely to become more sophisticated, with AI playing an ever-greater role. Nevertheless, and despite the rhetoric, the limitations of cyber operations are clear. They have been exposed very obviously in Ukraine, where Russia has mounted one of the largest and most sustained offensives. Yet the Russian attacks on Ukraine have been far less effective than Leon Panetta’s prediction of a ‘cyber Pearl Harbour’ suggests. The contrast with physical ordnance is telling. Air strikes, for instance, cause massive explosions. They are visible and immediate acts of destruction which inflict death, injury, and misery. In war, the element of shock is critical. Consequently, military forces prioritise firepower; they require immediate, tangible results. Russia has mounted many cyberattacks against Ukraine, and the effects have been significant. Yet Russia’s military actions have been far more important. Russian forces have actually seized and occupied Ukrainian territory. Russian artillery and airpower have destroyed Ukrainian towns and cities; they have killed many Ukrainian soldiers and civilians. The contrast with cyber operations, inconvenient and disturbing though they are, is profound. Ciaran Martin, the former head of the UK’s National Cyber Security Centre, has noted the ‘severe limitations of cyber as a wartime capability’.54 Compared with a military campaign, cyber operations are not normally dramatic or decisive. They have a modest, gradual effect. Although most militaries will use cyber to augment and assist their operations, it is unlikely that cyber operations can replace actual military forces, firepower, and fighting, no matter how sophisticated AI becomes. Cyber operations are an important new sphere for AI. Yet, the idea that an AI agent will be able to develop and insert malware which will instantly destroy infrastructure or bring a nation to a standstill—freely, easily, and anonymously—is fantasy. Cyber operations seem to affirm the point which becomes clear when we consider planning and targeting. AI is becoming an important element of military operations. The armed forces are developing their capabilities in this area, in competition with each other. Yet cyber operations, however enabled by AI, will not
revolutionise military operations or warfare. They will simply be employed to support traditionally brutal ways of fighting. Cyber operations have not been as apocalyptic as some have claimed, and it seems unlikely they will be. However, within their narrower sphere, there is still an important question to answer: Can AI automate cyber operations? That is, is an AI agent capable of conducting a cyber operation autonomously? In some small special cases, AI agents already have. For instance, some algorithms, especially ones that protect against malware, operate independently of human control and intervention. Defensive software must be autonomous if it is to spot malware instantly, at computer speed. Yet it is wrong to extrapolate that, in the future, cyber operations as a whole will be automated, totally controlled and directed by AI. On the contrary, while specific functions have been automated, identifiable actors, states, the armed forces, and non-state actors have organised and directed cyber operations. They have planned and executed espionage, sabotage, and subversion operations. For instance, the Stuxnet attack demonstrated that an effective cyberattack requires a huge investment of time, expertise, and money. That virus took American and Israeli experts many months to develop. Cyberattacks are rarely anonymous, either. Ciaran Martin has dispelled the myths around cyber capabilities; the area of cyber operations is not ‘some magic invisible battlefield where you can do stuff you can’t get away with normally’.55 Cyberattacks are easily attributable because they need an animating intelligence and intention. They need human actors who are trying to subvert their opponents. Human expertise was critical to the success of Operation Glowing Symphony. Without the skill of the people who hacked the ISIS system and even knew their targets’ passwords, the operation would have failed. Indeed, specific individuals are important here. As one US officer observed, ‘Talent matters’. Junaid Hussain, for example, was critical to the success of ISIS’s cyber campaigns. The human element in cyber operations is most apparent in information operations. For instance, although bots have been heavily exploited by various political actors in the past decade, subversion campaigns
consist of three elements: leaders, bots, and believers. At the apex, any information or psychological campaign requires a committed activist group, dedicated to the cause, which has a clear political goal. The Russian state, the GRU, the FSB, the IT Army of Ukraine, and General Budanov have developed communication strategies, programmed bots, and initiated and changed storylines in order to manipulate public perceptions. States—and their secret services— and non-state groups have often been involved in this process; both Russia and China have been exposed on a number of occasions. Having established a strategy, the activists have then employed bots and algorithms to replicate, multiply, and disseminate the message. The spread of bots and deepfakes is worrying, precisely because they have proved to be effective. But although bots and algorithms have amplified narratives, they have not autonomously invented and initiated them, much less orchestrated and commanded a cyber campaign independently. The final requirement is public networks of believers who accept and promulgate the message—not only on social media but within their own faith communities on a face-toface basis.56 Although individuals within this network often have few followers, they have a number of weak connections to mutual contacts. The result is that a bigger network, composed of localised communities, is mobilised. They are united into a homophily, which can also integrate more isolated, dispersed individuals. The weak links between the networks of believers also facilitate the communication of propaganda to external non-believers. Successful information campaigns rely on faith communities: ‘The cohesiveness of the group indicates how a coordinated effort can create a trend in a way that a less cohesive network could not accomplish’.57 During the Nagorno-Karabakh War, the Armenian diaspora showed that these human audiences for information operations, however compelling they might be, and however sophisticated the software behind them, are not mere receptacles. The audience members exert collective agency to interpret and enact the messages and ultimately to manipulate the algorithms. Ultimately, they define the meaning of the message, not AI.
This chapter and the previous two have identified three principal military functions for AI: planning, targeting, and cyber operations. In each case, the application of AI has offered the armed forces some significant advantages. AI has become remarkably capable. It can process mountains of data and can promulgate messages on the internet almost infinitely. However, in each of the three military functions—planning, targeting, and cyber—AI is not the dominant agent at work. In every case, AI has been programmed, directed, and controlled by expert human individuals, teams, and organisations. The profoundly important development is that those human collectives are no longer just military. In order to employ AI to help with planning, targeting, and cyber operations, the armed forces are increasingly reliant on the tech sector. Tech companies are becoming immediately involved in military operations. Military operations are not being automated by AI. However, the use of AI has seen the participation of new actors in military operations and the appearance of novel collaborations between tech experts and military personnel. A new form of civil-military agency is emerging.
9 The Human-Machine Team In the current literature on AI and the armed forces, some scholars have refined their understanding of how the supposed automation of war will occur. Instead of a simple substitution of humans by AI, they have proposed a concept of ‘human-machine teaming’. When scholars use the concept of the human-machine team, they acknowledge that humans will continue to play a critical role in military operations; AI is not about to take over the battlefield. Nevertheless, while avoiding the more fantastical predictions about AI, the idea of the human-machine team endows AI with independent agency. AI is not just a passive tool but a creative, independent actor which interacts with its human operators; AI will become a member of the armed forces, not just one of its weapon systems. In his recent book on AI, James Baker describes the emergent AI-enabled human-machine as a ‘centaur’: part human, part machine.1 AI has, according to Baker, huge potential for lethal autonomy. In order to remain competitive, the armed forces will have to embrace AI and lethal autonomy. Military centaurs will appear. However, at this point, the centaur confronts a dilemma: ‘how to reap the benefit of AI for national security purposes without losing control of the consequences’.2 To this end, Baker discusses the problems of having humans ‘in the loop’, ‘on the loop’, or ‘out of the loop’. He emphasises the importance of reliability, accountability, and verifiability. In order to overcome the centaur’s dilemma, Baker proposes the human-machine team. Humans will remain in control;
they will be the head of the centaur. However, AI might be used ‘to mitigate risk, repetition, fear, fatigue and speed’. On this centaur model, the humans remain the dominant partner, directing and framing the application of AI and especially lethal autonomous weapons, but within its designated areas—data processing and target identification, where AI is superior to humans—AI should be free to exercise its full agency. Baker, then, describes a highly distinctive military-tech assemblage; part human, part machine, to maximise the capabilities of partners. He proposes not a complete automation of war but a partial automation of some aspects of warfare. AI Fetishism Military professionals have been attracted by the concept of the human-machine team. For instance, the Israeli brigadier general Yossi Sariel, who commands one of the elite Israeli intelligence units which employs AI, believes that human-machine teaming will be critical to future military capability. Echoing the discussion in chapter 7, Sariel identifies targeting as one of the central challenges of contemporary military operations: ‘Big data will be the key to finding and understanding rivals and enemies. Data from hundreds of thousands of drones will be part of the basic information about everything’.3 At present, ‘humans are the bottleneck that prevent the creation of tens of thousands of targets’.4 However, AI offers a solution to this problem: ‘AI is the reality when a machine can absorb information from the environment, process it, and reorganise it’.5 To exploit the potential of AI, the armed forces need to reform themselves. The armed forces need to create headquarters in which military personnel work together with AI agents as a single cybernetic team. AI should be allowed to exercise its full autonomous powers to enhance human capability. According to Sariel, a fully enabled human-machine team ‘has the ability to create tens of thousands of targets before a battle begins, and to assemble thousands of new targets every day during a war […] Imagine 80,000 relevant targets that are produced before combat and 1,500
new targets every day during a war’.6 Sariel believes that once the IDF have created a model, they will be able not only to target existing terrorists but to ‘formulate the characteristics of potential terrorists’ too.7 Sariel’s professions may seem hyperbolic. Yet, as already noted in chapter 7, during the present war in Gaza, the IDF have drawn on Lavender, an AI-enabled targeting system of precisely the type which Sariel talks about. Indeed, it seems highly likely that the unit which he has commanded has been involved heavily in this operation and in AI-enabled targeting. For Sariel, human-machine teaming which harnesses the full capability of AI is the future. Sariel is evangelistic, but his position is broadly shared by other professionals. In the last decade, the retired Australian general Mick Ryan has become a prominent figure in debates about the transformation of war. He is more careful and modest. Yet he too sees huge potential in human-machine teaming to enhance military capability. Differently to Sariel, Ryan sees two distinct kinds of team: the human-robot team and the human-AI team. Ryan discusses the introduction of robotic systems at length. He suggests that, in the future, ground forces will include thousands of drones and robots as part of their combat teams. ‘Human-Robot Teams’ will appear, in which these independent systems cooperate with human soldiers. However, Ryan is equally interested in the application of AI by the armed forces. Here, he advocates the formation of ‘Human-AI Teams’: ‘The marriage of humans and AI for strategic and operational planning, as well as for the analysis of future activities, are key applications. This requires analytical focus that is related to, but distinct from, human-robot teaming’.8 AI offers major benefits to the armed forces in terms of situational awareness, intelligence, planning, and targeting: ‘The application of AI as a strategic decision support tool may also address some of the human fragility and bias inherent in strategy development and implementation. AI is not subject to physical issues such as fatigue and can be built to take account of other psychological dimensions of strategy such as cognitive load, risk taking and aversion, and bias. It can assess large amounts of data, challenge long-held human assumptions, and
recognize patterns missed by humans’.9 However, AI is also fragile and prone to error. Consequently, although it should be allowed full autonomy in its areas of competence, an AI agent needs to be integrated and supervised by a human team. In the future, effective command will be determined by the quality of that fusion between AI and military personnel. Academics have also found the concept of the human-machine team fruitful. For instance, in their study of the Australian Army, leading security-studies scholars Alex Neads, David Galbreath, and Theo Farrell have addressed the concept of human-machine teaming in detail. They are agnostic about the concept itself. They do not recommend human-machine teaming as the answer to the question of how the armed forces should apply AI. Rather, they analyse the way the Australian Army understands the concept of human-machine teaming and how it is trying to implement it. Their work is very useful, therefore, in understanding the concept of the humanmachine team. Like military forces of its Western allies, the Australian Army has begun to invest in AI and in remotely controlled and autonomous systems. Yet the Australian Defence Force is small. Consequently, in the face of growing security challenges from China, it is seeking to exploit every avenue of advantage. AI, and above all autonomy, is one obvious area of leverage: ‘Through autonomy the Australian Army hopes to be tactically sized, lightweight and deployable […] but still lethal’.10 The aim is to augment current forces with AI-enabled battle-management systems, coordinating units which consist of human and robotic elements working in concert. Drone swarms, uncrewed ground vehicles, and autonomous weapon platforms will support human combat groups. The potential of these policies is profound: ‘Autonomous systems will become as integral to the delivery of military effect as the soldiers that operate alongside them; in a functional sense, they will cease to be mere tools and become de facto team members in their own right’.11 The Australian Army believes that its human-machine teams will operate according to its doctrine of mission command, in which subordinates are given the latitude to make independent decisions in line with the mission. Presumably, AI agents will also be given the authority to
exercise their initiative, just like human subordinates. Clearly, the Australian Army is not contemplating total military automation. Humans will remain vital in these emergent teams. Yet it is also clear that, in this conception, AI is not simply another form of military technology. It has agency. It will be a genuine teammate, not just a weapon or a piece of equipment. There is much to be gained from the concept of the humanmachine team. Certainly, it avoids some of the hyperbole surrounding AI. It does not assert that AI is about to automate strategy, war, and warfare. Yet even this concept has its flaws. For instance, Baker, Sariel, Ryan, and Neads et al. imply that, because it is able to perform functions no human can, an AI agent will have the exact same status as a military professional in emergent humanmachine teams. For instance, in a military headquarters, where AI is most likely to be used, AI will be as much a member of the headquarters as the commander and the staff officers are. An AI agent will effectively be another staff officer, working alongside its human colleagues. Yet it seems strange to identify a software program as an actor, the equivalent of a staff officer or even a commander. AI-enabled software programs can operate independently of their users; they process data autonomously. In a military headquarters, they perform important functions for the staff officers. The UK’s Enhanced Command and Control Spearhead’s Microworld automates detailed route planning and surveillance. Elbit Systems’ Torch is able to collate and fuse all the data which the IDF uploads, identifying targets and recommending weapons. Project Maven trawled thousands of hours of full-motion video. Lavender generated thousands of targets which IDF intelligence analysts could never have identified. These AI-enabled battle-management systems do things that people find difficult and time-consuming—sometimes impossible. Yet it seems inaccurate to call the Microworld, Torch, Project Maven, or Lavender a full, active member of staff. These AI programs did not define the mission or the overall plan. They did not set the parameters for identifying targets. AI did not determine the collective understanding of the staff or define how staff officers
should collaborate with each other. Military personnel still had to decide how they interpreted and used the results of the AI program. Even if a commander defers to an AI system, as the IDF seem to have deferred to Lavender during the war in Gaza, that is a collective human choice. AI is ultimately just a sophisticated tool for the staff officer. The relationship between an AI agent and a human commander or staff officer is quite different to the relationship between military professionals themselves. The attribution of human-like agency to AI seems to be mistaken definitionally. It misrepresents the actual practice of working with AI. In addition to this misattribution of agency, there is a serious empirical shortcoming with the concept of the human-machine team. When scholars, based on how they see military personnel using AI in the present, use the term ‘human-machine team’, they attribute agency to AI. Since AI performs some autonomous functions, it is accorded the status of an independent agency. Yet this is a narrow, constricted way of looking at AI. It ignores all the investment which has gone into developing AI. The theory of human-machine teaming has forgotten the work of the human programmers who have trained the AI in the first place and the efforts of engineers who continue to maintain it. That indispensable human labour is conveniently excluded from the analysis. In the end, the concept of a human-machine team descends into technological determinism. When the technology is viewed on its own, independent of its proper social and organisational milieu, it is very easy for people to impute agency to it. Fortunately, there are many philosophical correctives to this position. For instance, in one of the most intriguing and celebrated sections of the first chapter of Capital, Karl Marx proposed his theory of commodity fetishism. He argued that the central feature of the capitalist economy is the commodity, an object defined by its market value. The commodity is an apparently mundane entity. Yet it is the mechanism by which surplus value is extracted from the proletariat. Workers are paid only for the objective labour power they put into the commodity, not the eventual market value of the commodity itself. The commodity is thus the vehicle of all exploitation, according to Marx. Since
exploitation is so obvious and egregious, Marx asks why it has been so difficult to identify it. Here, the commodity again plays a crucial role. The commodity not only extracts surplus value from the labourer but also systematically conceals its expropriation because, in a capitalist society, no one sees the labour which constituted the commodity; they see only the commodity. They see the physical product but not the human labour, unseen in a thousand invisible factories where the proletariat toils, which work actually produced the commodity in the first place. In one of Marx’s most famous and lyrical passages, he describes commodity fetishism at work: There, the existence of the things qua commodities, and the value relation between the products of labour which stamps them as commodities, have absolutely no connection with their physical properties and with the material relations arising therefrom. There it is a definite social relation between men, that assumes, in their eyes, the fantastic form of a relation between things.12 The capitalist market economy seems to consist only of an exchange of things: commodities. The price of commodities has nothing to do with humans; it derives from their exchange value—money. The social relations of production and the human labour which created these commodities disappears. The concept of the human-machine team is a form of fetishism. It imbues AI with autonomy only because it forgets the human labour which originally constituted it and which sustains it. It omits the weeks of work by teams of computer scientists, data engineers, and programmers in developing, testing, training, and refining an algorithm or a model. The result is that AI has often been misrepresented. The successes of AI have been imputed to its software programming, algorithms, and neural networks. It is presumed that because each of these entities has a real existence and function, so that AI can process data independently of immediate human intervention, AI itself is autonomous. Like Marx’s commodity, this interpretation systematically ignores the human capital which is vital to the development, maintenance, and refinement of AIs. Yet this human labour is indispensable, prodigious, and constant. It is possible to
see it whenever AI is at work. Indeed, human agency is apparent even when AI is apparently at its most powerful. In chapter 1, I discussed AlphaGo as one of the most celebrated successes of AI in the last decade. When AlphaGo defeated Lee Sedol at the game of Go in March 2016, it was widely regarded as a seminal moment in the development of AI, one that seemed to presage the rise of genuine autonomous artificial intelligence. Yet a closer look at the development of AlphaGo and that game reveals the constitutive role of human expertise at every point. For instance, when AlphaGo played against the European champion, Fan Hui, in 2015, the match exposed weaknesses in the software which the DeepMind engineers resolved only by locking down the code.13 Indeed, when Sedol beat AlphaGo in game 3, DeepMind CEO Demis Hassabis and his chief programmer announced they ‘would need to go back and analyze why [AlphaGo] had made such a lousy move’.14 Hassabis and his colleagues then refined AlphaGo’s program. The eventual victory of AlphaGo demonstrated the human contribution even more. Notably, when Sedol was defeated in the final game, the DeepMind engineers celebrated. In their box above the arena where the match was held, ‘Hassabis punched the air. Team members hugged and high fived’.15 The team was unified in a moment of ecstasy. The scene reveals something very important: AlphaGo did not defeat Sedol on its own. Sedol was defeated by a team of brilliant computer scientists, data engineers, and software engineers who had been working together on this project for years. They programmed AlphaGo and continually refined AlphaGo’s software in the course of its competitions until it was good enough to win. It is wrong to claim that AlphaGo won autonomously. Rather, the team at DeepMind used AlphaGo to defeat Sedol. A human team of engineers, programmers, and scientists used software—a machine— to help them beat another human. AI helped a team of humans who were not adept at Go to beat a champion. Their labour in this process was invisible when AlphaGo eventually triumphed. That work took place months and years before, in Cambridge and at Google’s headquarters in Silicon Valley, as the team collaborated on the
software—discussing, testing, and affirming how to improve the system. AlphaGo shows that collective human labour and intelligence is actually essential to artificial intelligence. The indispensability of human cooperation is equally evident in other sectors in which AI has been applied. In his work on the use of AI to regulate financial services, Anthony Amicelle has discussed the human capital behind banks’ software. It is impossible for banks to monitor all their transactions manually; the volume of transactions is too vast. In order to trace fraud and criminality, banks have therefore employed algorithms which alert them to risky transactions. AI has become crucial to financial regulation. Yet, even though they operate automatically, these algorithms are not truly autonomous. Creating and sustaining them requires human labour. As one banker noted: ‘It’s a lot of work, it took a lot of time because you want to make sure that at the end of the day everything has been mapped’.16 Amicelle observes, ‘This quote sheds light on the invisible and hard work that makes big data surveillance possible’.17 That work is human. It is done by expert groups in the banks, under the direction of management. Financial security is a collective product of these collaborations between programmers themselves and programmers and managers, as they identify what assets need to be protected and how best to protect them. AI has become very potent, but only insofar as humans have designed and redesigned it in the light of their changing needs. We return to groups of human experts , working together on collective endeavours. Several scholars have made a similar observation. For instance, Matthew Johnson and Alonso Vera, two prominent AI researchers, have conducted some very interesting work on human-machine teaming. They claim that even when using the term ‘human-machine team’, it is wrong to think of AI as replacing humans, as ‘technology does not work in isolation from people’.18 AI is brittle; it does not understand why it is trying to achieve a goal.19 Consequently, because it has no conception of its wider purpose, it struggles to take over complete tasks. For Johnson and Vera, full replacement—
total automation—is a very unlikely scenario for AI. The most successful applications of AI are narrower. Johnson and his colleagues participated in an important DARPA robotics challenge. From that experiment, they learnt that full automation is unlikely, concluding, ‘The solution probably won’t be a fully automated robot’. On the contrary, they said, the success of AI is typically determined by ‘how it functions with people’.20 AI will be most effective insofar as it is integrated as a tool into a cohesive team. According to Johnson and his colleagues, one of the prerequisites of employing AI and robotics is identifying the mission, and working out the specific functions, in which AI and robotics might assist humans. The question then is how to build a human team which can integrate that technology to improve its own collective performance.21 Johnson and Vera give us the example of the US Navy’s littoral combat ship. This vessel is heavily automated, such that it requires a crew of only fifty-five—about one-third the crew of a slightly larger ship. However, the training burden has tripled, and the ship is sailed only by experienced senior sailors, at rank E-5 and above. The use of AI on littoral combat ships requires a more skilled crew; ‘the littoral combat ship works because the crews have enabled it through great effort and hard-earned human expertise’.22 The key point is that AI may change what type of human teams emerge, but it will not replace humans; ‘the future lies in its ability to work with people’. In the future, the best AI will not be independent; it will be a tool used by a human team whose members are able to collaborate with each other more efficiently. AI works insofar as it is integrated completely into the human operating team which it is supporting and whose members refine and adapt the software and algorithms as new problems arise. Indeed, if littoral combat ships are any guide, AI will demand even more skilled and capable teams of human operators. Johnson’s work is highly pertinent for understanding how the armed forces will evolve as an organisation as they try to exploit data and use AI to help them. Against the millennial vision of the rise of automated forces on the battlefield—or, indeed, humanmachine teams, in which AI enjoys equal agency to humans—
Johnson’s work implies an ever-closer integration of military professionals, soldiers, staff officers, and commanders around the artificial intelligence which they employ to help them process data and make decisions. Johnson envisages the rise not of the robots but of ever-more-professional military teams, highly adept in the use of computer programs, AI, and data. Just as tech companies have formed ever-closer relationships not with defence ministries but also with operational units themselves, Johnson describes a future in which AI and data will be an integral part of professional military teams. Johnson is not alone. Experts in the tech sector who are familiar with the capabilities of AI emphasise the human dimension of this technology. For instance, when I interviewed an executive in a prominent tech defence company, he described in detail just how important the team was to developing AI. He gave an example of a general knowledge quiz: ‘If you, as an academic, and I had a general knowledge quiz, you might beat me because you have more historical knowledge. But if I took you on with three colleagues, we would outstrip you’. He continued to describe the collective expertise required to develop software: Having the team is crucial. Software is not an egalitarian sport. The engineers are really important. What is a good software engineer? It is like linguistics. There are different levels of competence. Can you speak basic Spanish? Or can you defend me in a Spanish court in a technical legal case? You need highly skilled engineers. But then you also need the right skills. You need someone who can speak Spanish but also someone else who can speak Serbo-Croatian. When you start to get a density of talent, things begin to change. You need to apply that talent to a problem. You can then recognise a solution to the problem.23 Teams of expert engineers have emergent properties. The solutions which they generate exceed the individual skills of each engineer. Successful tech companies have employed a sufficiently large and skilled staff to benefit from the aggregating effects of talent.
The concept of the human-machine team may avoid the hyperbole of much contemporary discussion of AI—but in the end, it, too, incorrectly imputes an agency to AI. It tends towards an argument for partial military automation because it ignores the human labour which produces and sustains AI. Above all, the idea of a human-machine team overlooks new patterns of collaboration in the armed forces as hybrid groups of human experts emerge. It ignores the rise of a military-tech complex and the appearance of military-tech teams. Even though these social and organisational changes are the more important developments, AI is accorded the status of a new military actor in already existing teams. Johnson and colleagues’ analysis of human-machine teaming shows that AIenabled technology in fact demands highly capable and cohesive human teams in order to function at all. The AI program augments rather than substitutes for advanced human teamwork. On this account, humans and human teams are prior and primary. The concept of the human-machine team fails to acknowledge the appearance of new kinds of military teams: ones capable of using AI. Indeed, successful teams will have to evolve and change so that they, like the crews of the US Navy’s littoral combat ships, can harness AI. AI does not have full agency; it is never a true member of these expert teams—it is only ever their subject. Despite its extraordinary capabilities, AI remains a device, a tool, a piece of equipment for the human teams that use it. The Wounding of General Gerasimov I have already described many examples of the actual application of AI by the armed forces. In each case, military personnel—teams— have used AI as a tool to help them plan operations, target opponents, and conduct cyber operations. Yet, in order to understand how expert human teams have exploited AI more fully, it is worth considering a recent example in considerable detail: the wounding of General Valery Gerasimov, the Russian chief of defence staff, on 1 May 2022. This attack on Russia’s most senior military officer was ultimately only a minor one in a long and brutal war, but
it exemplifies some very interesting developments in relation to AIenabled targeting, discussed in chapter 6. Also, it highlights the enduring importance of the Special Operations Forces in the adoption of AI by the armed forces. By investigating this case, we may better understand the character of the groups of human experts which are using AI in military headquarters and apprehend how they are applying AI to operational problems. The Russo-Ukraine War began on 24 February 2022, when Russian forces invaded Ukraine in an unprovoked act of aggression. It has been brutal and bloody, characterised by a series of gruelling sieges: Kyiv, Kharkiv, Mariupol, Severodonetsk, and Bakhmut. Much of the fighting has been reminiscent of the First World War and Stalingrad. Over a million Russian and Ukrainian soldiers have been killed or wounded. There is no immediate end in sight. This war is without question the richest empirical source for understanding contemporary high-intensity warfare. Almost every engagement and certainly every major battle in it is pertinent to the study of war. AI, which has played an important role in this war, has been the focus of much attention and discussion. On 1 May 2022, according to media reports, General Gerasimov, President Putin’s closest military aide and the most senior Russian general, was visiting a Russian army headquarters in the tiny village of Zabavne, just north of Izyum. It seemed an anonymous, invisible location. Yet that very same day, the headquarters was devastated by Ukrainian artillery, probably by a US-provided HIMARS (highmobility artillery rocket system). Major General Andrei Simonov, who was the head of the Russian Army’s electronic warfare division, and about twenty staff officers were killed. Gerasimov himself was reportedly wounded in the thigh by shrapnel. How did the Ukrainian armed forces mount such a precise strike against a command post? More to the point, it seems almost inconceivable that the location and timing of the strike were a coincidence. It seems more likely that the attack was a deliberate attempt on Gerasimov’s life. The Ukrainians, it seems, were capable of precise, dynamic targeting at long ranges. They could not only
find and strike an important command post but could do so when a specific individual was inside it. The strike on Zabavne seems an improbable starting point, but it is a useful way of mapping the appearance of expert teams of military personnel and civilian technicians who are applying AI to military operations. The Zabavne strike and the possible wounding of Gerasimov—almost certainly as a result of a targeting process orchestrated by the US in support of the Ukrainian armed forces—is a useful methodological device, therefore. Much of this targeting process remains highly classified, but enough evidence has now filtered into the public domain for us to be able to reconstruct some elements of it. In order to explain the strike, it is necessary to analyse the command structure which enabled it. This analysis takes us a long way from Zabavne. Following the Russian invasion of Ukraine, XVIII Airborne Corps, under Lieutenant General Christopher Donahue, assumed command of the Security Assistance Group—Ukraine (SAG-U), based in Wiesbaden, Germany. The command was known as Task Force Dragon, after the emblem of XVIII Airborne Corps. SAG-U had coordinated the training of Ukrainian forces since 2015. However, once the war started, the role of the assistance group expanded dramatically. The command was no longer merely administering training for the Ukrainians; it began to provide them with essential command support. Some of this support involved planning and delivering logistics. However, SAG-U also provided immediate operational support to Ukrainian forces, including, crucially, targeting. Because the role was so demanding and complex, Donahue deployed XVIII Airborne Corps’ forward headquarters to the Security Assistance Group base in Wiesbaden. Its main headquarters, providing support, remained at Fort Liberty (formerly Fort Bragg), North Carolina. At the same time, 82nd Airborne Division, subordinated to Donahue and co-located at Fort Liberty, deployed its forward headquarters to Rzeszów, in Poland. XVIII Airborne Corps had begun to exploit AI well before its deployment to Wiesbaden. From 2019 to 2022, the corps had been commanded by Lieutenant General Erik Kurilla. Kurilla has become a
storied figure in the US Army with a stellar career as a commander in the Rangers, 82nd Airborne Division, and the US Special Operations Forces. He became famous as a robust and fearless battalion commander in Mosul in 2005. As the journalist Michael Yon wrote: ‘The enemy hated Erik so much they put a bounty on him. The enemy hated Deuce Four [Kurilla’s battalion]. I loved them. They were incredibly aggressive and so there was constant contact with the enemy’.24 In one of many engagements in that tour, Kurilla was involved in a close fight with insurgents which journalists, including Yon, witnessed. He was shot three times; the photograph that Yon published of Kurilla lying on the ground, shielded by his deputy, became famous. Yet Kurilla continued to engage enemy soldiers and command his troops, earning him the Bronze Star for valour.25 From 2012 to 2014, he served as assistant commanding general of Joint Special Operations Command (JSOC). As a result of his work with US Special Operations Forces, Kurilla had been exposed to the way in which they employed AI to identify targets. Although he did not serve in the JSOC in Baghdad, he was aware of their work. However, while deployed to Afghanistan as a brigadier general in the Special Operations Forces, he had employed some of the innovations introduced under Project Maven. He had seen the potential of AI for data exploitation in Afghanistan.26 There, he had worked closely with Palantir and been impressed by their software and their technicians. Indeed, following his experiences in Afghanistan, he had told senior US officers: ‘I have seen what it can do. We are never going back’.27 He had then sought to implement AI to analyse data when he commanded 82nd Airborne Division and XVIII Airborne Corps. Kurilla commanded XVIII Airborne Corps until 2022, when he was succeeded by Donahue as the corps deployed to Wiesbaden. Donahue was well-positioned to extend Kurilla’s work on AI data processing. Like Kurilla, he had served in the Special Operations Forces—in Delta Force, in particular—for much of his career. He had fought in Iraq as part of McChrystal’s Joint Special Operations Forces and in Afghanistan with Delta Force, eventually commanding the unit in the war against ISIS. During his time in JSOC in Baghdad in 2004,
Donahue saw the way in which McChrystal exploited data and AI to conduct military operations and how he partnered with civilian experts, such as Anshu Roy.28 He also held a job in the Department of Defense when Project Maven was running in 2017 and 2018. There, he had learned all he could from US Marine Colonel Drew Cukor, who was administering that project.29 Consequently, by the time he assumed command of XVIII Airborne Corps, he, like Kurilla before him, had already seen AI-enabled operations at work. Building on Kurilla’s work, he had applied these methods of AIenabled targeting to operations in Afghanistan as a special forces operator and then as commander of 82nd Airborne Division. He was one of the last US soldiers off the ground in Kabul in August 2021 (at the end of the evacuation). Kurilla and Donahue wanted XVIII Airborne Corps to harness AI so that they could exploit the potential of the massive amount of data available to them. They wanted to be able to target more quickly, more accurately, and deeper in the enemy’s battlespace. However, to conduct AI-enabled targeting, they had to make some radical organisational changes to their headquarters. In 2022, Donahue appointed Jared Summers, a well-known Silicon Valley entrepreneur and executive, to the role of chief technology officer. The appointment of Summers was critical. Between 2016 and 2021, Summers had acted as Exxon’s chief technology officer and its chief data officer. Donahue hired him to ensure that the corps became a ‘data-centric’ organisation. Donahue created a data cell, with civilian and military data experts, in the corps. In Wiesbaden, he was able to recruit a large number of technicians and create an environment in which they could work safely. Indeed, one staff officer noted that the headquarters was very unusual because ‘there were lots of civilian contractors wandering around’. Summers was responsible for orchestrating the work of private contractors within the headquarters and especially within the data cell. Summers served throughout 2022 with the corps for the Russo-Ukraine War. He left the role in January 2023, after the corps had returned from Wiesbaden. For his work with Task Force Dragon, he received the Meritorious Civilian Service Award. On Summers’s retirement from the post, Donahue said of
him, ‘Jared was exactly the right person to serve as our first Chief Technology Officer—and he has really laid the groundwork for organizations across the Department of Defense to emulate’.30 Above all, Summers was able to integrate civilian technicians into the headquarters. Donahue’s own leadership was also crucial in integrating military experts and civilian technicians into an operational military headquarters. McChrystal has emphasised that, when he was commander of JSOC in Baghdad, he needed to adopt a collaborative leadership style. He could not be authoritarian or narrowly military. He consciously sought to be inclusive and supportive so that all his staff, whether civilian or military, could contribute. In his own words, ‘The heroic “hands-on” leader whose personal competence and force of will dominated battlefields and boardrooms for generations had been overwhelmed by accelerating speed, swelling complexity, and interdependence’.31 McChrystal recognised he was at the heart of a politico-military-intelligence network, every member of which might be useful. He had to form relations with civilian data scientists such as Anshu Roy and other experts. Donahue seems to have adopted a similar method in Task Force Dragon. He was widely recognised as a talented, inclusive officer before his appointment as the commander of XVIII Airborne Corps and the Security Assistance Group. In the course of my research, several interviewees described interactions with Donahue which had impressed them. For instance, one interlocutor described how, when Donahue was commander of 82nd Airborne, he had participated in an important exercise called Warfighter, in which formations are formally assessed for readiness. During the exercise, he saw a group of contractors, supporting the division, in the parking lot. He asked them who they were. When he realised that one of the group was from the NSCAI, Donahue invited that individual to accompany a group from industry which was attending the exercise. As that individual noted, Donahue was actively ‘grabbing human capital’ to improve his command, whatever their background and affiliations.32 Other personnel recorded a similar inclusivity. For instance, in 2022, XVIII Airborne Corps had staged war games to predict the
outcome of Ukrainian operations. At the end of one of the working days, when there were only a few people still in the room, finishing the final pieces of work, Donahue looked in quietly without announcing his presence. He tapped the elbow of an intelligence officer, a major, who was running the war game, and asked him whether he would provide him with personal feedback from the war game after it was over. The intelligence officer was amazed that a three-star general would assume such a modest and informal manner.33 Donahue was certainly a demanding commander; he was a highly capable soldier, with a keen military apprehension. Yet he actively developed an inclusive method of leadership so that he was able to interact with all his staff—both military and civilian—to ensure that he had the best possible information. He fostered collaboration among his staff, facilitating the formation of the cohesive teams which he needed in order to make good decisions. How did AI help XVIII Airborne Corps to target Russian forces and, very probably, to wound General Gerasimov on 1 May 2022? During this war, Ukraine and its partners, like XVIII Airborne Corps, have systematically exploited data from open sources, decrypted mobile-phone and radio messages, and satellite images. Opensource data has been especially important. From the start of the war, Ukrainian civilians had taken many photographs or texted descriptions of Russian troops and equipment, and been encouraged to upload their data onto a Ukrainian government application. This information has significantly helped the Ukrainian armed forces to target Russian troops, especially in 2022 during the first phase of the invasion. The result has been an explosion in data—vast quantities of imagery tracking Russian troop movements. Ukraine and its allies have also used the imagery posted by Russian civilians and soldiers to identify Russian targets. For instance, the 31 December 2022 strike on the barracks in Makiivka, in which the Ukrainians claimed they killed six hundred Russian recruits—the Russians admitted about eighty—was made possible by Russian soldiers posting pictures on social media. US software monitoring open-source traffic seems to have geo-located the imagery and was able to recommend
the strike. AI has been used to sift and trawl the mass of opensource data in order to identify targets. Satellite imagery has, of course, been crucial too. The US has concentrated many satellites over the theatre to try and develop saturation coverage; Elon Musk’s Starlink constellation has helped here. XVIII Airborne Corps was able to collect, fuse, and analyse data from different sources to develop very accurate targets. On 1 May 2022, it seems that the corps played a role in triangulating a range of sources to geo-locate Gerasimov with complete precision. Task Force Dragon has played a critical role in developing an AI-enabled system of targeting for the Ukrainians, but, even with a chief technology officer and a data cell, by itself the corps could not process all the data it required; it did not have the expertise to curate the data or to write the algorithms to be able to process it. Consequently, the corps—and the US military more widely—relied on a partnership with the civilian defence tech company Palantir Technologies to develop and refine targeting software. In chapter 5 it was described how, from 2009, Palantir began to sell software directly to US military units on operations in Iraq and Afghanistan. The Special Operations Forces were the first to buy Palantir software, in 2010.34 Having developed a capable software which both Kurilla and Donahue had seen in action, XVIII Airborne Corps hired Palantir to provide technical services under their commands. By 2022, Palantir had considerably enhanced the corps’ targeting process. By then, it had adapted the software and algorithms it refined during the campaign against ISIS for the war in Ukraine. Its Metaconstellation software was capable of curating, fusing, and analysing diverse data sources to identify patterns and targets. In a Washington Post article, David Ignatius has described this process in as much detail as possible, given the classification of the targeting process. He connected decisions in Ukraine with activities at a NATO headquarters—presumably Task Force Dragon—outside Ukraine: By applying artificial intelligence to analyze sensor data, NATO advisers outside Ukraine can quickly answer the essential questions
of combat: Where are allied forces? Where is the enemy? Which weapons will be most effective against enemy positions? They can then deliver precise enemy location information to Ukrainian commanders in the field […] The “kill chain” that I saw demonstrated in Kyiv is replicated on a vast scale by Ukraine’s NATO partners from a command post outside the country [e.g. XVIII Airborne Corps]. The system is built around the same software platform developed by Palantir that I saw in Kyiv, which can allow the United States and its allies to share information from diverse sources—ranging from commercial satellite imagery to the West’s most secret intelligence tools […] […] The system I saw in Kyiv uses a limited array of sensors and AI tools, some developed by Ukraine, partly because of classification limits. The bigger, outside system can process highly classified data securely, with cyber protections and restricted access, then feed enemy location data to Ukraine for action.35 Ignatius was careful and oblique in his article. However, it is possible to reconstruct the process from his descriptions of targeting processes inside and outside Ukraine. Palantir has helped XVIII Airborne Corps to process a mass of open-source, encrypted, and satellite data in order to identify the precise location of Russian headquarters and other targets a long way from the front line. In February 2023, Alex Karp, Palantir’s CEO, described the process and the competitive advantage which AI and data exploitation generated: We have seen artificial intelligence transformed from coddled experiments in the research lab to resilient models that provide a real advantage on the battlefield. The ingenuity and bravery of the Ukrainian people is now showing the world how advanced technology can be harnessed to provide modern militaries with a new advantage: a decision advantage. The ability to outperform your adversary in terms of the scale and speed with which you can make decisions informed by data is a key part of how Ukrainians are beating back the Russian invaders, who are starved of data and analytically sluggish. One year in, we now see that those with the
best trained artificial intelligence models can outmaneuver adversaries who lack them by using data collection, decision-making, and importantly, human action.36 Karp has every interest in promoting Palantir and its vision of data-enabled and AI-centric warfare. Yet the evidence from a variety of sources shows that Palantir—and other companies—were actively involved in this process of outmanoeuvring Russia, with AI playing an important role. However, in order to deliver this AI-enabled capability, to develop the software, and to process data at scale and speed, many Palantir contractors deployed forward into the headquarters at Wiesbaden. To be effective, they could not just design software in the abstract and transfer it to the military users. They had to work closely with Donahue and his staff on a daily basis, refining the software and AI programs as the mission changed and the data matured. As one Palantir executive noted: ‘You need to be as close to the environment as possible. You need to understand what are the constraints. You need to quickly iterate and push experimentation out and deliver updates. You need continuous development of your software.’37 The result of this collaboration is obvious; the Ukrainian forces have been able to kill dozens of Russian generals and destroy scores of command posts. It is not just the accuracy of the fire. The targeting process which Donahue constructed at XVIII Airborne Corps was extremely rapid. For instance, one staff officer who had worked in the targeting cell for the Battle of Mosul during Operation Inherent Resolve was surprised by the difference: ‘I couldn’t understand how quickly they are targeting the Russians and firing HIMARS. In Mosul, it took ages’.38 The reason for the difference, at one level, was AI. However, in order for AI to function technically, a far more important organisational reformation had taken place. Civilian experts from Palantir and other tech companies had deployed forward and had worked alongside their military colleagues on immediate operational problems inside the data cell. This collective human expertise was decisive.
Task Force Dragon Commentators and scholars have described the rise of AI as humanmachine teaming. They would presumably see Task Force Dragon as an example of a human-machine team at work. Yet this is a misconception. The AI model which Task Force Dragon employed was very sophisticated. It was programmed by technicians from Palantir and other companies to process data automatically in line with its algorithms. It would seem strange to describe that model as a team member, in the way that General Donahue, Jared Summers, staff officers, or technicians were members of the task force’s headquarters. The headquarters relied on the model, but the software, vital though it was to Task Force Dragon, could not be accorded the same status as a human staff officer—or technician. The Palantir model neither defined the corps’ mission in Ukraine nor set the operational parameters in which particular targets were invested with importance; it did not determine how staff officers worked with each other. The results of the software were interpreted and acted on by the staff officers as they supported the commander to make decisions. The staff officers applied the software to the problems which the commander had set. Even when Donahue and his staff officers in Task Force Dragon drew on Palantir algorithms to process the data they were seeing, they affirmed the result. The AI just identified targets for them more quickly and more accurately than they could have on their own. It was an advanced tool. Task Force Dragon was not so much a human-machine team as an expert human ensemble consisting of military and civilian professionals working together to solve problems. This team was unusual; it consisted of military and civilian personnel on operations together, as equals. Task Force Dragon’s targeting depended on their cooperation so that AI algorithms could process data efficiently. The armed forces are beginning to apply AI to military operations. Many scholars have described the emergence of humanmachine teams in which AI will have equal status. In fact, the military application of AI involves local working partnerships between expert tech-company employees and military personnel. AI is not a
discrete, autonomous platform which works on its own. It is a service which requires continual engineering. Consequently, in order to exploit AI on operations, active military units have had to integrate civilian technicians into their operational headquarters. Here, we begin to see something striking—not the emergence of new human-machine teams, but rather the formation of civilianmilitary ensembles; commanders, staff officers, and civilian technicians, contracted into the headquarters, working on operational problems together. The technicians curate and manage the data and revise and rewrite the code and algorithms, on close instruction and advice from commanders and staff officers. The commanders and staff officers integrate the results from the AI algorithms to make decisions about the military campaign. We are seeing the appearance of new concrete practices of cooperation within headquarters to harness the potential of AI as a resource and a tool. This is not human-machine teaming but military-tech teaming.
10 War at the Speed of Light It is widely believed that we are on the cusp of another military revolution—that AI will transform warfare just as gunpowder or airpower did. The implications of AI for the armed forces, for war, and for warfare are profound. A common presumption is that AI is not just about to alter how the armed forces organise themselves, and the way the armed forces fight, but about to automate war itself. AI will take over strategy; it will colonise military decisionmaking, displacing political leaders and human commanders. ‘Warbots’ will appear. Programmed with massive amounts of data, and controlled by sophisticated algorithms, drone swarms and killer robots will dominate the battlespace. War will become a competition not between people but between AI and lethal autonomous systems. It is a mesmerising vision of the future. It is easy to be bewitched by the potential of AI. Indeed, strategists and military scientists have regularly exaggerated the revolutionary potential of new technologies. In the 1920s and 1930s, the airpower theorists Giulio Douhet, Billy Mitchell, and Hugh Trenchard believed that the strategic bomber was about to revolutionize warfare. In 1921, Douhet published his highly influential work, Command of the Air, in which he asserted that the destructive potential of airpower was limitless. Since aeroplanes could fly over mountains, rivers, forests, walls, and fortifications, he said, ‘aerial warfare admits of no defence’.1 Because an enemy’s cities were defenceless before it, airpower could win wars on its own.
Douhet’s views were utopian. Yet many contemporaries and practitioners, including Arthur Harris and Curtis LeMay, also believed that strategic bombing could be decisive. In the event, strategic bombing in the Second World War was never decisive. For instance, up to August 1941, the RAF’s attacks on German cities were ineffective. The Butt Report of 1941 recorded that only one in five of the RAF’s bombs landed within five miles of its target.2 After a huge investment of resources, US and British strategic bombing in Europe became effective and, by 1944, very destructive. Yet it was never strategically decisive. Airpower did not remotely win the war on its own. On the contrary, one of the most important effects of the Allied air campaign was the degradation of the Luftwaffe in the skies over Germany, not the collapse of the German economy and state. As a result of that degradation, Allied forces enjoyed air superiority in their land campaigns from 1943 onwards. In this way, the Allied air campaign achieved a major operational success. Airpower had become a vital part of military operations, but its effects were far less decisive than prophesied by Douhet and his airpower colleagues. The Digitised Military Today’s proponents of AI seem to commit a similar fallacy. They exaggerate the capabilities of AI. They profess that AI will transform the character—even the nature—of war, as AI supersedes human commanders and combatants. In this book I have dispelled the idea that AI is about to automate war. Political leaders and military commanders will still need to make decisions; they will design strategies and execute operations. They will need to interpret complex, delicate situations and develop plans to address them. To achieve their goals, they will have to negotiate with opponents, allies, and subordinates. Human combatants will remain vital. Lethal autonomous weapons, drone swarms, and armed robots will become ever-more-useful parts of existing armouries, increasing the capability of human combat teams that will exploit their potential. Yet the capabilities of these systems are often overstated. Moreover,
even when these systems do provide a significant additional capability, opponents will adapt quickly; they will develop countertechniques or adopt the same technology themselves. In the present Russo-Ukraine War, small, first person–view drones have proliferated on both sides. No one moves on the battlefield without first deploying drones to identify enemy positions. Yet, precisely because the proliferation is equal on both sides, the opponents have neutralised each other, and the ultimate tactical effect of drones is limited. The use of drones has accentuated the positional, attritional character of contemporary land warfare, rather than transformed it. Although AI will not automate war, it would be quite wrong to underplay the significance of AI. In the last twenty years, and especially in the last decade, all the leading military powers have recognised the potential of AI. They have sought to harness AI for military operations. Remotely controlled and autonomous systems will continue to proliferate. Yet autonomous drone swarms and robots have not been the focus of these efforts by the armed forces. Rather than as a means of automating war through computerised decision-making and autonomous lethal weapons, militaries have sought to exploit AI as a means of processing data. AI has been employed primarily for military intelligence, conceived in the broadest way; it has improved situational awareness and understanding. AI algorithms, trained on large datasets, have been used to recognise patterns and signatures. AI has been used to mine the vast resources of cyberspace. It has allowed the armed forces to map the cyber domain so that they can operate more effectively in physical reality. Consequently, AI has been employed for three principal military functions: planning, targeting, and cyber operations. AI has automated some of the more mechanistic elements in the planning of operations. For instance, it is adept at plotting routes and developing courses of action. It has played an important role in collating data for battle-management systems. Targeting may be the area in which AI makes its most important contribution. Because a trained model can process a vast quantity of information, connecting data from different feeds, it is able to plot
opponents across the battlespace to a depth, and with a fidelity and speed, hitherto impossible. In the Russo-Ukraine War, with direct assistance from the US, Ukrainian forces have been able to fuse data from satellites, open sources, and encrypted signals intelligence to plot the precise location of Russian command posts and logistics hubs hundreds of miles behind the front line. AI models are vital to all kinds of cyber operations, as they can help the armed forces to hack into an enemy’s systems—to carry out cyber sabotage, espionage, and subversion—and to protect their own systems from penetration. AI software can be trained to identify infiltrations quickly. AI has also been increasingly employed for information and psychological operations. States, non-state actors, and criminal organisations have trained and programmed software which can multiply existing messaging or which can generate new content, including deepfakes, automatically. The application of AI to these activities is very significant. AI has generated new techniques and opened up new possibilities in these areas. In order to remain competitive, the armed forces have had to embrace AI. Yet this development does not amount to a complete rupture in the character of war or warfare. Many military scientists have defined AI as a discrete technology. They identify a suite of specific capabilities which, as a technology, AI enacts independently. In this way, AI is accorded agency; it is an actor. It will influence the armed forces. It will transform the armed forces and military operations; it will become a team member, maybe even a commander. This is not a helpful way of understanding AI. AI may be an advanced technology, but it is better understood as a complex suite of capabilities; it requires data, computing power, and expertise. It is a service, rather than a platform. Consequently, AI does not function on its own. Its development and application require constant human effort. Rather than focusing on AI as an exquisite and discrete technology, it is more fruitful to locate it in its social and organisational context. In order to leverage AI, the armed forces have, therefore, had to forge an alliance with Silicon Valley and the tech sector. Only the tech sector has the data, the expertise, the software, the computing power, and the capital, provided by
venture capitalists, to help the armed forces develop and exploit AI. Consequently, over the last decade, ministries of defence have reformed the regulatory environment to facilitate cooperation between the tech and defence sectors. Defence ministries have signed contracts with Microsoft, Google, Amazon Web Services, and other tech companies to provide them the services they require. New alliances have appeared among these commercial companies, academia, governments, defence ministries, and the armed forces. A new ecology is appearing: a military-tech complex, which is altering —and, in some cases, displacing—the old military-industrial complex. In the last ten years, tech primes, like Google and Amazon, and tech defence primes, like Palantir and Anduril, have cultivated partnerships with the armed forces, especially in the US and the UK. They work closely with these forces to process data and to support operations. Unlike the arms companies of the twentieth century, however, tech companies have not developed software platforms independently of the armed forces and then simply sold these products to them. In order to apply AI to specific operational problems, data engineers and programmers require immediate access to the available data. They need to be as close to the specific, concrete problem as possible. The armed forces have, therefore, not merely outsourced their AI and software problems to private tech companies. They have integrated data scientists and programmers into operational headquarters, where they can work together with commanders and staff officers. The alliance between the armed forces and the tech sector is not just an abstract regulatory arrangement. It is realised in the emergence of new expert professional military-tech teams actively cooperating on specific missions. Hybrid professional groupings have appeared inside operational headquarters. Commentators have become used to the concept of the humanmachine team to describe the incorporation of AI into military operations. For them, a novel cybernetic organism is appearing, a centaur: part human, part software. Yet the algorithms and AIenabled software used by military forces in the last decade constitute not so much the rise of a human-machine team as the
birth of new human teams. These teams span the defence and tech sectors, consisting of military personnel and civilian technicians who, together, are developing new operational practices. AI is highly capable. Once programmed, it processes data automatically. Yet we are wrong to attribute the status of an agent to AI. AI is not really a member of these new hybrid teams. It neither defines missions nor determines plans in active collaboration with its fellow team members. It does not negotiate how it might cooperate with its teammates or decide how they should work together. Staff officers and commanders collectively determine those issues. AI is just a very sophisticated tool for these human teams, one which is programmed, trained, and refined to process specific bodies of identified data in order to answer particular questions set by human commanders. AI has been a technical means by which novel human ensembles have tried to cooperate with each other more closely, fusing their different expertise, so that the armed forces can be more effective and efficient. Using AI, these teams have planned more efficiently; they have targeted more quickly and precisely; they have been able to spy on, sabotage, and subvert their opponents in digital space. The human actors, bringing special expertise and resources from the military and tech sectors, are constitutive here, not the AI. The organisational transformation of the armed forces—the rise of a military-tech complex—may seem rather banal in comparison with the utopia of military automation. It is not as arresting as the image of supercomputers controlling drone swarms as they swoop through the ruins of cities. Yet, in fact, the alliance of private tech companies and public-sector military forces is a profound historic development. The economic and political structure of the armed forces is changing; the relationship between the armed forces and the private sector, and the relationship between military society and civil society, is evolving. A reformation of civil-military relations is occurring. In the last thirty years, the armed forces, especially in the US and the UK, have outsourced some of their functions. Private military and security companies have emerged.3 The armed forces have remained
under exclusive state ownership and authority, but since the 1990s they have tended to be supported ever more closely by the private sector. Commonly, they have made use of private military and security companies. In the US and the UK, these companies are not traditional mercenary organisations; they generally offer supporting services. For example, Kellogg Brown & Root have provided logistics, and other companies have delivered technical maintenance services. In some cases, the employees of companies like Blackwater and Aegis have supplied specialist security contractors to protect sensitive facilities or to provide close-protection teams. These companies have rarely supplied combat forces for actual military operations in which they might be described as mercenaries. The Russian organisation the Wagner Group, formerly directed by Yevgeny Prigozhin, is the most notorious organisation that has done so. With AI, a new settlement is emerging. A military-tech complex is appearing. The relationship between the armed forces and tech companies is not an extension of traditional outsourcing. Unlike traditional private military and security companies, tech companies are not operating on the periphery in support of a professional state military core which conducts operations on its own. Tech companies have not just signed contracts with the armed forces to deliver logistics, servicing, and security functions. They have formed longterm partnerships with the armed forces, developing their software alongside warfighters. They have been integrated into defence ministries and the armed forces. At the same time, data engineers and other employees of tech companies have been incorporated into military headquarters and operational units themselves so that their capital and skills can be applied effectively to specific problems. Tech companies are operating in the very core of the armed forces, participating in missions themselves. The famous sociologist Max Weber defined a ‘state’ as a polity which had monopolised the means of legitimate physical force.4 By the middle of the twentieth century, states completely owned and paid for their armed forces. They monopolised all military capabilities. They also supported a private indigenous defence
industry to supply their forces. The state had effectively integrated the whole of the defence sector. Today, the situation has reversed. The emergence of a militarytech complex constitutes a significant revision of the Weberian settlement. Private tech companies own satellite communications, data, computing, and, of course, AI, all of which are vital to national security; states no longer own increasingly significant military capabilities. In the case of AI, the private tech sector, not the state, has provided immediate operational capabilities. A new civil-military settlement is appearing, in which private tech is operating inside the military, as part of the state forces. This level of privatization has not been seen since the nineteenth century, when the East India Company, the Hudson Bay Company, and other commercial companies pursued their own colonial policies in broad alignment with those of their host state. Indeed, it might be argued that the emergence of a military-tech complex echoes the early modern period, when European monarchies relied on a host of mercenaries and privateers to pursue their foreign policies and relied on private companies to supply them, in a system that historians have described as the ‘fiscal-military state’.5 The military-tech complex is major reformation of the armed forces and the defence sector. It reconstitutes the armed forces, how they operate, and their relations to the civilian government and to civil society. A new military regime is appearing. The tech sector has delivered indispensable capabilities to the armed forces, enabling the processing of data on a massive scale and at incredible speed. Yet the integration of private tech companies into the armed forces, and into military operations themselves, carries significant risks. In the early modern period, monarchs and dukes often relied on mercenaries, privateers, private banks, and manufacturers to support their military ventures. Philip II relied on the Fuggers for capital.6 Niccolò Machiavelli deplored the mercenaries on whom Italian city-states, including the Republic of Florence, relied: ‘Mercenaries are disunited, thirsty for power, undisciplined, and disloyal; they are brave among their friends and cowards before the
enemy; they have no fear of God, they do not keep faith with their fellow men; they avoid defeat just so long as they avoid battle; in peacetime you are despoiled by them, and in wartime by the enemy’.7 To Machiavelli, mercenaries lacked all civic virtue and patriotism. States relied on private companies to prosecute war; but, of course, when it was convenient, mercenaries followed their own interests. The emerging military-tech complex has begun to introduce private political interests into state military operations once again. The armed forces increasingly rely on tech primes for essential services. Yet those tech primes are so powerful that it is now almost impossible for them not to exert some influence on the course of a campaign, if only passively. Because tech companies are involved in operations themselves, they exert an influence which the traditional defence industries, building only platforms, rarely did. The Russo-Ukraine War has exemplified the transformation. In February 2022, in the immediate aftermath of the invasion, many tech primes, including Google, Microsoft, and Amazon Web Services, supported the Ukrainian government. As discussed in chapter 8, Microsoft played a crucial role in the cyber defence of Ukraine; its support, pledged till the end of 2023, was worth $400 million.8 Google offered its services for free. Amazon delivered Snowball devices to Ukraine three days after the invasion, which allowed for the transfer of Ukrainian governmental data to the Amazon cloud. The Slovakian company ESET was also very important in protecting Ukrainian interests; ESET helped Ukrainian teams to repel the Industroyer2 malware attack.9 Elon Musk’s Starlink communications system played perhaps the most vital role. Starlink is a secure network, supported by Musk’s own private constellation of over six thousand satellites. On any given day in 2022, about forty Starlink satellites were orbiting above Kherson, Ukraine. Starlink has been particularly useful in Ukraine because its satellites orbit at an unusually low altitude of about 550 kilometres. Consequently, Starlink satellites traverse the sky quickly, providing coverage for only a short time. However, because of their low altitude, the latency period of Starlink is short; its satellites can
transmit data to earth quickly—and because the coverage of each satellite is small, the bandwidth necessary for each signal is relatively modest. Starlink has gathered almost immediate data about the battlefield. At the beginning of the war, Ukraine appealed personally to Musk for assistance. Mykhailo Fedorov, the Ukrainian vice prime minister and minister of digital transformation, urged Musk publicly, on Twitter, to provide connectivity. Two days later, five hundred Starlink terminals arrived in Ukraine. In February 2022, Musk granted the Ukrainian government and military the use of Starlink— initially for free. Musk participated in a personal video call with President Zelensky to discuss the provision of more systems. On 2 March 2022, another two thousand terminals arrived. Within days of the invasion, the company was providing communications for Ukrainian special operations brigades and connecting them with the US Joint Special Operations Command (JSOC). By July, Starlink had provided fifteen thousand terminals. The Ukrainians fully admitted their dependence on Musk: ‘Without Starlink, we would have been losing the war’, a Ukrainian officer said.10 Starlink was crucial to Ukraine’s war effort. It allowed the government and the armed forces to communicate securely. Consequently, the battlemanagement systems Delta and Kropyva, which the Ukrainians had developed, could operate effectively. Delta depended on Starlink: ‘Starlink has become the linchpin of what military types call C4ISR (command, control, communications, computers, intelligence, surveillance and reconnaissance)’.11 Thanks to Starlink, Delta has been able to upload, collate, and share all the data from Ukrainian sensors for Ukrainian commanders and their forces. Without Starlink, the defence of Ukraine might have been impossible. The cost to Starlink was considerable, though. Musk had provided 2,000 terminals for free, 300 at a discount, and waived monthly service fees for about 5,500 more. Starlink’s total contribution to the Ukrainian war effort in 2022 was $80 million. The historian Niall Ferguson, who raised $5 million for Starlink, said, ‘I cannot overstate the importance of the role Starlink has played in keeping the communications of the Ukrainian government from being taken out
by the Russians’.12 Eventually, in the autumn of 2022, Musk’s company SpaceX persuaded the Pentagon that it would no longer provide free Starlink services for military purposes. It argued that the Pentagon should pay $145 million for its services. There was a backlash, and Musk agreed to pay for free terminals already in Ukraine. However, the Pentagon did start to pay SpaceX for its services in Ukraine. Musk had provided a vital military capability, initially for free, and eventually on a (discounted) contractual basis. However, this was not the end of the controversy. In September 2023, Walter Isaacson published a biography of Musk in which it was described how, in September 2022, Musk had temporarily withdrawn Starlink services from Ukraine. The Russian ambassador to the US, Anatoly Antonov, had personally warned Musk that a Ukrainian attack on Crimea would be a red line leading to nuclear war.13 Musk was concerned that the Ukrainians were planning to launch a surprise attack against Russian ships in Sevastopol, using uncrewed surface vessels packed with explosives. He believed that such a move was potentially escalatory. Fedorov tried to assure Musk that it was an entirely Ukrainian operation: ‘We made the sea drones, they can destroy any cruiser or submarine. I did not share this information with anyone. I just wanted you—the person who is changing the world through technology—to know this’.14 Musk discussed the issue with President Biden’s national security adviser, Jake Sullivan, and with General Mark Milley, the chair of the Joint Chiefs of Staff.15 He also spoke to Antonov, affirming that Starlink would be used for defensive purposes only. Indeed, inside the company itself, there was significant concern. For instance, Gwynne Shotwell, the president of SpaceX, believed that the company should not subsidise Ukrainian military operations at all—it should only participate in humanitarian missions. Musk himself was sceptical: ‘Starlink was not meant to be involved in wars. It was so people can watch Netflix and chill and get online for school and do good peaceful things, not drone strikes’.16 Musk conversed intimately with Fedorov about the issue, advising him, ‘Once Russia is fully mobilised, they will destroy all
infrastructure throughout Ukraine’. He urged Fedorov to avoid World War Three: ‘Seek peace while you have the upper hand’.17 In the end, Musk blocked Ukrainians’ access to Starlink so that they could not mount the attack on the Crimea. His action deeply concerned the Pentagon. Air Force Secretary Frank Kendall criticised Musk’s decision: ‘If we’re going to rely upon commercial architectures or commercial systems for operational use, then we have to have some assurances that they’re going to be available. We have to have that. Otherwise they are a convenience and maybe an economy in peacetime, but they’re not something we can rely upon in wartime’.18 The fact that the operation was impossible without Starlink demonstrated just how important Musk’s company was to the Ukrainian war effort. Musk’s critics were right to berate him. Since 2017, Silicon Valley has realigned itself with national defence policy, and, in theory, Musk should have followed the White House’s strategy. Yet, in reality, companies of Starlink’s importance are not simply agents of the US state; they are actors in their own right. Musk was not obliged to support the Ukrainian war effort at all. Yet, once he did support that effort, it was inevitable that he would influence policy. In the course of the debates about the Crimean strike, it is noticeable that Musk was in direct communications with the key political and military leaders in the US, Ukraine, and Russia. Whether they liked it or not, each of those actors—Fedorov, Biden, Milley, Zelensky, and Antonov—was forced to accept Musk’s undeniable power and influence. It seems very likely that Microsoft, Google, Amazon Web Services, and Palantir have probably all influenced US policy on Ukraine in some way. Their influence may have been less controversial but no less real. They have not just been passive subjects of the White House, the Ukraine strategy having already been formulated. In each case, these companies are likely to have been involved in the process of strategizing. They have been part of the strategic community, as has Musk. The withholding of Starlink services from Ukraine in September 2022 starkly highlights an unavoidable fact; the rise of a militarytech complex amounts to a quasi-privatisation of national defence
strategy itself. Tech primes are becoming part of the strategic community; they are becoming political actors. They are informing, influencing, and steering strategy and even military operations themselves. A new civil-military settlement is appearing. This is a long way from automation. Indeed, it is deeply ironic: AI has not automated strategy, as many have feared, but politicised it. Private tech companies are now involved in strategy too. Data Issues Many scholars have been deeply concerned about the ethical implications of AI. Because they believe that AI will automate war, they fear that the most pressing ethical and legal concern is the control of autonomous weapons systems. In the near future, the worry is, AI will control lethal weapons, and consequently it will determine whether individuals are targeted and killed, independently of human judgement. Such a prospect is unconscionable. Consequently, opponents of AI argue that autonomous weapons must be regulated.19 States must prevent AI from taking over. Humans must always have the final decision whenever lethal force is employed. Military automation is certainly troubling, and lethal autonomous weapons are likely to proliferate. Yet, in the next five years at least, the main function of AI will not be automation or lethal autonomy. AI may present an alternative military risk, though. In the last decade, AI has been employed to process enormous amounts of data to identify signatures and patterns in that data so that human commanders and their staff can be decide how best to strike their opponents. But AI is not fail-safe. It is as prone to error and omission as any other source of military intelligence and information. In the next five years at least, the principal danger is not that AI might go rogue and strike targets independently, but rather that the data is inadequate or faulty or the models are not properly trained. Since an AI relies entirely on the data on which it was trained, if that data was faulty, limited, or biased, the software will become unreliable. In this situation, the AI might produce false results.
Indeed, there is an egregious recent example of the failure of AI. On 7 October 2023, thousands of Hamas fighters streamed out from Gaza into southern Israel, under the cover of a massive rocket barrage. They broke down the barrier which the Israelis had erected around the Gaza Strip and stormed into Israel in cars, motorbikes, and paragliders. In the next few hours, they massacred, raped, and mutilated twelve hundred Israelis, almost all of them civilians, and took more than two hundred hostages. The suddenness and brutality of the assault was entirely unprecedented, and Israel was shocked. Since the IDF’s withdrawal from Gaza in 2005, the Israeli security forces had monitored the Gaza Strip intensely. Israeli surveillance included physical obstacles, such as the perimeter barrier, but it also included a network of human agents and a proliferation of sensors monitoring Palestinian movements and communications in Gaza. The Israelis constructed one of the most sophisticated and intrusive spying systems, the purpose of which was to monitor Palestinian, especially Hamas, activity. By 2021, they had established an elaborate architecture of digital surveillance. AI played an important role in collating and processing the data flowing in from open-source digital sites, from a forest of cameras, from satellites, and from signals intelligence. This system played a decisive role in Operation Guardian of the Walls, Israel’s eleven-day campaign against Hamas in 2021. Yet, on 7 October 2023, the Israeli digital surveillance system, and all its AI agents and software, failed. They remained mute before and during the attack. They provided no warning of the attack nor actionable intelligence until the attack was over. This was because Hamas had employed about forty home-made suicide drones to destroy a network of remotely controlled weapons, sentry towers, cameras, and communications devices. Hamas’s drone capability was limited, but the organisation had worked out how to reconfigure the settings of their DJI Phantom drone to avoid electronic countermeasures. The 7 October attack demonstrates that while it would be unwise for military forces to eschew the advantages which AI brings, AI is not a panacea. It does not resolve
all military problems. On the contrary, it is as limited and vulnerable as any other form of military technology. If the data is lacking, then AI will be as useless as Israel’s sensors were on 7 October. In cases in which it is confused by poor or manipulated data, AI may produce the wrong results. The mistakes of AI have often been so egregious that it is obvious when AI generates a false result. Yet, in some cases, AI produces answers that are mistaken but plausible. Commanders will have to be aware that, just like humans, AI is fallible. The armed forces cannot simply rely on AI to plan or to target—much less to direct operations—on its own. AI will need to be incorporated into a planning system which corroborates the findings of AI, ensuring that its models are producing results which are consistent and plausible. The concern among scholars that responsibility for lethal decision-making is about to be abrogated to AI—that algorithms, coded into autonomous weapons systems, will decide whether to engage and kill an individual—is not irrelevant. It is possible that autonomous weapon systems might decide, independently of human guidance, whether to strike; some weapons already do this. However, as emphasised in the last chapter, AI is rarely employed to make a complete cycle of decisions. AI does not normally identify a target, decide to engage it, and strike it. Rather, AI has been employed to process data to identify targets which commanders and their staff have already designated as significant and whose significance for that phase of the operation they confirm. Military professionals decide whether and how they will prosecute targets, by reference to the overall campaign. In the past, this was legally unproblematic. Commanders and staff were formal military personnel, recognised as combatants, with the legal authority to engage military targets and to kill opponents. However, military headquarters no longer consist solely of military personnel. In order to employ AI, military headquarters now substantially consist of civilian technicians. For instance, in Task Force Dragon, civilian contractors, many from Palantir and other companies, helped to develop software and machine-learning programs which processed a large amount of data
so that XVIII Airborne Corps could provide Ukrainian forces with a list of targets. Task Force Dragon was indirectly responsible for the deaths of many senior Russian officers and hundreds of soldiers and, probably, for the wounding of General Gerasimov. The status of the civilian technicians who were contracted in to support XVIII Airborne Corps is not entirely clear. These civilians were certainly not mercenaries in any sense; they were not involved in combat. They did not directly order or oversee attacks. However, they provided services which were essential to those strikes. They were part of the ‘kill chain’. Under the laws of armed conflict, civilians cannot be combatants; any civilian who engages in armed combat in a war relinquishes their non-combatant rights. Yet because Ukraine’s defence of its own sovereign territory is plainly legitimate, there have been no legal repercussions for civilians who participated in military kill chains in this conflict. Those civilians were helping a nation to defend itself against an illegal invasion, and the deaths of Russian personnel on Ukrainian territory were legal. Nevertheless, the legal status of the civilian technicians was not entirely clear. They were not military personnel; they were not under Title X authority. On another operation, the legal status of which was more problematic, the participation of civilians in the targeting process might begin to raise concerns. Indeed, in the current war in Gaza, although most will be reserves the IDF may be drawing on civilian expertise in Unit 8200 and in other headquarters to target Hamas fighters. The Israelis have a clear right of self-defence. Yet their campaign against Hamas, in which thousands of civilians have been killed or injured, has been troubling. Indeed, in early 2024, the UN’s International Court of Justice investigated claims, sponsored by South Africa, that Israel had committed genocide in Gaza. In the end, the court did not accuse Israel of genocide, but affirmed only that Israel should avoid any military actions which might lead to such a situation. On 24 May 2024, the International Criminal Court controversially issued a warrant against Israeli prime minister Benjamin Netanyahu and Israeli minister of defence Yoav Gallant for war crimes in Gaza. It is
possible that the status of any Israeli civilian experts inside the targeting cycle may be legally questioned on the grounds that contravenes the laws of armed conflict. Data-enabled operations raise many practical, ethical, and legal questions. Commentators have often preferred to address the issue of autonomy. They are concerned about AI taking over the responsibility for killing. Yet there are many other, more pertinent issues which the military application of AI raises; the resolution of some of these seems much more difficult. Future War In the 1990s, the French scholar Paul Virilio proposed that war, like everything else, was accelerating.20 Whereas military action against an enemy had once taken weeks or months, it was now possible to mount strikes—including nuclear ones—in seconds. War had become instantaneous—and ubiquitous. More recently, many commentators, impressed by the rapid proliferation of remotely piloted aerial systems, have suggested that autonomous drone swarms will also speed up military conflict: strikes will soon be rapid, unexpected, and unstoppable. The image of war becoming faster and faster is beguiling. Attrition will be eliminated. Military forces, enabled by AI, will be able to strike immediate blows against an enemy’s political and military systems. War will be sharp, destructive, and decisive. In some cases, instantaneous strikes across the depth of the battlefield have become a reality. It is certainly true that the speed and precision of many contemporary strikes is both impressive and concerning. The strike against General Gerasimov at Zabavne (see chapter 9) was an example of the rapidity and accuracy of AIenabled modern warfare. There is little reason to doubt that strikes of this type, perhaps prosecuted by drone swarms, will become more frequent. Some targeting processes and some individual weapon systems have become faster and more precise. Yet although targeting is now often quicker, more precise, and deeper, it does not follow that war
itself will accelerate and become more decisive. The evidence from recent conflicts suggests that, ironically, the application of AI to military operations has, in fact, retarded military operations. The armed forces are now able to draw massive datasets from a constellation of sensors located across the entire battlespace—from satellites, open-source feeds, radar, ground sensors, and signals intelligence, as well as smartphones in the hands of civilians. AI has helped to analyse this data promptly, whereas previously the armed forces would have been incapable of digesting this information. The armed forces have, therefore, been able to target to an unprecedented depth, accuracy, and speed. Although each strike is rapid, however, operations have, paradoxically, slowed. It is very difficult to move without being seen.21 It is therefore very difficult to move without being targeted. Precisely because strikes are so rapid and remorseless, the pace of military operations has decreased. To survive on the battlefield and to mount offensive operations, it is imperative to degrade the sensor system of the enemy. That is a long and difficult process, as recent wars have shown. AI has accelerated individual strikes, but under the regime of AI-enabled targeting, multiple sensors, and long-range precision weapons, warfare has stagnated and slowed. So even though data is the fastest thing on the planet—traveling along fibre-optic cables and satellite communications at the speed of light22—data and AI have actually slowed military operations down. John Antal suggested that the Second Nagorno-Karabakh War was the first robotic war. Others have argued that Israel’s eleven-day action in Gaza in 2021, Operation Guardian of the Walls, was the ‘first digital war’. Yet the Russo-Ukraine War stands out as the first major interstate war of the twenty-first century in which AI has played an indispensable—at certain moments, even decisive—role. Perhaps surprisingly, then, the war has been anything but rapid or decisive. The Battle of Bakhmut may provide the best example of the way that war has congealed under AI. Today, Bakhmut is in ruins, destroyed after eight months of intense combat. Before the war, Bakhmut was a small Ukrainian town of about fifty-six thousand
inhabitants, about eighty kilometres from the Russian border, an unlikely location for a major urban battle. But in May 2022, then on the front line of the Russian invasion, it became a crucible. Having failed in their attempt to depose President Zelensky, to seize Kyiv, and to annex the entire country, Vladimir Putin and his generals had reoriented themselves to smaller goals. From April 2022, they sought to secure the Donbas (Luhansk and Donetsk provinces), and Zaporizhzhia, incorporating those regions into a greater Russia. Russian forces withdrew from the north and concentrated on the eastern and southern parts of the country instead. In the Donbas, they began to scour their way successively through Ukrainian towns, against heavy Ukrainian resistance. By June, they had taken Rubizhne, Severodonetsk, and Lysychansk. In each battle, Russian generals adopted a slow, cautious approach. They bombarded the town with massive artillery and air strikes before ordering their troops forward to seize the ruins. In this way, about 80 per cent of the buildings in Severodonetsk, for example, were damaged or destroyed. Yet the tactic worked. Despite their determination, Ukrainian defenders were slowly pushed back across the Siverskyi Donets River.23 Russian commanders could not occupy the entire Donbas, however, if they did not take Bakhmut. It was essential to take the town in order to advance on the more important locations of Kramatorsk and Slovyansk. The Battle of Bakhmut began in earnest in August 2022. By November 2022, the most brutal combat raged in and around the town. There was intense fighting inside Bakhmut itself. Ukrainian troops turned Bakhmut into a fortress. Slowly, by December, Russian forces secured western Bakhmut, building by building.24 At the same time, there was much fighting in the field on the flanks of Bakhmut. Ukrainian and Russian troops dug bunkers, trench systems, and shelters in the countryside around Bakhmut. As combat intensified, the shell-pocked battlefield and flooded trenches began to resemble the western front during the First World War. After Bakhmut eventually fell in May 2023, The Economist compared the fighting to that at Verdun in the First World War.25 Indeed, although the casualties were fewer, the battle for Bakhmut
lasted four months longer than the fighting at Verdun. ‘Despite today’s precision weaponry, the artillery battle in Bakhmut has been rudimentary, exhausting ammunition supplies […] Shelling has forced soldiers into trenches or underground’.26 Even while he ordered his mercenary forces to attack the town, Prigozhin deplored Bakhmut as a ‘meat-grinder’: a hecatomb remorselessly consuming troops on both sides. The Economist concurred: ‘This orgy of indecisive human and material destruction over a trench-scarred landscape is not what military technologists had in mind when they talked up the RMA [Revolution in Military Affairs]’.27 Bakhmut is not what AI proponents contemplate when they profess the imminent automation of military operations. Undoubtedly, the vicious combat in Bakhmut was shocking. Yet, at the same time as soldiers were hacking each other to death in waterlogged trenches in hills above the town, Russian and Ukrainian commanders were harnessing all the capabilities of the most recent technological innovations—AI, digitisation, and data. During the Battle of Bakhmut, thousands of drones were used; on any given day, dozens were flying over the town. First person–view drones struck Russian forces as they attacked the town, but most drones identified targets and gathered a mass of data.28 Both sides have employed electronic warfare to jam each other’s signals and to identify targets. They have operated in cyberspace, attacking each other’s civilian and military communications and information systems in both Ukraine and Russia and beyond. They have employed digital communications systems to plan and command operations. The Ukrainians, in particular, have relied on a dense constellation of satellites to communicate, to plan, and to coordinate their forces. The result is strange. Roman legionnaires would have recognised the carnage in the fields north of Bakhmut. Yet that horror was facilitated by satellites orbiting silently hundreds of miles above the fighting and by AI processing data in military headquarters in Ukraine and beyond.
The Next War The Russo-Ukraine War is a contest between two military forces of medium capabilities. Ukraine, with the help of the US, has substantially digitised its operations and has harnessed AI, while the Russian army, very poor though it is, has digitised some of its activities. The Russo-Ukraine War is a presage of the future. It is the best example thus far of intense interstate war in the twenty-first century. Yet it raises a question: What might a war between two AIenabled military superpowers look like? In short, what would a war between the US and China be like? Prediction is dangerous and difficult. Yet, on the basis of the war in Ukraine, it may be possible to offer some observations. Any war which might occur between China and the US in the Western Pacific, the South China Sea, the East China Sea, Taiwan, Japan, or the Korean Peninsula is likely, in the first instance, to be primarily maritime. Command of the sea is the central question for both superpowers in this region. Accordingly, any initial operations are likely to be maritime. It is possible that a maritime battle in the Indo-Pacific will itself be as decisive as other naval battles, such as Salamis, Lepanto, Trafalgar, Tsushima, and Midway. This naval battle will be different, though. It will be enabled by AI. AI may significantly alter the character of naval warfare. Satellite imagery now makes it possible to track vessels anywhere on the ocean.29 Assisted by AI, China and the US can see each other’s ships with unprecedented facility. It is likely that, in the near future, it will be possible to locate submarines too. Consequently, maritime warfare may become a duel in which warring navies are able to target each other at great range. Hypersonic missiles, with ranges of three thousand kilometres, are likely to play a crucial role here, and it is possible that surface vessels, including aircraft carriers, will become very vulnerable. Ships may be engaged accurately from hundreds, perhaps thousands, of miles away. Naval operations in Ukraine have already demonstrated the vulnerability of surface vessels to rockets launched from land. In the early months of the war, the Moskva, the flagship of the Russian
Black Sea Fleet, was supporting Russian operations in southern Ukraine, serving as an air defence and targeting platform. On 14 April 2022, the Moskva was sailing about eighty miles south of Odesa. Ukrainian forces near Odesa, perhaps assisted by allies, located the Moskva and sank her with two Neptune missiles. Neptune missiles are relatively primitive short-range rockets. However, they demonstrated the potential of land-based fires against capital ships. Since the start of the war in Gaza, the Houthis have been attacking shipping in the Red Sea with drones and rockets from the land. Because naval vessels can now be tracked more easily, they have become more vulnerable to long-range land-based missiles. Peninsulas, islands, atolls, and archipelagos are likely to become operationally, even strategically, important in ‘blue water’ maritime warfare. They may provide bases on which rocket and missile systems can be located. Fed by intelligence from sensors in the air, on the sea, in space, and in cyberspace, these bases will be able to engage enemy shipping using aircraft and land-based rocket systems. The Chinese have already fortified island chains in the East China and South China Seas precisely because they have recognised their strategic significance. They have invested heavily in a rocket force which could interdict a hostile naval force. The US Marine Corps has also developed a new concept, Expeditionary Advanced Base Operations, in which islands and land bases are decisive. The US Marine Corps is relinquishing its traditional role as an amphibious assault force. Instead, it is planning to pre-deploy forces onto a network of islands, which will provide hubs around which US naval and air operations will orbit. Ironically, as a consequence of AI’s ability to quickly locate and target naval vessels, land bases may become decisive. AI-enabled operations may generate a novel form of maritime operations in which a few key islands become vital. Blue-water naval supremacy may be won by possession of a chain of islands and atolls from which rockets and missiles can be launched with the help of AI targeting systems. AI, when combined with drones, long-range missiles and rockets, is likely to complicate naval supremacy; perhaps together these technologies will decide it.
The prospect of a land war between US and China is unpalatable. The most likely location for such a war is Taiwan. If China gained maritime and air control of the South China Sea, it might mount an amphibious assault on Taiwan. The Battle of Taiwan would likely culminate in a battle for Taipei City. The city offers Taiwanese defenders excellent terrain from which to obstruct and attrit Chinese forces. The evidence from Ukraine and other recent wars suggests that—rather than a quick and easy fight, enabled by precision weapons and rapid AI decision-making—any urban battle for Taipei City would be slow, positional, and brutal. Because combatants would be able to target each other so accurately and at such long ranges, close combat would likely be very slow, characterised by limited mobility in assault and in defence. The positional conditions which we have seen in Ukraine would likely be replicated—and magnified, considering the superior firepower and targeting capabilities of the US and Chinese forces. Whether a war between China and the US would be primarily maritime, or whether it would eventually intensify into a land campaign, it seems improbable that it would be rapid and decisive. More likely, AI would enable both belligerents to mount increasingly accurate and longer-range strikes. The result of that mutual negation would be slower, more destructive, more attritional warfare. Position and attrition, not manoeuvre and decision, are likely to dominate.30 War at the speed of data may, ironically, descend into a long, slow struggle for small pieces of terrain. The development of AI is a major technological transformation. AI has already begun to change our society. As the armed forces have sought to apply AI to military problems, AI is beginning to change how the armed forces are organised and how they conduct military operations. It has enhanced planning and targeting processes. It has improved situational awareness and, therefore, informed the way in which commanders make decisions. Any state which wants to wield military power has to embrace AI with which to enable and augment its armed forces. To do otherwise would be like failing to adopt gunpowder, airpower, tanks, or aircraft—or perhaps, even more aptly, failing to adopt mapping and charts.
Yet AI is not miraculous; it is not magic. AI offers novel capabilities, but its potential can be harnessed only through profound organisational reformation. A new relationship between the armed forces and the tech sector is required. Consequently, as armed forces pursue AI, a military-tech complex is appearing. In the next decade, the partnership between the state military forces and the private tech companies is likely to consolidate and deepen. Utopian or dystopian visions of war conducted by supercomputers and killer drone swarms are phantasmagorical. Nevertheless, an emerging military-tech complex transforms the way in which states defend themselves and fight each other. The increasing reliance of the armed forces on the data, the expertise, the computing power, and the capital of the tech sector has profound implications not just for the armed forces but for civilmilitary relations themselves. It revises the political settlements which were established between governments, industry, and the armed forces in the twentieth century. It alters the relationship between the armed forces and the private sector, the relationship between the armed forces and the state, and the relationship between the armed forces and citizens. It also changes the relationship between rival militaries. A military-tech complex does not change the nature of war, of course. It may not even revolutionise the character of war; it seems to be affirming only the current trend towards attritional and positional warfare. In the next decade, warfare is not going to fulfil the fantasies of science-fiction writers. Cyborgs will not take over. Supercomputers and killer robots and drones will not replace humans. Nevertheless, it is imperative that no one be under any illusion. The development of AI and its accompanying military-tech complex will change the armed forces, the way they fight, and their relations to the state and society. We are on the edge of a historic reformation of military affairs.
NOTES 1. Robot Wars 1. Ray Kurzweil, The Singularity is Near: When Humans Transcend Biology (London: Duckworth, 2005). 2. Ray Kurzweil, The Singularity is Nearer: When We Merge with AI (Oxford: Bodley Head, 2024). 3. Kurzweil, The Singularity is Nearer, 10. 4. James Lovelock, The Novacene: The Coming Age of Hyperintelligence (London: Penguin Books, 2020), 111. 5. Mustafa Suleyman with Michael Bhaskar, The Coming Wave: AI, Power and the Twenty-first Century’s Greatest Dilemma (London: Bodley Head, 2023), 3. 6. Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans (London: Pelican, 2019), 198–9. 7. Suleyman, The Coming Wave, 53. 8. Suleyman, The Coming Wave, 51. 9. Suleyman, The Coming Wave, 53. 10. Marcus du Sautoy, The Creativity Code: Art and Innovation in the Age of AI (Cambridge, MA: The Belknap Press, 2019), 31. 11. Matthew Sparkes, ‘DeepMind’s Protein-Folding AI Cracks Biology’s Biggest Problem’, New Scientist, 28 July 2022, https://www.newscientist.com/article/2330866-deepminds-protein-folding-aicracks-biologys-biggest-problem. 12. Sparkes, ‘DeepMind’s Protein-Folding AI Cracks Biology’s Biggest Problem’. 13. Chiara Longoni and Carey Morewedge, ‘AI Can Outperform Doctors. So Why Don’t Patients Trust It?’, Harvard Business Review, 30 October 2019, https://hbr.org/2019/10/ai-can-outperform-doctors-so-why-dont-patients-trust-it. 14. Jeremy Hsu, ‘AI Discovers New Class of Antibiotics to Kill Drug-Resistant Bacteria’, New Scientist, 20 December 2023, https://www.newscientist.com/article/2409706-ai-discovers-new-class-ofantibiotics-to-kill-drug-resistantbacteria/#:~:text=The%20researchers%20have%20begun%20using,such%20as %20osteoarthritis%20and%20cancer. 15. Hsu, ‘AI Discovers New Class of Antibiotics to Kill Drug-Resistant Bacteria’.
16. Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford: Oxford University Press, 2017), 9. 17. Viktor Mayer-Schönberger and Kenneth Cukier, Big Data: A Revolution that Will Transform How We Live, Work, and Think (New York: Mariner Books, Houghton Mifflin Harcourt, 2014), 145-6. 18. Brad Stone, Amazon Unbound: Jeff Bezos and the Invention of a Global Empire (London: Simon & Schuster, 2021); Brad Stone, The Everything Store: Jeff Bezos and the Age of Amazon (London: Corgi, 2018). 19. Constance Hays, ‘What Wal-Mart Knows about Customers’ Habits’, The New York Times, 14 November 2004. https://www.nytimes.com/2004/11/14/business/yourmoney/what-walmart-knowsabout-customers-habits.html. 20. Donald Mackenzie, Trading at the Speed of Light: How Fast Algorithms Are Transforming Financial Markets (Princeton, NJ: Princeton University Press, 2021). 21. The Economist, ‘AI for All’, 27 January 2024, 10. 22. The Economist, ‘AI for All’, 16. 23. The Economist, ‘Chatbots for the Bottom Four Billion’, 27 January 2024, 16. 24. PM London Tech Week Speech, 12 June 2023, https://www.gov.uk/government/speeches/pm-london-tech-week-speech-12-june2023#:~:text=Combined%20with%20the%20computational%20power,I%20know %20people%20are%20concerned. 25. Sara Brown, ‘Why Neural Net Pioneer Geoffrey Hinton is Sounding the Alarm on AI’, MIT Sloan School of Management, 23 May 2023. https://mitsloan.mit.edu/ideas-made-to-matter/why-neural-net-pioneer-geoffreyhinton-sounding-alarm-ai. 26. Paul Taylor, ‘Llamas, Pizzas, Mandolins: AI Doomerism’ [Review of The Coming Wave: Technology, Power and the Twenty-first Century’s Greatest Dilemma, by Mustafa Suleyman, and The World I See: Curiosity, Exploration and Discovery at the Dawn of AI, by Fei-Fei Li], London Review of Books, 46(6) (21 March 2024), 17. 27. Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher, The Age of AI and Our Human Future (London: John Murray, 2021), 17. 28. Kissinger, Schmidt, and Huttenlocher, The Age of AI and Our Human Future, 159, 161. 29. Kissinger, Schmidt, and Huttenlocher, The Age of AI and Our Human Future, 173–4. 30. Kenneth Payne, I, Warbot: The Dawn of Artificially Intelligent Conflict (London: Hurst, 2011), 96. 31. Payne, I, Warbot, 83. 32. Denise Garcia, ‘Lethal Artificial Intelligence and Change: The Future of International Peace and Security’, International Studies Review, 20(2) (2018), 334. 33. Denise Garcia, The AI Military Race: Common Good Governance in the Age of Artificial Intelligence (Oxford: Oxford University Press, 2023), 4.
34. Garcia, The AI Military Race, 2. 35. Roberto Gonzalez, War Virtually: The Quest to Automate Conflict, Militarize Data, and Predict the Future (Oakland, CA: University of Oakland, 2022), 6. 36. James Johnson, The AI Commander: Centaur Teaming, Command, and Ethical Dilemmas (Oxford: Oxford University Press, 2024), 171. 37. Johnson, The AI Commander, 51. 38. Johnson, The AI Commander, 171. 39. John Williams, ‘Locating LAWS: Lethal Autonomous Weapons, Epistemic Space and “Meaningful” Human Control’, Journal of Global Security Studies, 6(4) (2021), 1–17; James Johnson, ‘Inadvertent Escalation’, European Journal of International Security, 7(3) (2022), 337–59; Denise Garcia, ‘Lethal Artificial Intelligence and Change’, International Studies Review, 20(2) (2018), 334–41; Ingvild Bode and Hendrik Huelss, ‘Autonomous Weapons and Changing Norms in International Relations’, Review of International Studies, 44(3) (2018), 393–413; Michael-Carl Haas and Sophie-Charlotte Fischer, ‘The Evolution of Targeted Killing Practices’, Contemporary Security Policy, 38(2) (2017), 281–306; Louise Amoore, ‘Algorithmic Warfare: Everyday Geographies of the War on Terror’, Antipode, 41(1) (2009), 49–62; Lucy Suchman, ‘Algorithmic Warfare and the Reinvention of Accuracy’, Critical Studies on Security, 8(2) (2020), 175–87. 40. Future of Life Institute, ‘Autonomous Weapons Open Letter: AI & Robotics Researchers’, 28 July 2015, https://www.stopkillerrobots.org/wpcontent/uploads/2013/03/FLI_LtrJuly2015.pdf. 41. DUST, ‘Sci-Fi Short Film: “Slaughterbots”’, YouTube video, 7:58, 17 October 2019, https://www.youtube.com/watch?v=O-2tpwW0kmU. 42. Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control (London: Allen Lane, 2019). 43. Stuart Russell, ‘AI in Warfare’, Reith Lecture 2021 (audio recording), 57:54, BBC Radio 4, 8 December 2021, https://www.bbc.co.uk/programmes/m00127t9. 44. Russell, ‘AI in Warfare’. 45. Russell, ‘AI in Warfare’. 46. Eric Schmidt, ‘Innovation Power: Why Technology Will Define the Future of Geopolitics’, Foreign Affairs, March/April 2023, https://www.foreignaffairs.com/united-states/eric-schmidt-innovation-powertechnology-geopolitics. 47. Jürgen Altmann and Frank Sauer, ‘Autonomous Weapon Systems and Strategic Stability’, Survival, 59(5) (2017), 131. 48. Suchman, ‘Algorithmic Warfare and the Reinvention of Accuracy’, 183. 49. E.g., Williams, ‘Locating LAWS’. 50. Ben Buchanan and Andrew Imbrie, The New Fire: War, Peace and Democracy in the Age of AI (Cambridge, MA: MIT Press, 2022), 1–2. 51. Buchanan and Imbrie, The New Fire, 139–40. 52. David Hambling, Swarm Troopers: How Small Drones Will Conquer the World (Archangel Ink, 2015), 203.
53. Hambling, Swarm Troopers, 223, 226, 238. 54. Paul Scharre, Four Battlegrounds: Power in the Age of Artificial Intelligence (New York: W.W. Norton and Company, 2023), 3. 55. Scharre, Four Battlegrounds, 2. 56. Paul Scharre, Army of None: Autonomous Weapons and the Future of War (New York: W.W. Norton and Co., 2019), 4. 57. James Baker, Centaur’s Dilemma: National Security Law for the Coming AI Revolution (Washington, DC: Brookings Institute Press, 2021), 38. 58. John Antal, 7 Seconds to Die: A Military Analysis of the Second NagornoKarabakh War and the Future of Warfighting (Oxford: Casement, 2022), 51–2. 59. Seth Frantzman, Drone Wars: Pioneers, Killing Machines, Artificial Intelligence, and the Battle for the Future (New York: Post Hill Press/Bombardier, 2021), 179. 60. John Arquilla, Bitskrieg: The New Challenge of Cyberwarfare (Cambridge: Polity Press, 2022), xvii. 61. Ronald Arkin, ‘The Case for Ethical Autonomy in Unmanned Systems’, Journal of Military Ethics, 9(4) (2010), 332–41. 62. Elsa Kania, Battlefield Singularity: Artificial Intelligence, Military Revolution and China’s Future Military Power, Center for a New American Security, 28 November 2017, https://www.cnas.org/publications/reports/battlefield-singularityartificial-intelligence-military-revolution-and-chinas-future-military-power, 6. 63. Quoted in Michael Horowitz, Gregory Allen, Elsa Kania, and Paul Scharre, Strategic Competition in an Era of Artificial Intelligence, Center for a New American Security, 25 July 2018, https://www.cnas.org/publications/reports/strategic-competition-in-an-era-ofartificial-intelligence, 8. 64. Gonzalez, War Virtually, 62. 65. Quoted in David Ignatius, ‘How the Algorithm Tipped the Balance in Ukraine’ (opinion), The Washington Post, 19 December 2022, accessed 2 November 2023, https://www.washingtonpost.com/opinions/2022/12/19/palantiralgorithm-data-ukraine-war. 66. Rand Waltzman, Lillian Ablon, Christian Curriden, Gavin S. Hartnett, Maynard A. Holliday, Logan Ma, Brian Nichiporuk, Andrew Scobell, and Danielle C. Tarraf, Maintaining the Competitive Advantage in Artificial Intelligence and Machine Learning (Santa Monica, CA: Rand, 2020), 6. 67. Austin Wyatt, ‘Charting Great Power Progress toward a Lethal Autonomous Weapon System Demonstration Point’, Defence Studies, 20(1) (2020), 10. 68. Jon Harper, ‘Pentagon Requesting More than $3B for AI and JADC2’, DefenseScoop, 13 March 2023, https://defensescoop.com/2023/03/13/pentagonrequesting-more-than-3b-for-ai-jadc2. 69. Altmann and Sauer, ‘Autonomous Weapon Systems and Strategic Stability’, 123.
70. Kania, Battlefield Singularity, 22–23; Elsa Kania, Chinese Military Innovation in Artificial Intelligence: Hearing of the US-China Economic and Security Review Commission, Center for a New American Security, 7 June 2019. 71. US Army E-6, interview, National Training Centre, 16 May 2023. 72. Avi Goldfarb and Jon Lindsay, ‘Prediction and Judgment: Why Artificial Intelligence Increases the Importance of Humans in War’, International Security, 46(3) (Winter 2021/22), 8, https://direct.mit.edu/isec/article/46/3/7/109668/Prediction-and-Judgment-WhyArtificial. 73. Goldfarb and Lindsay, ‘Prediction and Judgment’, 9. 74. Goldfarb and Lindsay, ‘Prediction and Judgment’, 19. 75. Goldfarb and Lindsay, ‘Prediction and Judgment’, 9. 76. Goldfarb and Lindsay, ‘Prediction and Judgment’, 36. 77. Goldfarb and Lindsay, ‘Prediction and Judgment’, 9. 78. Benjamin Jensen, Christopher Whyte, and Scott Cuomo, ‘Algorithms at War: The Promise, the Peril and Limits of Artificial Intelligence’, International Studies Review, 22(3) (2020), 531. 79. Jensen, Whyte, and Cuomo, ‘Algorithms at War’, 532. 80. Jensen, Whyte, and Cuomo, ‘Algorithms at War’, 541. 81. Jensen, Whyte, and Cuomo, ‘Algorithms at War’, 541. 82. Lauren Wilcox, ‘Embodying Algorithmic War: Gender, Race, and the Posthuman in Drone Warfare’, Security Dialogue, 48(1) (2017), 11–28. 83. Jensen, Whyte, and Cuomo, ‘Algorithms at War’, 540. 84. Jensen, Whyte, and Cuomo, ‘Algorithms at War’, 543. 85. Cameron Hunter and Bleddyn Bowen, ‘We’ll Never Have a Model of an AI Major-General: Artificial Intelligence, Command Decisions, and Kitsch Visions of War’, Journal of Strategic Studies, 47(1) 2024, 128–9. 86. Hunter and Bowen, ‘We’ll Never Have a Model of an AI Major-General’, 134. 87. Scharre, Four Battlegrounds, 3. 88. Officer, RAF, anonymous, personal communication, 10 November 2022. 89. DARPA, ‘AlphaDogfight Trials Foreshadow Future of Human-Machine Symbiosis’ (news release), 26 August 2020, https://www.darpa.mil/newsevents/2020-08-26. 90. Quoted in Joseph Trevithick, ‘Future of Artificial Intelligence Dominated Air Combat Showcased in New Air Force Video’, The Warzone, 5 July 2023, https://www.twz.com/future-of-artificial-intelligence-dominated-air-combatshowcased-in-new-air-force-video. 91. David Hume, An Enquiry concerning Human Understanding (Cambridge: Hackett Publishing Company, 1977), 41–2. 92. David Hume, A Treatise of Human Nature (Oxford: Oxford University Press, 1985), 141. 93. Quoted in Ignatius, ‘How the Algorithm Tipped the Balance in Ukraine’.
94. James Hockenhull, ‘How Open Source Intelligence Has Shaped the RussiaUkraine War’, Royal United Services Institute for Defence and Security Studies, speech, 7 November 2022. https://www.gov.uk/government/speeches/how-opensource-intelligence-has-shaped-the-russia-ukrainewar#:~:text=Open%20source%20offers%20us%20an,gain%20from%20our%20 global%20network. 95. Yoram Evron and Richard Bitzinger, The Fourth Industrial Revolution and Military-Civil Fusion: A New Paradigm for Military Innovation? (Cambridge, MA: Cambridge University Press, 2023). 96. Hockenhull, ‘How Open Source Intelligence Has Shaped the Russia-Ukraine War’. 97. Hockenhull, ‘How Open Source Intelligence Has Shaped the Russia-Ukraine War’. 2. What AI Can Do 1. Gary Marcus and Ernest Davis, Rebooting AI: Building Artificial Intelligence We Can Trust (New York: Pantheon Books, 2019), 44, 54–5. 2. Helga Nowotny, In AI We Trust: Power, Illusion and Control of Predictive Algorithms (Cambridge: Polity Press, 2021), 27. 3. Suleyman, The Coming Wave, 32. 4. Quoted in Margaret Boden, AI: Its Nature and Future (Oxford: Oxford University Press), 10. 5. Brian Cantwell Smith, The Promise of Artificial Intelligence: Reckoning and Judgment (Cambridge, MA: MIT Press, 2019), 28. 6. Hubert Dreyfus, What Computers Still Can’t Do: A Critique of Artificial Reason (Cambridge, MA: MIT Press, 1992), x–xi. 7. Cantwell Smith, The Promise of Artificial Intelligence, 48. 8. Dreyfus, What Computers Still Can’t Do, 72. 9. Dreyfus, What Computers Still Can’t Do, 82. 10. Dreyfus, What Computers Still Can’t Do, 82. 11. Buchanan and Imbrie, The New Fire, 14. 12. Buchanan and Imbrie, The New Fire, 14. 13. Quoted in Gonzalez, War Virtually, 134. 14. Suleyman, The Coming Wave, 33. 15. Edge Delta, ‘Breaking Down the Numbers: How Much Data Does the World Create Daily in 2024?’, 11 March 2024, https://edgedelta.com/company/blog/howmuch-data-is-created-perday#:~:text=An%20internet%20user%20generates%20an%20average%20of%2 0146%2C880%20MB%20daily.&text=Researchers%20have%20stated%20that%2 0the,create%20about%20506%2C736%20MB%20daily. 16. Cited in Shoshana Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (London: Profile Books, 2019), 428.
17. Kate Crawford, Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence (London: Yale University Press, 2021), 113. 18. Boden, AI, 47. 19. Boden, AI, 47. 20. Boden, AI, 47. 21. Boden, AI, 47. 22. Martin Ford, Architects of Intelligence: The Truth about AI from the People Building It (Birmingham: Packt Publishing, 2018), 228. 23. Yann LeCunn in Martin Ford, Architects of Intelligence, 123. 24. Taylor, ‘Llamas, Pizzas, Mandolins’, 18. 25. Mitchell, Artificial Intelligence, 81–2. 26. Mitchell, Artificial Intelligence, 182–208. 27. Register Dynamics, ‘The Exponential Growth of Computational Power behind AI’, accessed 29 October 2024, https://www.registerdynamics.co.uk/blog/the-exponential-growth-of-computational-power-behind-ai. 28. Buchanan and Imbrie, The New Fire, 64, 75. 29. Crawford, Atlas of AI, 98, 116–17. 30. Alistair Barr, ‘Google Mistakenly Tags Black People as “Gorillas”, Showing Limits of Algorithms’, The Wall Street Journal, 1 July 2015, https://www.wsj.com/articles/BL-DGB-42522; also Crawford, Atlas of AI, 128. 31. Marcus and Davis, Rebooting AI, 27. 32. Mitchell, Artificial Intelligence, 296. 33. Quoted in Mitchell, Artificial Intelligence, 297. 34. The Economist, ‘The Race is on to Control the Global Supply Chain for AI Chips’, 30 July 2024, https://www.economist.com/schools-brief/2024/07/30/therace-is-on-to-control-the-global-supply-chain-for-ai-chips. 35. Ethan Mollick, Co-Intelligence: Living and Working with AI (London: WH Allen, 2024), 10. 36. Kurzweil, The Singularity is Nearer, 47; Mollick, Co-Intelligence, 10. 37. Kurzweil, The Singularity is Nearer, 47. 38. Suleyman, The Coming Wave, 66. 39. Mollick, Co-Intelligence, 12. 40. Mollick, Co-Intelligence, 19. 41. Benjamin Jensen and Dan Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’, War on the Rocks, 12 April 2023, https://warontherocks.com/2023/04/how-large-language-models-canrevolutionize-military-planning. 42. The Economist, ‘The Age of Pseudocognition’, 22 April 2023, 18. 43. Quoted in The Economist, ‘The Age of Pseudocognition’, 18. 44. The Economist, ‘How to Worry Wisely about AI’, 22 April 2023, 9. 45. Suleyman, The Coming Wave, 8. 46. The Economist, ‘A Second Flight’, 30 September 2023, 17. 47. The Economist, ‘A Second Flight’, 20.
48. James Muldoon, Mark Graham, and Callum Cant, Feeding the Machine: The Hidden Human Labour Powering AI (Edinburgh: Canongate, 2024), 50. 49. Muldoon, Graham, and Cant, Feeding the Machine, 54. 50. Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell, ‘On the Dangers of Stochastic Parrots: Can Language Models be too Big? FAccT ’21: Proceedings of the 2021 Association for Computing Machinery Conference on Fairness, Accountability, and Transparency, 610–23. 51. Muldoon, Graham, and Cant, Feeding the Machine, 59. 52. Muldoon, Graham, and Cant, Feeding the Machine, 60. 53. Carl Benedikt Frey and Michael Osborne, The Future of Employment: How Susceptible are Jobs to Computerisation? (Oxford: Oxford Martin Programme on Technology and Employment, 2013), https://omswww.files.svdcdn.com/production/downloads/academic/future-of-employment.pdf, 1, 40; Muldoon, Graham, and Cant, Feeding the Machine, 52. 54. Muldoon, Graham, and Cant, Feeding the Machine, 53. 55. Muldoon, Graham, and Cant, Feeding the Machine, 52. 56. Paul Scharre, Army of None, 4. 57. Lee Gomes, ‘Hidden Obstacles for Google’s Self-driving Cars’, MIT Technology Review, 28 August 2014, https://www.technologyreview.com/2014/08/28/171520/hidden-obstacles-forgoogles-self-driving-cars. 58. Alex Davies, ‘Google’s Self-driving Car Caused its First Crash’, Wired, 29 February 2016, https://www.wired.com/2016/02/googles-self-driving-car-maycaused-first-crash. 59. Goldfarb and Lindsay, ‘Prediction and Judgment’, 23. 60. Matthew Johnson and Alonso Vera, ‘No AI is an Island: The Case for Teaming Intelligence’, AI Magazine, 40(1) (2019), https://doi.org/10.1609/aimag.v40i1.2842, 18. 61. Walter Isaacson, Elon Musk (London: Simon & Schuster, 2023), 248. 62. Stone, The Everything Store, 256. 63. Stone, The Everything Store, 255. 64. Stone, The Everything Store, 302. 65. Stone, Amazon Unbound, 232–3. 66. Stone, Amazon Unbound, 151. 67. Stone, Amazon Unbound, 101. 68. Stone, Amazon Unbound, 101. 69. Stone, Amazon Unbound, 102. 70. Stone, The Everything Store, 257. 71. Stone, Amazon Unbound, 10. 72. Jason del Rey, ‘How Robots are Transforming Amazon Warehouse Jobs—for Better and Worse’, Vox, 11 December 2019, https://www.vox.com/recode/2019/12/11/20982652/robots-amazon-warehousejobs-automation.
73. Stone, Amazon Unbound, 14. 74. Stone, The Everything Store, 241. 75. Isaacson, Elon Musk, 281–2. 76. Kai-Fu Lee, AI Superpowers: China, Silicon Valley, and the New World Order (Boston: Houghton Mifflin Harcourt, 2018), 71. 3. AI Strategy 1. Quoted in Kania, Battlefield Singularity, 13, and Kania, Chinese Military Innovation in Artificial Intelligence. 2. Quoted in Jack Shanahan, ‘AI—A Game Changer and Decisive Edge’, Chief Digital and Artificial Intelligence Office, 11 September 2019, https://www.ai.mil/blog_09_11_19.html. 3. Cheryl Pellerin, ‘Work: Human-Machine Teaming Represents Defense Technology Future’, US Department of Defense, 8 November 2015, https://www.defense.gov/News/News-Stories/Article/Article/628154/work-humanmachine-teaming-represents-defense-technology-future. 4. ‘Remarks by Deputy Secretary Work on Third Offset Strategy’, US Department of Defense, 28 April 2016, https://www.defense.gov/News/Speeches/Speech/Article/753482/remarks-bydeputy-secretary-work-on-third-offset-strategy. 5. ‘Remarks by Deputy Secretary Work on Third Offset Strategy’. 6. ‘Remarks by Deputy Secretary Work on Third Offset Strategy’. 7. US Department of Defense, Summary of the 2018 National Defense Strategy of the United States of America: Sharpening the American Military’s Competitive Edge, 2018, https://dod.defense.gov/Portals/1/Documents/pubs/2018-NationalDefense-Strategy-Summary.pdf, 1. 8. US Department of Defense, Summary of the 2018 National Defense Strategy, 5. 9. US Department of Defense, Summary of the 2018 National Defence Strategy, 3. 10. The Joint Staff, Description of the National Military Strategy 2018, https://www.jcs.mil/Portals/36/Documents/Publications/UNCLASS_2018_National_ Military_Strategy_Description.pdf. 11. The National Security Commission on Artificial Intelligence (NSCAI), Final Report, 2021, https://nscai.wpenginepowered.com/wpcontent/uploads/2021/03/Full-Report-Digital-1.pdf, 23. 12. NSCAI, Final Report, 79. 13. NSCAI, Final Report, 80. 14. NSCAI, Final Report, 111. 15. NSCAI, Final Report, 77. 16. NSCAI, Final Report, 77. 17. US Department of Defense, Summary of the Joint All-Domain Command and Control (JADC2) Strategy, March 2022,
https://media.defense.gov/2022/Mar/17/2002958406/-1/-1/1/SUMMARY-OF-THEJOINT-ALL-DOMAIN-COMMAND-AND-CONTROL-STRATEGY.PDF, 1. 18. US Department of Defense, Summary of the Joint All-Domain Command and Control (JADC2) Strategy, 4. 19. US Department of Defense, Summary of the Joint All-Domain Command and Control (JADC2) Strategy, 4. 20. US Department of Defense, Summary of the Joint All-Domain Command and Control (JADC2) Strategy, 5. 21. Christopher Moran, Joe Burton, and George Christou, ‘The US Intelligence Community, Global Security, and AI: From Secret Intelligence to Smart Spying’, Journal of Global Security Studies, 2(2) (2023), 1–18. 22. David Hambling, ‘AI is Helping the US Air Force to Decide Which Targets to Strike’, New Scientist, 28 September 2021, https://www.newscientist.com/article/2291586-ai-is-helping-us-air-force-to-decidewhich-targets-to-strike. 23. US Department of the Army, Field Manual 3-0 Operations, October 2022, https://armypubs.army.mil/epubs/DR_pubs/DR_a/ARN36290-FM_3-0-000-WEB2.pdf, 1–2. 24. US Department of the Army, Field Manual 3-0 Operations, 1. 25. US Department of the Army, Field Manual 3-0 Operations, 5–8. 26. Lieutenant General Jack Shanahan, personal interview, 25 May 2023, Quantico, Virginia. 27. OF-3, Major, US Army, interviewee 44, personal interview, 30 June 2022. 28. UK Ministry of Defence, Data Strategy for Defence, 27 September 2021, https://www.gov.uk/government/publications/data-strategy-for-defence/datastrategy-for-defence. 29. UK Ministry of Defence, Data Strategy for Defence. 30. UK Ministry of Defence, Data Strategy for Defence. 31. UK Ministry of Defence, Defence’s Response to a More Contested and Volatile World, CP 901, July 2023, https://assets.publishing.service.gov.uk/media/64b55dd30ea2cb000d15e3fe/Defen ce_Command_Paper_2023_Defence_s_response_to_a_more_contested_and_volati le_world.pdf, 34. 32. UK Ministry of Defence, Defence’s Response to a More Contested and Volatile World, 8. 33. Major-General, OF-7, British Army, interviewee 50, personal interview, 9 August 2022. 34. Major, OF-3, British Army, interviewee 52, personal interview, 2 September 2022. 35. NATO, ‘Summary of the NATO Artificial Intelligence Strategy’, 22 October 2021, https://www.nato.int/cps/en/natohq/official_texts_187617.htm. 36. NATO, Draft Concept 0.8: Operating Concept for Multi-Domain Operations in the Urban Environment, January 2023.
37. Quoted in Crawford, Atlas of AI, 193. 4. A Military-Tech Complex 1. Kelly DeVries, ‘Catapults Are Not Atomic Bombs: Towards a Redefinition of “Effectiveness” in Premodern Military Technology’, War in History, 4(4) (1997), 466, 467; Bert Hall and Kelly DeVries, ‘Review of The Military Revolution: Military Innovation and the Rise of the West, 1500–1800, by G. Parker’, Technology and Culture, 31(3) (1990), 520–70. 2. Sebastian Mallaby, The Power Law: Venture Capital and the Making of the New Future (London: Allen Lane, 2022), 20. 3. Mallaby, The Power Law, 19. 4. Mallaby, The Power Law, 39. 5. Mallaby, The Power Law, 81. 6. Mallaby, The Power Law, 82–95. 7. Simona R. Soare, Pavneet Singh, and Meia Nouwens, Software-Defined Defence: Algorithms at War, International Institute for Strategic Studies, February 2023, https://www.iiss.org/globalassets/media-library---content-migration/files/research-papers/iiss_software-defined-defence_17022023.pdf, 12. 8. Briana Reilly, ‘Pentagon’s Priority on AI Spending Could Shield it from Cuts’, Roll Call, 5 March 2024. https://rollcall.com/2024/03/05/pentagons-priority-on-aispending-could-shield-it-from-cuts. 9. Soare, Singh, and Nouwens, Software-Defined Defence, 13. 10. International Institute for Strategic Studies, The Military Balance 2024 (London: Taylor & Francis, 2024), 90, 96. 11. International Institute for Strategic Studies, The Military Balance 2024, 151. 12. Soare, Singh, and Nouwens, Software-Defined Defence, 12. 13. International Institute for Strategic Studies, The Military Balance 2024, 359. 14. Sharon Wrobel, ‘Israel Invests NIS 500m in R&D Infrastructure to Keep Pace in Global AI Race’, The Times of Israel, 18 September 2024, https://www.timesofisrael.com/israel-invests-nis-500m-in-rd-infrastructure-tokeep-pace-in-global-ai-race. 15. Tiago Bianchi, ‘Annual Revenue of Google 2002-2023’, Statista, 22 May 2024, https://www.statista.com/statistics/266206/googles-annual-global-revenue. 16. Daniela Coppola, ‘Annual Net Sales Revenue of Amazon from 2006 to 2023, by Segment’, Statista, 22 May 2024, https://www.statista.com/statistics/266289/net-revenue-of-amazon-by-region. 17. Frederica Laricchia, ‘Global Revenue of Apple from 2004 to 2023’, Statista, 21 May 2024, https://www.statista.com/statistics/265125/total-net-sales-of-applesince-2004. 18. Stacy Jo Dixon, ‘Annual Revenue Generated by Meta Platforms from 2009 to 2023’, Statista, 16 February 2024, https://www.statista.com/statistics/268604/annual-revenue-of-
facebook/#:~:text=In%202023%2C%20the%20revenue%20general,in%20the% 20previous%20fiscal%20year. 19. Macrotrends, ‘Microsoft Revenue 2010-2024’, accessed 29 October 2024, https://www.macrotrends.net/stocks/charts/MSFT/microsoft/revenue#:~:text=Mic rosoft%20revenue%20for%20the%20twelve,a%206.88%25%20increase%20from %202022. 20. Frederica Laricchia, ‘Apple Inc’s Expenditure on Research and Development from Fiscal Year 2007 to 2023’, Statista, 21 May 2024, https://www.statista.com/statistics/273006/apple-expenses-for-research-anddevelopment/#:~:text=Apple%20Inc.,billion%20from%20its%202022%20total. 21. Macrotrends, ‘Alphabet Research and Development Expenses 2010-2024’, accessed 29 October 2024, https://www.macrotrends.net/stocks/charts/GOOGL/alphabet/researchdevelopmentexpenses#:~:text=Alphabet%20annual%20research%20and%20development,a% 2025.15%25%20increase%20from%202021. 22. Macrotrends, ‘Meta Platforms Research and Development Expenses 20102024’, accessed 29 October 2024, https://www.macrotrends.net/stocks/charts/META/meta-platforms/researchdevelopment-expenses. 23. Finbox, ‘R&D Expense for Amazon.com Inc’, accessed 29 October 2024, https://finbox.com/NASDAQGS:AMZN/explorer/rd_exp/#:~:text=Amazon.com’s% 20r%26d%20expenses%20increased,to%2085.622%20billion%20in%202023. 24. Macrotrends, ‘Microsoft Research and Development Expenses 2010-2024’, accessed 29 October 2024, https://www.macrotrends.net/stocks/charts/MSFT/microsoft/researchdevelopmentexpenses#:~:text=Microsoft%20annual%20research%20and%20development%2 0expenses%20for%202023%20were%20%2427.195,a%2018.32%25%20increase %20from%202021. 25. Scharre, Four Battlegrounds, 17. 26. Technician, tech sector, interviewee 111, personal interview, 15 May 2022. 27. Mallaby, The Power Law, 20–21. 28. Gonzalez, War Virtually, 62–3. 29. Gonzalez, War Virtually, 64. 30. Gonzalez, War Virtually, 68. 31. Isaacson, Elon Musk, 289–91. 32. Chafkin, The Contrarian, 39. 33. Eric M. Jackson, The PayPal Wars: Battles with eBay, the Media, the Mafia, and the Rest of Planet Earth (Los Angeles: World Ahead Publishing, 2004), 19. 34. Max Chafkin, The Contrarian: Peter Thiel and Silicon Valley’s Pursuit of Power (London: Bloomsbury, 2021), 112. 35. Chafkin, The Contrarian, 111–12.
36. Chafkin, The Contrarian, 145. 37. Eric Schmidt, ‘I Used to Run Google. Silicon Valley Could Lose to China’ (opinion), The New York Times, 27 February 2020, https://www.nytimes.com/2020/02/27/opinion/eric-schmidt-ai-china.html. 38. Schmidt, ‘I Used to Run Google’. 39. SCSP, ‘Conflict and Competition in the Digital Age: Lessons from Ukraine’, 22-2, 21 April 2022, https://scsp222.substack.com/p/conflict-and-competition-inthe-digital. 40. Scharre, Four Battlegrounds, 59. 41. Defense Innovation Unit, ‘About DIU’, accessed 24 October 2024, https://www.diu.mil/about. 42. Defense Innovation Unit, ‘About DIU’. 43. Defense Innovation Unit, ‘Annual Report FY 2022’, accessed 24 October 2024, https://downloads.ctfassets.net/3nanhbfkr0pc/5guJIhcMGwIgoop4z9r5QM/89f44fe 62f981e5f0d28932618719196/DIU_Annual_Report_FY22_Final_0131.pdf, 9–10. 44. Defense Innovation Unit, ‘Annual Report FY 2022’, 10. 45. US Deputy Secretary of Defense, ‘Memo: Establishment of an Algorithmic Warfare Cross-Functional Team (Project Maven)’, 26 April 2017, https://dodcio.defense.gov/Portals/0/Documents/Project%20Maven%20DSD%20M emo%2020170425.pdf. 46. Lieutenant General Jack Shanahan, US Air Force, interviewee 16, personal interview, 25 May 2023. 47. Lieutenant General Jack Shanahan, US Air Force, interviewee 16, personal interview, 25 May 2023. 48. Scharre, Four Battlegrounds, 214. 49. Scharre, Four Battlegrounds, 204. 50. Special Competitive Studies Project, interviewee 115, personal interview, 22 May 2022. 51. UK Ministry of Defence, civil servant, interviewee 65, personal interview, 19 April 2023. 52. Executive, Elbit Systems, interviewee 85, personal interview, 19 December 2022. 53. UK Ministry of Defence, ‘Digital Strategy for Defence’, 27 May 2021, https://www.gov.uk/government/publications/digital-strategy-for-defencedelivering-the-digital-backbone-and-unleashing-the-power-of-defencesdata/digital-strategy-for-defence. 54. Defence Artificial Intelligence Centre, ‘Defence Artificial Intelligence Centre —GOV.UK’, accessed 29 October 2024, https://www.gov.uk/government/groups/defence-artificial-intelligence-centre. 55. Defence Artificial Intelligence Centre, ‘Defence Artificial Intelligence Centre —GOV.UK’. 56. Soare, Singh, and Nouwens, Software-Defined Defence, 31.
57. Jeffrey Dastin, ‘Palantir Lands 75 Million Pound Deal with British Military’, Reuters, 21 December 2022, https://www.reuters.com/business/aerospacedefense/palantir-lands-75-million-pound-deal-with-british-military-2022-12-21. 58. Soare, Singh, and Nouwens, Software-Defined Defence, 32. 59. Soare, Singh, and Nouwens, Software-Defined Defence, 10. 60. Edward Luttwak and Eitan Shamir, The Art of Military Innovation: Lessons from the Israel Defense Forces (Cambridge, MA: Harvard University Press, 2023), 59–60. 61. Executive, Elbit Systems, interviewee 85, personal interview, 19 December 2022. 5. The Special Relationship 1. Peter Thiel with Blake Masters, Zero to One: Notes on Start-ups, or How to Build a Future (London: Virgin Books, 2014), 145. 2. Thiel, Zero to One, 146. 3. Thiel, Zero to One, 147. 4. Thiel, Zero to One, 137. 5. Thiel, Zero to One, 132. 6. Thiel, Zero to One, 132. 7. Chafkin, The Contrarian, 147. 8. As the commander of the 5th Stryker Brigade in Kandahar in 2009 and 2010, Tunnell had rejected population-centric counter-insurgency methods embraced by the NATO forces in Iraq. He trained his brigade to hunt terrorists aggressively. The 5th Stryker Brigade was responsible for many civilians’ deaths during its deployment. Later it emerged that a deviant subculture had developed in one of the brigade’s units; a small group of soldiers had formed a ‘Kill Club’. These soldiers had murdered civilians and collected body parts from their victims. 9. David Barno and Nora Bensahel, Adaptation under Fire: How Militaries Change in Wartime (Oxford: Oxford University Press, 2023), 164. 10. Chafkin, The Contrarian, 149. 11. Barno and Bensahel, Adaptation under Fire, 165. 12. Barno and Bensahel, Adaptation under Fire, 164. 13. Barno and Bensahel, Adaptation under Fire, 165. 14. Chafkin, The Contrarian, 152. 15. Chafkin, The Contrarian, 150. 16. Ashlee Vance and Brad Stone, “Palantir: The War on Terror’s Secret Weapon,” Bloomberg, 22 November 2011, https://www.bloomberg.com/news/articles/2011-11-22/palantir-the-war-onterrors-secret-weapon. 17. Mark Bowden, The Finish: The Killing of Osama bin Laden (New York: Atlantic Monthly Press, 2012), 102. 18. Quoted in Chafkin, The Contrarian, 153. 19. Chafkin, The Contrarian, 283–90.
20. Quoted in Chafkin, The Contrarian, 215. 21. Tech executive and technician, interviewees 29 and 30, personal interview, 5 January 2021. 22. Tech executive and technician, interviewees 29 and 30, personal interview, 5 January 2021. 23. Chafkin, The Contrarian, 153. 24. Executive, interviewee 120, personal interview, 6 June 2023. 25. Chafkin, The Contrarian, 284, 288. 26. Thiel, Zero to One, 147. 27. Thiel, Zero to One, 141. 28. Thiel, Zero to One, 143. 29. Thiel, Zero to One, 149–50. 30. Rune Henriksen, ‘Warriors in Combat—What Makes People Actively Fight in Combat?’, Journal of Strategic Studies, 30(2) (2007), 187–223; Tone Danielson, Making Warriors in a Global Era: An Ethnographic Study of the Norwegian Naval Special Operations Commando (Lanham, MD: Lexington Books, 2018); Bernd, No Ordinary Men: Special Operations Forces Missions in Afghanistan (Toronto: Dundurn, 2014). 31. Anthony King, ‘The Special Air Service and the Concentration of Military Power’, Armed Forces & Society, 35(4) (2009), 646–66. 32. Eitan Shamir and Eyal Ben-Ari, ‘The Rise of Special Operations Forces: Generalized Specialization, Boundary Spanning and Military Autonomy’, Journal of Strategic Studies, 41(3) (2018), 335–71. 33. US Special Operations Command, Fiscal Year 2024, Budget Estimates, March 2023, https://comptroller.defense.gov/Portals/45/Documents/defbudget/fy2024/budget_j ustification/pdfs/01_Operation_and_Maintenance/O_M_VOL_1_PART_1/SOCOM_O P-5.pdf, 4, 1. 34. Jessica Glicken Turnley, Kobi Michael, and Eyal Ben-Ari (eds.), Special Operations Forces in the 21st Century: Perspectives from the Social Sciences (London: Palgrave, 2017). 35. Shamir and Ben-Ari, ‘The Rise of Special Operations Forces’, 356. 36. Tech executive, interviewee 54, personal email, 16 September 2022. 37. Anthony King, ‘What is Special about Special Operations Forces?’, in Jessica Glicken Turnley, Kobi Michael, and Eyal Ben-Ari (eds.), Special Operations Forces in the 21st Century: Perspectives from the Social Sciences (London: Palgrave, 2017), 273–84. 38. Tech executive, interviewee 54, personal interview, 16 September 2022. 39. Tech executive, former Special Operations Forces officer, interviewee 46, personal interview, 8 July 2022. 40. Nick Lopez and Kyle Atwell, ‘Artificial Intelligence in Counterterrorism and Counterinsurgency, with Retired Gen. Stanley McChrystal and Dr. Anshu Roy’, Modern War Institute, Irregular Warfare Podcast, 51:09, 1 January 2021,
https://mwi.usma.edu/artificial-intelligence-in-counterterrorism-andcounterinsurgency-with-retired-gen-stan-mcchrystal-and-dr-anshu-roy. 41. Lopez and Atwell, ‘Artificial Intelligence in Counterterrorism and Counterinsurgency’. 42. Lopez and Atwell, ‘Artificial Intelligence in Counterterrorism and Counterinsurgency’. 43. Lopez and Atwell, ‘Artificial Intelligence in Counterterrorism and Counterinsurgency’. 44. Tech executive, interviewee 54, personal interview, 8 November 2023. 45. Anonymous source, 4 December 2020. 46. Anonymous source, 4 December 2020. 47. Anonymous source, 1 December 2022. 48. Anonymous source, 1 December 2022. 49. Anonymous source, 1 December 2022. 50. Tech executive, interviewee 54, personal interview, 8 November 2023. 51. Tech executive, interviewee 54, personal interview, 8 November 2023. 52. Tech executive, interviewee 54, personal interview, 8 November 2023. 53. Anonymous source, 1 December 2022. 54. Anonymous source, Israel Defense Forces, interviewee 56, personal interview, 18 October 2022. 55. See chapter 8. 56. IDF officer, OF-3, interviewee 57, personal interview, 18 October 2022. 57. IDF officer, OF-3, interviewee 57, personal interview, 18 October 2022. 6. AI and Planning 1. Anthony King, Command: The Twenty-first-Century General (Cambridge: Cambridge University Press, 2019), 66–7. 2. Martin van Creveld, Command in War (Cambridge, MA: Harvard University Press, 1985), 7. 3. See chapter 3. 4. The Economist, ‘Predict and Survive’, 23 July 2022, 67. 5. The Economist, ‘Predict and Survive’, 68. 6. Paolo de Heer, Nico de Reus, Lucia Tealdi, and Philip Kerbusch, ‘Intelligence Augmentation for Urban Warfare Operation Planning Using Deep Reinforcement Learning’, in Tien Pham (ed.), Proceedings of SPIE—The International Society for Optical Engineering, 11006, 35. Bellingham, WA: International Society for Optics and Photonics, 2019 https://www.researchgate.net/publication/333073489_Intelligence_augmentation_ for_urban_warfare_operation_planning_using_deep_reinforcement_learning. 7. de Heer, de Reus, Tealdi, and Kerbusch, ‘Intelligence Augmentation for Urban Warfare Operation Planning Using Deep Reinforcement Learning’.. 8. The last chapter described how the JAIC had already developed similar software in order to map wildfires.
9. Hannes Mueller, Andre Groeger, Jonathan Hersh, Andrea Matranga, and Joan Serrat, ‘Monitoring War Destruction from Space Using Machine Learning’, Proceedings of the National Academy of Sciences of the United States of America, 118(23) (2021), 8. 10. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 11. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 12. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 13. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 14. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 15. UK Ministry of Defence, ‘Realising the Potential of Sustained Artificial Intelligence Development and Integration in Defence’, whitepaper by the Enhanced C2 Spearhead Team, Army Information Directorate, on behalf of the Defence Innovation Unit (2021), 1. 16. UK Ministry of Defence, ‘Realising the Potential of Sustained Artificial Intelligence Development and Integration in Defence’, 2. 17. Janes, ‘Janes at the Heart of the British Army’s First AI-Driven Battlefield Operation’ (news release), 24 August 2021, https://www.janes.com/osintinsights/defence-news/defence/janes-at-the-heart-of-the-british-army’s-first-aidriven-battlefield-operation. 18. UK Ministry of Defence, ‘Realising the Potential of Sustained Artificial Intelligence Development and Integration in Defence’, 2. 19. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 20. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 21. Lieutenant Colonel, OF-4, British Army, Commander EC2SPH, interviewee 32, personal interview, 27 January 2022. 22. Elbit Systems executive, interviewee 62, personal interview, 10 January 2023. 23. Rob Bassett Cross, Adarga, presentation, RUSI Land Warfare Conference, June 2023. 24. Sam Schechner and Daniel Michaels, ‘Ukraine Has Digitized its Fighting Forces on a Shoestring’, The Wall Street Journal, 3 January 2023, https://www.wsj.com/articles/ukraine-has-digitized-its-fighting-forces-on-ashoestring-11672741405. 25. Schechner and Michaels, ‘Ukraine Has Digitized its Fighting Forces on a Shoestring’.
26. Anduril, ‘Lattice’, https://web.archive.org/web/20220215021601/https://www.anduril.com/lattice. 27. Anduril, ‘Counter Intrusion’, https://www.anduril.com/capability/land. 28. The Economist, ‘Model Major-General’, 22 June 2024, 18. 29. Quoted in The Economist, ‘Model Major-General’, 18. 30. The Economist, ‘Model Major-General’, 18. 31. Special Competitive Studies Project (SCSP), Generative AI: The Future of Innovation Power, 2023, https://www.scsp.ai/wp-content/uploads/2023/09/GenAIweb.pdf. 32. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 33. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 34. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 35. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 36. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 37. Jensen and Tadross, ‘How Large-Language Models Can Revolutionize Military Planning’. 38. Interviewee 115, Special Competitive Studies Project, personal interview, 22 May 2023. 39. Interviewee 115, Special Competitive Studies Project, personal interview, 22 May 2023. 40. Owen J. Daniels, ‘How Revisiting Naval Aviation’s Lessons Can (and Cannot) Inform Military AI Innovation’ (opinion), Breaking Defense, 25 August 2023, https://breakingdefense.com/2023/08/how-revisiting-naval-aviations-lessons-canand-cannot-inform-military-ai-innovation. 41. Jeremy Black, ‘A Revolution in Military Cartography? Europe 1650–1815’, Journal of Military History, 73(1) (2009), 64-65. 42. Black, ‘A Revolution in Military Cartography?’ 7. AI and Targeting 1. Eli Berman, Jacob Felter, and Joseph Shapiro, Small Wars, Big Data: The Information Revolution in Modern Conflict (Princeton, NJ: Princeton University Press, 2018), xiii. 2. Berman, Felter, and Shapiro, Small Wars, Big Data, 13. 3. US Deputy Secretary of Defense, ‘Memo: Establishment of an Algorithmic Warfare Cross-Functional Team (Project Maven)’. 4. John (Jack) N.T. Shanahan, ‘Artificial Intelligence, Ubiquitous Sensors, and Human-Machine Teams: How AI Will Transform the Intelligence Cycle’, in William J.
Lahneman and Florina Cristiana Matei (eds.), Intelligence and Technology: Trends, Challenges, and Choices (Boulder, CO: Lynne Rienner Publishers, forthcoming), 2. 5. Jason Brown, quoted in Scharre, Four Battlegrounds, 55. 6. Shanahan, ‘Artificial Intelligence, Ubiquitous Sensors, and Human-Machine Teams’, 1–2. 7. Scharre, Four Battlegrounds, 54–5. 8. Shanahan, ‘Artificial Intelligence, Ubiquitous Sensors, and Human-Machine Teams’, 4. 9. Crawford, Atlas of AI, 189. 10. Quoted in Scharre, Four Battlegrounds, 56. 11. Lieutenant General Jack Shanahan, US Air Force, interviewee 16, personal interview, 25 May 2023. 12. Scharre, Four Battlegrounds, 56. 13. Scharre, Four Battlegrounds, 57. 14. Quoted in Scharre, Four Battlegrounds, 57. 15. Shanahan, ‘Artificial Intelligence (AI) and Intelligence’, 116. 16. Shanahan, ‘Artificial Intelligence (AI) and Intelligence’, 34. 17. John (Jack) N.T. Shanahan, Lieutenant General, US Air Force (retired), interviewee 16, personal interview, 25 May 2023. 18. Scharre, Four Battlegrounds, 58. 19. University of Liverpool, Covid-SMART rapid antigen community testing evaluations, 23 December 2020, https://www.liverpool.ac.uk/research/researchthemes/infectious-diseases/coronavirus-research/covid-smart-pilot. 20. Professor Iain Buchan, Chair of Public Health and Civic Informatics at Liverpool University, interviewee 47, personal interview, 12 July 2022. 21. Buchan, interviewee 47, personal interview, 12 July 2022. 22. Buchan, interviewee 47, personal interview, 28 November 2022. 23. Brigadier Joe Fossey OBE, OF-6, Commander 8 Engineer Brigade, British Army, interviewee 43, personal interview, 14 June 2022. 24. Iain Buchan, ‘Closing the Data-Action Gap for Better Health and Care: A Civic Blueprint’, slideshow presentation, 16 June 2022. 25. Fossey, interviewee 43, personal interview, 14 June 2022. 26. Captain Tom de Silva, OF-2, GEO cell, 8 Brigade, interviewee 45, personal interview, 4 July 2022. 27. de Silva, OF-2, interviewee 45, personal interview, 4 July 2022. 28. Fossey, interviewee 43, personal interview, 14 June 2022. 29. de Silva, OF-2, interviewee 45, personal interview, 4 July 2022. 30. Major Luke Palmer, OF-3, Royal Engineers, British Army, interviewee 52, personal interview, 2 September 2022. 31. Palmer, interviewee 52, personal interview, 2 September 2022. 32. Fossey, interviewee 45, personal interview, 14 June 2022. 33. de Silva, interviewee 45, personal interview, 4 July 2022. 34. de Silva, interviewee 45, personal interview, 4 July 2022.
35. Fossey, interviewee 45, personal interview, 14 June 2022. 36. de Silva, interviewee 45, personal interview, 4 July 2022. 37. de Silva, interviewee 45, personal interview, 4 July 2022. 38. de Silva, interviewee 45, personal interview, 4 July 2022. 39. de Silva, interviewee 45, personal interview, 4 July 2022. 40. de Silva, interviewee 45, personal interview, 4 July 2022. 41. de Silva, interviewee 45, personal interview, 4 July 2022. 42. University of Liverpool, Covid-SMART rapid antigen community testing evaluations. 43. Iain Buchan, ‘Civic Informatics in Covid-19 Response: Clarifying Mass Testing and Reopening Mass Gathering’, slideshow presentation, 30 April 2022. 44. Other armed forces, such as the Dutch Army, used data in similar ways during the pandemic. See Marijn Hoijtink, ‘ “Prototype Warfare”: Innovation, Optimisation, and the Experimental Way of Warfare’, European Journal of International Security, 7(3) (2022), 322–36. 45. Anonymous source, Israel Defense Forces, interviewee 56, personal interview, 18 October 2022. 46. Eliran Rubin, ‘Tiny IDF Unit is Brains behind Israeli Army Artificial Intelligence’, Haaretz, 15 August 2017, https://www.haaretz.com/israelnews/2017-08-15/ty-article/tiny-idf-unit-is-brains-behind-israeli-army-artificialintelligence/0000017f-e35b-d7b2-a77f-e35fc8f40000. 47. ‘The Future of Artificial Intelligence in the IDF’, Israel Defense, 2 July 2017, https://www.israeldefense.co.il/en/node/30189. 48. Sefi Cohen, quoted in Rubin, ‘Tiny IDF Unit is Brains behind Israeli Army Artificial Intelligence’. 49. Jewish Institute for National Security America (JINSA), ‘Gaza Conflict 2021: Observations and Lessons’, 28 October 2021, https://jinsa.org/jinsa_report/gazaconflict-2021-assessment-observations-and-lessons, 10. 50. JINSA, ‘Gaza Conflict 2021’, 16. 51. Quoted in Yuval Abraham, ‘[[thinspace]“A Mass Assassination Factory”: Inside Israel’s Calculated Bombing of Gaza’, +972 Magazine, 30 November 2023, https://www.972mag.com/mass-assassination-factory-israel-calculated-bombinggaza. 52. Ron Ben-Yishai, ‘How Data and AI Drove the IDF Operation in Gaza’, Ynet News, 29 May 2021, https://www.ynetnews.com/magazine/article/SJ2rHS6Y00. 53. JINSA, ‘Gaza Conflict 2021’, 19. 54. Yuval Abraham, ‘ “Lavender”, the AI Machine Directing Israel’s Bombing Spree in Gaza’, +972 Magazine, 3 April 2024, https://www.972mag.com/lavenderai-israeli-army-gaza. 55. Harry Davies, Bethan McKernan, and Dan Sabbagh, ‘ “The Gospel”: How Israel Uses AI to Select Bombing Targets in Gaza’, The Guardian, 1 December 2023, https://www.theguardian.com/world/2023/dec/01/the-gospel-how-israeluses-ai-to-select-bombing-targets.
56. The Guardian, ‘Israel Defence Forces’ Response to Claims about use of “Lavender” AI Database in Gaza’, 3 April 2024, https://www.theguardian.com/world/2024/apr/03/israel-defence-forces-responseto-claims-about-use-of-lavender-ai-database-in-gaza. 57. Fossey, interviewee 43, personal interview, 14 June 2022. 58. Buchan, interviewee 47, personal interview, 12 July 2022. 59. Davies, McKernan, and Sabbagh, ‘ “The Gospel.”’ 8. AI and Cyber Operations 1. Elisabeth Bumiller and Thom Shanker, ‘Panetta Warns of Dire Threat of Cyber Attack on U.S.’, The New York Times, 12 October 2012, https://www.nytimes.com/2012/10/12/world/panetta-warns-of-dire-threat-ofcyberattack.html. 2. Jon R. Lindsay, ‘Stuxnet and the Limits of Cyber Warfare’, Security Studies, 22(3) (2013), 367. 3. Carl von Clausewitz, On War, translated by Peter Paret and Michael Howard (Princeton, NJ: Princeton University Press, 1984), 75. 4. Thomas Rid, ‘Cyber War Will Not Take Place’, Journal of Strategic Studies, 35(1) (2012), 6. 5. Rid, ‘Cyber War Will Not Take Place’, 6. 6. Rid, ‘Cyber War Will Not Take Place’, 11. 7. ‘General Sir Patrick Sanders, ‘Chief of the General Staff Speech at RUSI Land Warfare Conference’, 28 June 2022, https://www.gov.uk/government/speeches/chief-the-general-staff-speech-at-rusiland-warfare-conference 8. Altmann and Sauer, ‘Autonomous Weapon Systems and Strategic Stability’, 123. 9. Lindsay, ‘Stuxnet and the Limits of Cyber Warfare’, 369. 10. Lindsay, ‘Stuxnet and the Limits of Cyber Warfare’, 373. 11. Rid, ‘Cyber War Will Not Take Place’. 12. Lindsay, ‘Stuxnet and the Limits of Cyber Warfare’, 391. 13. Lindsay, ‘Stuxnet and the Limits of Cyber Warfare’, 390. 14. Rid, ‘Cyber War Will Not Take Place’, 13. 15. Susan Landau, ‘Cyberwar in Ukraine: What You See is Not What’s Really There’, Lawfare, 30 September 2022, https://www.lawfaremedia.org/article/cyberwar-ukraine-what-you-see-not-whatsreally-there. 16. National Cyber Security Centre, ‘UK and Allies Expose Russian Intelligence Services for Cyber Campaign of Attempted Political Interference’, 7 December 2023, https://www.ncsc.gov.uk/news/uk-and-allies-expose-cyber-campaignattempted-political-interference, 4. 17. UK Foreign & Commonwealth Office, National Cyber Security Centre, and Lord (Tariq) Ahmad of Wimbledon KCMG, ‘Foreign Office Minister Condemns
Russia for NotPetya Attacks’, 15 February 2018, https://www.gov.uk/government/news/foreign-office-minister-condemns-russiafor-notpetya-attacks. 18. HYPR, ‘What is NotPetya? 5 Fast Facts’, accessed 29 October 2024,https://www.hypr.com/security-encyclopedia/notpetya. 19. Rid, ‘Cyber War Will Not Take Place’, 20–21. 20. SCSP, ‘Conflict and Competition in the Digital Age’. 21. The Economist, ‘A Nest of Wipers’, 76. 22. Brad Smith, ‘Defending Ukraine: Early Lessons’, Microsoft on the Issues (blog), 22 June 2022, https://blogs.microsoft.com/on-theissues/2022/06/22/defending-ukraine-early-lessons-from-the-cyber-war. 23. The Economist, ‘A Nest of Wipers’, 75. 24. The Economist, ‘A Nest of Wipers’, 76. 25. Daniel Moore, quoted in The Economist, ‘A Nest of Wipers’, 76. 26. The Economist, ‘A Nest of Wipers’, 76. 27. SCSP, ‘Conflict and Competition in the Digital Age’. 28. Stefan Soesanto, ‘Ukraine’s IT Army’, Survival, 65(3) (2023), 94. 29. Soesanto, ‘Ukraine’s IT Army’, 99. 30. Soesanto, ‘Ukraine’s IT Army’, 96. 31. Soesanto, ‘Ukraine’s IT Army’, 95. 32. Thomas Rid, Active Measures: The Secret History of Disinformation and Political Warfare (London: Profile Books, 2021), 12. 33. Rid, Active Measures, 13. 34. Rid, Active Measures, 13–14. 35. Rid, Active Measures, 402, 408. 36. Scharre, Four Battlegrounds, 141. 37. Jarred Prier, ‘Commanding the Trend: Social Media as Information Warfare’, Strategic Studies Quarterly, 11(4) (2017), 54. 38. Scharre, Four Battlegrounds, 127. 39. Scharre, Four Battlegrounds, 132. 40. Andrew Hoskins and Pavel Shchelin, ‘The War Feed: Digital War in Plain Sight’, American Behavioral Scientist, 67(3) (2023), 449–63. 41. Hoskins and Shchelin, ‘The War Feed’. 42. Drew Harwell, ‘Ukraine is Scanning Faces of Dead Russians, then Contacting the Mothers’, The Washington Post, 15 April 2022, https://www.washingtonpost.com/technology/2022/04/15/ukraine-facialrecognition-warfare. 43. Audrey Kurth Cronin, ‘Open Source Technology and Public-Private Innovation Are the Key to Ukraine’s Strategic Resilience’, War on the Rocks, 25 August 2023, https://warontherocks.com/2023/08/open-source-technology-andpublic-private-innovation-are-the-key-to-ukraines-strategic-resilience. 44. Scharre, Four Battlegrounds, 129. 45. The Economist, ‘An Unspooked Spook’, 24 June 2023, 29–30.
46. Michael Martelle and Audrey Alexander, ‘Operation Glowing Symphony: The Missing Piece in the US Online Counter-ISIS Campaign’, in Michael Sexton and Eliza Campbell (eds.), Cyber War and Cyber Peace: Digital Conflict in the Middle East (London: I.B. Tauris, 2022), 119–136. 47. Dina Temple-Raston, ‘How the U.S. Hacked ISIS’, NPR, 26 September 2019, https://www.npr.org/2019/09/26/763545811/how-the-u-s-hacked-isis. 48. Martelle and Alexander, ‘Operation Glowing Symphony’. 49. Daniel Chernobrov, ‘Diasporas as Cyberwarriors: Infopolitics, Participatory Warfare and the 2020 Karabakh War’, International Affairs, 98(2) (2002), 636. 50. Quoted in Chernobrov, ‘Diasporas as Cyberwarriors’, 638. 51. Chernobrov, ‘Diasporas as Cyberwarriors’, 643. 52. Chernobrov, ‘Diasporas as Cyberwarriors’, 642. 53. Chernobrov, ‘Diasporas as Cyberwarriors’, 643. 54. Quoted in The Economist, ‘A Nest of Wipers’, 76. 55. Quoted in The Economist, ‘A Nest of Wipers’, 76. 56. Prier, ‘Commanding the Trend’, 55. 57. Prier, ‘Commanding the Trend’, 73. 9. The Human-Machine Team 1. Baker, Centaur’s Dilemma, 4. 2. Baker, Centaur’s Dilemma, 4. 3. Y.S. [Yossi Sariel], The Human-Machine Team: How to Create Synergy between Human and Artificial Intelligence that Will Revolutionize Our World (independently published, 2021), 48. 4. Y.S., The Human-Machine Team, 51. 5. Y.S., The Human-Machine Team, 27. 6. Y.S., The Human-Machine Team, 50. 7. Y.S., The Human-Machine Team, 76. 8. Mick Ryan, Human-Machine Teaming for Future Ground Forces (Washington, DC: Center for Strategic and Budgetary Assessments, 2018), 17. 9. Ryan, Human-Machine Teaming for Future Ground Forces, 33–34. 10. Alex Neads, David Galbreath, and Theo Farrell, From Tools to Teammates: Human-Machine Teaming and the Future of Command and Control in the Australian Army, Australian Army Occasional Paper No. 7 (Australian Army Research Centre, 2021), https://researchcentre.army.gov.au/sites/default/files/AARC%20Occasional%20Pap er%20No.7%20-%20From%20Tools%20to%20Teammates.pdf, 13; see also Alex Neads, Theo Farrell, and David Galbreath, ‘Evolving towards Military Innovation: AI and the Australian Army’, Journal of Strategic Studies, 24 April 2023, https://doi.org/10.1080/01402390.2023.2200588, 10. 11. Neads, Galbreath, and Farrell, From Tools to Teammates, 3. 12. Karl Marx, Capital: A Critique of Political Economy, vol. 1, translated by Samuel Moore and Edward Aveling (Chicago: Charles H. Kerr and Co., 1932), 83.
13. du Sautoy, The Creativity Code, 28. 14. du Sautoy, The Creativity Code, 37. 15. du Sautoy, The Creativity Code, 36. 16. Quoted in Anthony Amicelle, ‘Big Data Surveillance across Fields: Algorithmic Governance for Policing & Regulation’, Big Data & Society, 9(2) (2022). 17. Amicelle, ‘Big Data Surveillance across Fields’. 18. Johnson and Vera, ‘No AI is an Island’, 18; see also Matthew Johnson, Brandon Shrewsbury, Sylvain Bertrand, and Duncan Calvert, ‘Team IHMC’s Lessons Learned from the DARPA Robotics Challenge: Finding Data in the Rubble’, Journal of Field Robotics, 34(2) (2016), http://dx.doi.org/10.1002/rob.21674, 18. 19. Johnson and Vera, ‘No AI is an Island’, 23. 20. Matthew Johnson, Jeffrey Mark Bradshaw, Robert Hoffman, Paul J. Feltovich, and David Woods, ‘Seven Cardinal Virtues of Human-Machine Teamwork: Examples from the DARPA Robotic Challenge’, IEEE Intelligent Systems, 29(6) (2014), 74–80. 21. Johnson et al., ‘Seven Cardinal Virtues of Human-Machine Teamwork’. 22. Johnson and Vera, ‘No AI is an Island’, 26. 23. Tech executive, interviewee 54, personal email, 8 November 2023. 24. Michael Yon, ‘Deuce Four: B. General Erik Kurilla Makes an Inspirational Speech Mentioning in Detail Heroes Like Scott Smiley’, Facebook, 1 January 2016, https://www.facebook.com/MichaelYonFanPage/photos/deuce-four-b-general-erikkurilla-makes-an-inspirational-speech-mentioning-in-de/10153342117445665.] 25. Michael Yon, ‘Gates of Fire’, Michael Yon Online, 31 August 2005; Corey Dickstein, ‘Battling Terrorism in Afghanistan from Over-the-Horizon is “Extremely Difficult,” Says Nominee to Command CENTCOM’, Stars and Stripes, 8 February 2022, https://www.stripes.com/theaters/us/2022-02-08/senate-centcom-generalkurilla-army-afghanistan-iran-isis-4841105.html 26. See chapter 7. 27. Lieutenant General Jack Shanahan, US Air Force, interviewee 16, personal interview, 25 May 2023. 28. See chapter 5. 29. Scharre, Four Battlegrounds, 57. 30. Quoted in Matthew Visser, ‘XVIII Airborne Corps Honors Outgoing Chief Technology Officer’, Defense Visual Information Service, 20 January 2023, https://www.dvidshub.net/news/437019/xviii-airborne-corps-honors-outgoingchief-technology-officer. 31. Stanley McChrystal, with Tantum Collins, David Silverman, and Chris Fussell, Team of Teams: New Rules of Engagement for a Complex World (London: Penguin Books, 2015), 225. 32. Interviewee 115, Special Competitive Studies Project, personal interview, 22 May 2023. 33. Major, OF-3, US Army, interviewee 63, personal interview, 14 November 2022.
34. Barno and Bensahel, Adaptation under Fire, 164. 35. Ignatius, ‘How the Algorithm Tipped the Balance in Ukraine’. 36. Quoted in SCSP, ‘Dr. Alex Karp of Palantir Reflects on the Lessons Learned from the War in Ukraine’, 2-2-2, 24 February 2023, https://scsp222.substack.com/p/special-guest-edition-dr-alex-karp. 37. Tech executive, informal communication, 8 December 2022. 38. Major, OF-4, US Army, interviewee 114, personal interview, 18 May 2022. 10. War at the Speed of Light 1. Douhet, Giulio, The Command of the Air (Tuscaloosa: University of Alabama, 2009), 55. 2. Richard Overy, The Bombing War: Europe, 1939–1945 (London: Penguin Books, 2014), 267. 3. Christopher Spearin, Private Military and Security Companies and States: Force Divided (London: Palgrave, 2017); Peter Singer, Corporate Warriors: The Rise of the Privatized Military Industry (Ithaca, NY: Cornell University Press, 2003); Eugene Smith, ‘The New Condottieri and US Policy: The Privatization of Conflict and its Implications’, Parameters, 32(4) (2002), 104–19; Joakim Berndtsson and Christopher Kinsey (eds.), The Routledge Research Companion to Security Outsourcing (Oxford: Routledge, 2016); Deborah Avant, The Market for Force: The Consequences of Privatising Security (Cambridge: Cambridge University Press, 2005); Christopher Kinsey, Corporate Soldiers and International Security: The Rise of Private Military Companies (London: Routledge, 2006); Christopher Kinsey, Private Contractors and the Reconstruction of Iraq: Transforming Military Logistics (London: Routledge, 2009); Christopher Kinsey and Malcolm Patterson (eds.), Contractors and War: The Transformation of United States’ Expeditionary Operations (Stanford, CA: Stanford University Press, 2012); Elke Krahmann, States, Citizens and the Privatization of Security (Cambridge: Cambridge University Press, 2010). 4. Max Weber, ‘Politics as a Vocation’, in H. Gerth and C. Wright Mills (eds.), From Max Weber: Essays in Sociology (London: Routledge, 2007), 78. 5. E.g., Peter Wilson, ‘Prussia as a Fiscal-Military State, 1640–1806’, in Christopher Storrs (ed.), The Fiscal-Military State in Eighteenth-Century Europe: Essays in Honour of P.G.M. Dickson (London: Routledge, 2009), 95–124. 6. Fernand Braudel, The Mediterranean and Mediterranean World in the Age of Philip II (Berkeley: University of California Press, 1996). 7. Niccolo Machiavelli, The Prince, translated by G. Bull (London: Penguin Books, 1999), 39. 8. The Economist, ‘A Nest of Wipers’, 76. 9. The Economist, ‘A Nest of Wipers’, 76. 10. Quoted in Isaacson, Elon Musk, 429. 11. The Economist, ‘The Satellites that Saved Ukraine’, 7 January 2023, 13. 12. Quoted in Isaacson, Elon Musk, 430.
13. Isaacson, Elon Musk, 430. 14. Isaacson, Elon Musk, 431. 15. Isaacson, Elon Musk, 431. 16. Quoted in Isaacson, Elon Musk, 434. 17. Quoted in Isaacson, Elon Musk, 434. 18. Quoted in Tara Copp, ‘Elon Musk Blocking Starlink to Stop Ukraine Attack Troubling for DoD’, Military Times, 12 September 2023, https://www.militarytimes.com/news/your-military/2023/09/12/elon-muskblocking-starlink-to-stop-ukraine-attack-troubling-for-dod. 19. E.g. Altmann and Sauer, ‘Autonomous Weapon Systems and Strategic Stability’, 117–42. 20. Paul Virilio and Sylvere Lotringer, Pure War (Los Angeles: Semiotext(e), 1999); Paul Virilio, Speed and Politics (Los Angeles: Semiotext(e), 2006). 21. Jack Watling, The Arms of the Future: Technology and Close Combat in the Twenty-first Century (London: Bloomsbury, 2023). 22. Mackenzie, Trading at the Speed of Light. 23. Franz-Stefan Gady and Michael Kofman, ‘Ukraine’s Strategy of Attrition’, Survival, 65(2) (2023), 7–22. 24. John Spencer, ‘The Battle of Bakhmut’, Modern Warfare Institute, Urban Warfare Project Podcast, 35:38, 17 March 2023, https://mwi.westpoint.edu/thebattle-of-bakhmut. 25. The Economist, ‘Bakhmut and the Spirit of Verdun’, 3 June 2023, 30. 26. The Economist, ‘Bakhmut and the Spirit of Verdun’. 27. Shashank Joshi, ‘The War in Ukraine Shows How Technology is Changing the Battlefield’, special report, The Economist, 3 July 2023, https://www.economist.com/special-report/2023/07/03/the-war-in-ukraine-showshow-technology-is-changing-the-battlefield. 28. Stacie Pettyjohn, Evolution not Revolution: Drone Warfare in Russia’s 2022 Invasion of Ukraine (Washington, DC: Center for a New American Security, February 2024), 31. 29. Saadia M. Pekkanen, Setsuko Aoki, and John Mittleman, ‘Small Satellites, Big Data: Uncovering the Invisible in Maritime Security’, International Security, 47(2) (2022), 177–216. 30. Amos Fox, ‘Manoeuvre is Dead? Understanding the Conditions and Components of Warfighting’, The RUSI Journal, 166(6–7) (2021), 10–18; Watling, The Arms of the Future.
BIBLIOGRAPHY Abraham, Yuval, ‘ “Lavender”, the AI Machine Directing Israel’s Bombing Spree in Gaza’, +972 Magazine, 3 April 2024. https://www.972mag.com/lavender-aiisraeli-army-gaza. ———, ‘ “A Mass Assassination Factory”: Inside Israel’s Calculated Bombing of Gaza’, +972 Magazine, 30 November 2023. https://www.972mag.com/massassassination-factory-israel-calculated-bombing-gaza. Altmann, Jürgen, and Frank Sauer, ‘Autonomous Weapon Systems and Strategic Stability’, Survival, 59(5) (2017), 117–42. Amicelle, Anthony, ‘Big Data Surveillance across Fields: Algorithmic Governance for Policing & Regulation’, Big Data & Society, 9(2) (2022). Amoore, Louise, ‘Algorithmic Warfare: Everyday Geographies of the War on Terror’, Antipode, 41(1) (2009), 49–62. Antal, John, 7 Seconds to Die: A Military Analysis of the Second Nagorno-Karabakh War and the Future of Warfighting (Oxford: Casemate, 2022). Aradau, Claudia, and Tobias Blanke, ‘The (Big) Data-Security Assemblage: Knowledge and Critique’, Big Data & Society 2(2) (2015). Arkin, Ronald, ‘The Case for Ethical Autonomy in Unmanned Systems’, Journal of Military Ethics, 9(4) (2010), 332–41. Arquilla, John, Bitskrieg: The New Challenge of Cyberwarfare (Cambridge: Polity Press, 2022). Arquilla, John, and David Ronfeldt, In Athena’s Camp: Preparing for Conflict in the Information Age (Santa Monica, CA: Rand, 1997). Ayoub, Kareem, and Kenneth Payne, ‘Strategy in the Age of Artificial Intelligence’, Journal of Strategic Studies, 39(5–6) (2016), 793–819. Avant, Deborah, The Market for Force: The Consequences of Privatising Security (Cambridge: Cambridge University Press, 2005). Baker, James, Centaur’s Dilemma: National Security Law for the Coming AI Revolution (Washington, DC: Brookings Institution Press, 2021). Barno, David, and Nora Bensahel, Adaptation under Fire: How Militaries Change in Wartime (Oxford: Oxford University Press, 2023). Barr, Alistair, ‘Google Mistakenly Tags Black People as “Gorillas”, Showing Limits of Algorithms’, Wall Street Journal, 1 July 2015. https://www.wsj.com/articles/BLDGB-42522.
Bender, Emily, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell, ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? FAccT ’21: Proceedings of the 2021 Association for Computing Machinery Conference on Fairness, Accountability, and Transparency, 610–23. https://doi.org/10.1145/3442188.3445922. Ben-Yishai, Ron, ‘How Data and AI Drove the IDF Operation in Gaza’, Ynet News, 29 May 2021. https://www.ynetnews.com/magazine/article/SJ2rHS6Y00. Berman, Eli, Jacob Felter, and Joseph Shapiro, Small Wars, Big Data: The Information Revolution in Modern Conflict (Princeton, NJ: Princeton University Press, 2018). Berndtsson, Joakim, and Christopher Kinsey (eds.), The Routledge Research Companion to Security Outsourcing (Oxford: Routledge, 2016). Bianchi, Tiago, ‘Annual Revenue of Google 2002-2023’, Statista, 22 May 2024. https://www.statista.com/statistics/266206/googles-annual-global-revenue. Biddle, Stephen, Military Power: Explaining Victory and Defeat in Modern Battle (Princeton, NJ: Princeton University Press, 2004). Black, Jeremy, ‘A Revolution in Military Cartography? Europe 1650–1815’, Journal of Military History, 73(1) (2009), 49–68. ———, A Military Revolution? Military Change and European Society, 1550–1800 (London: Macmillan, 1991). Bode, Ingvild, and Hendrik Huelss, ‘Autonomous Weapons and Changing Norms in International Relations’, Review of International Studies, 44(3) (2018), 393– 413. Boden, Margaret, AI: Its Nature and Future (Oxford: Oxford University Press, 2016). Bostrom, Nick, Superintelligence: Paths, Dangers, Strategies (Oxford: Oxford University Press, 2017). Bowden, Mark, The Finish: The Killing of Osama bin Laden (New York: Atlantic Monthly Press, 2012). Braudel, Fernand, The Mediterranean and Mediterranean World in the Age of Philip II (Berkeley: University of California Press, 1996). Broekhof, Maarten, Martijn Kitzen, and Frans Osinga, ‘A Tale of Two Mosuls: The Resurrection of the Iraqi Armed Forces and the Military Defeat of ISIS’, Journal of Strategic Studies, 45(1) (2022), 96–118. Brose, Christian, The Kill Chain: Defending America in the Future of High-Tech Warfare (New York: Hachette, 2020). Brown, Sara, ‘Why Neural Net Pioneer Geoffrey Hinton is Sounding the Alarm on AI’, MIT Sloan School of Management, 23 May 2023. https://mitsloan.mit.edu/ideas-made-to-matter/why-neural-net-pioneergeoffrey-hinton-sounding-alarm-ai. Buchan, Iain, ‘Civic Informatics in Covid-19 Response: Clarifying Mass Testing and Reopening Mass Gathering’. PowerPoint presentation, 30 April 2022.
———, ‘Closing the Data-Action Gap for Better Health and Care: A Civic Blueprint’. PowerPoint presentation, 16 June 2022. Buchanan, Ben, and Andrew Imbrie, The New Fire: War, Peace and Democracy in the Age of AI (Cambridge, MA: MIT Press, 2022). Bumiller, Elisabeth, and Thom Shanker, ‘Panetta Warns of Dire Threat of Cyber Attack on U.S.’, The New York Times, 12 October 2012. https://www.nytimes.com/2012/10/12/world/panetta-warns-of-dire-threat-ofcyberattack.html. Calcara, Antonio, Andrea Gilli, Mauro Gilli, Raffaele Marchetti, and Ivan Zaccagnini, ‘Why Drones Have not Revolutionized War; The Enduring Hider-Finder Competition in Air Warfare’, International Security, 46(4) (2022), 130–71. Cantwell Smith, Brian, The Promise of Artificial Intelligence: Reckoning and Judgment (Cambridge, MA: MIT Press, 2019). Chafkin, Max, The Contrarian: Peter Thiel and Silicon Valley’s Pursuit of Power (London: Bloomsbury, 2021). Chernobrov, Daniel, ‘Diasporas as Cyberwarriors: Infopolitics, Participatory Warfare and the 2020 Karabakh War’, International Affairs, 98(2) (2002), 631–51. Copp, Tara, ‘Elon Musk Blocking Starlink to Stop Ukraine Attack Troubling for DoD’, Military Times, 12 September 2023. https://www.militarytimes.com/news/yourmilitary/2023/09/12/elon-musk-blocking-starlink-to-stop-ukraine-attacktroubling-for-dod. Coppola, Daniela, ‘Annual Net Sales Revenue of Amazon from 2006 to 2023, by Segment’, Statista, 22 May 2024. https://www.statista.com/statistics/266289/net-revenue-of-amazon-by-region. Crawford, Kate, Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence (London: Yale University Press, 2021). Cukier, Kenneth, Viktor Mayer-Schönberger, and Francis de Vericourt, Framers: Human Advantage in an Age of Technology and Turmoil (London: WH Allen, 2021). Cummings, M. L., ‘Lethal Autonomous Weapons: Meaningful Human Control or Meaningful Human Certification?’, IEEE Technology and Society Magazine, 38(4) (2019), 20–26. Daniels, Owen J., ‘How Revisiting Naval Aviation’s Lessons Can (and Cannot) Inform Military AI Innovation’ (opinion), Breaking Defense, 25 August 2023. https://breakingdefense.com/2023/08/how-revisiting-naval-aviations-lessonscan-and-cannot-inform-military-ai-innovation. Danielson, Tone, Making Warriors in a Global Era: An Ethnographic Study of the Norwegian Naval Special Operations Commando (Lanham, MD: Lexington Books, 2018). DARPA, ‘AlphaDogfight Trials Foreshadow Future of Human-Machine Symbiosis’ (news release), 26 August 2020. https://www.darpa.mil/news-events/2020-0826.
Dastin, Jeffrey, ‘Palantir Lands 75 Million Pound Deal with British Military’, Reuters, 21 December 2022. https://www.reuters.com/business/aerospacedefense/palantir-lands-75-million-pound-deal-with-british-military-2022-12-21. Davies, Alex, ‘Google’s Self-driving Car Caused its First Crash’, Wired, 29 February 2016. https://www.wired.com/2016/02/googles-self-driving-car-may-causedfirst-crash. Davies, Harry, Bethan McKernan, and Dan Sabbagh, ‘ “The Gospel”: How Israel Uses AI to Select Bombing Targets in Gaza’, The Guardian, 1 December 2023. https://www.theguardian.com/world/2023/dec/01/the-gospel-how-israel-usesai-to-select-bombing-targets. Defence Artificial Intelligence Centre, ‘Defence Artificial Intelligence Centre— GOV.UK’, accessed 29 October 2024. https://www.gov.uk/government/groups/defence-artificial-intelligence-centre. Defense Innovation Unit, ‘About DIU’, https://www.diu.mil/about. ———, ‘Annual Report FY 2022’, accessed 24 October 2024. https://downloads.ctfassets.net/3nanhbfkr0pc/5guJIhcMGwIgoop4z9r5QM/89f4 4fe62f981e5f0d28932618719196/DIU_Annual_Report_FY22_Final_0131.pdf. de Heer, Paolo, Nico de Reus, Lucia Tealdi, and Philip Kerbusch, ‘Intelligence Augmentation for Urban Warfare Operation Planning Using Deep Reinforcement Learning’, in Pham, Tien (ed.), Proceedings of SPIE—The International Society for Optical Engineering, 11006, 35. Bellingham, WA: International Society for Optics and Photonics, 2019. https://www.researchgate.net/publication/333073489_Intelligence_augmentati on_for_urban_warfare_operation_planning_using_deep_reinforcement_learning . del Rey, Jason, ‘How Robots are Transforming Amazon Warehouse Jobs—for Better and Worse’, Vox, 11 December 2019. https://www.vox.com/recode/2019/12/11/20982652/robots-amazonwarehouse-jobs-automation. DeVries, Kelly, ‘Catapults Are Not Atomic Bombs: Towards a Redefinition of “Effectiveness” in Premodern Military Technology’, War in History, 4(4) (1997), 454–70. ———, ‘Gunpowder Weaponry and the Rise of the Early Modern State’, War in History, 5(2) (1998), 127–45. Dixon, Stacy Jo, ‘Annual Revenue Generated by Meta Platforms from 2009 to 2023’, Statista, 16 February 2024. https://www.statista.com/statistics/268604/annual-revenue-of-facebook. Douhet, Giulio, The Command of the Air (Tuscaloosa: University of Alabama, 2009). Dreyfus, Hubert, What Computers Still Can’t Do: A Critique of Artificial Reason (Cambridge, MA: MIT Press, 1992). du Sautoy, Marcus, The Creativity Code: Art and Innovation in the Age of AI (Cambridge, MA: The Belknap Press, 2019).
DUST, ‘Sci-Fi Short Film: “Slaughterbots”’, YouTube video, 7:58, 17 October 2019. https://www.youtube.com/watch?v=O-2tpwW0kmU. Eagle, Nathan, and Alex Pentland, ‘Reality Mining: Sensing Complex Social Systems’, Personal and Ubiquitous Computing, 10 (2006), 255–68. The Economist, ‘The Age of Pseudocognition,’ 22 April 2023, 18. ———, ‘AI for All’, 27 January 2024, 10. ———, ‘Bakhmut and the Spirit of Verdun’, 3 June 2023, 30. ———, ‘Chatbots for the Bottom Four Billion’, 27 January 2024, 16–17. ———, ‘How to Worry Wisely about AI’, 22 April 2023, 9. ———, ‘Model Major-General’, 22 June 2024, 18. ———, ‘A Nest of Wipers’, 3 December 2023, 76. ———, ‘Predict and Survive’, 23 July 2022, 67–68. ———, ‘The Race is on to Control the Global Supply Chain for AI Chips’, 30 July 2024. https://www.economist.com/schools-brief/2024/07/30/the-race-is-on-tocontrol-the-global-supply-chain-for-ai-chips. ———, ‘The Satellites that Saved Ukraine’, 7 January 2023, 13. ———, ‘A Second Flight’, 30 September 2023, 17. ———, ‘The Unspooked Spook’, 24 June 2023, 29–30. Edge Delta, ‘Breaking Down the Numbers: How Much Data Does the World Create Daily in 2024?’, 11 March 2024. https://edgedelta.com/company/blog/howmuch-data-is-created-per-day. Evron, Yoram, ‘4IR Technologies in the Israel Defence Forces: Blurring the Boundaries’, Journal of Strategic Studies, 44(4) (2022), 572–3. Evron, Yoram, and Richard Bitzinger, The Fourth Industrial Revolution and MilitaryCivil Fusion: A New Paradigm for Military Innovation? (Cambridge, MA: Cambridge University Press, 2023). Finbox, ‘R&D Expense for Amazon.com Inc’, accessed 29 October 2024, https://finbox.com/NASDAQGS:AMZN/explorer/rd_exp. Fitzpatrick, Michael, ‘Artificial Intelligence and Nuclear Command and Control’, Survival, 61(3) (2019), 81–92. Fox, Amos, ‘Manoeuvre is Dead? Understanding the Conditions and Components of Warfighting’, The RUSI Journal, 166(6–7) (2021), 10–18. Ford, Martin, Architects of Intelligence: The Truth about AI from the People Building It (Birmingham: Packt Publishing, 2018). Ford, Matthew, and Andrew Hoskins, Radical War: Data, Attention and Control in the Twenty-first Century (London: Hurst and Co., 2022). Frantzman, Seth, Drone Wars: Pioneers, Killing Machines, Artificial Intelligence, and the Battle for the Future (New York: Post Hill Press/Bombardier, 2021). Future of Life Institute, ‘Autonomous Weapons Open Letter: AI & Robotics Researchers’, 28 July 2015. https://www.stopkillerrobots.org/wpcontent/uploads/2013/03/FLI_LtrJuly2015.pdf. Gady, Franz-Stefan, and Michael Kofman, ‘Ukraine’s Strategy of Attrition’, Survival, 65(2) (2023), 7–22.
Garcia, Denise, The AI Military Race: Common Good Governance in the Age of Artificial Intelligence (Oxford: Oxford University Press, 2023). ———, ‘Lethal Artificial Intelligence and Change’, International Studies Review, 20(2) (2018), 334–41. Glicken Turnley, Jessica, Kobi Michael, and Eyal Ben-Ari (eds.), Special Operations Forces in the 21st Century: Perspectives from the Social Sciences (London: Palgrave, 2017). Goldfarb, Avi, and Jon R. Lindsay, ‘Prediction and Judgment: Why Artificial Intelligence Increases the Importance of Humans in War’, International Security, 46(3) (Winter 2021/22), 7–50. https://direct.mit.edu/isec/article/46/3/7/109668/Prediction-and-JudgmentWhy-Artificial. Gomes, Lee, ‘Hidden Obstacles for Google’s Self-driving Cars’, MIT Technology Review, 28 August 2014. https://www.technologyreview.com/2014/08/28/171520/hidden-obstacles-forgoogles-self-driving-cars. Gonzalez, Roberto, War Virtually: The Quest to Automate Conflict, Militarize Data, and Predict the Future (Oakland, CA: University of Oakland, 2022). Graham, Stephen, Cities under Siege: The New Military Urbanism (London: Verso Books, 2010). Green, Mark, Marta García-Fiñana, Benn Barr, Girvan Burnside, Christopher Cheyne, David Hughes, Matthew Ashton, Sally Sheard, and Iain Buchan, ‘Evaluating Social and Spatial Inequality of Large Scale Rapid Lateral Flow SARs-CoV-2 Antigen Testing in COVID-19 Management: An Observational Study of Liverpool, UK (November 2020 to January 2021)’ The Lancet Regional Health, Europe, vol. 6 (2021), 100107. https://www.thelancet.com/journals/lanepe/article/PIIS2666-7762(21)000843/fulltext. Greenberg, Andy, ‘Mining Human Behavior at MIT,’ Forbes, 12 August 2010. https://www.forbes.com/forbes/2010/0830/e-gang-mit-sandy-pentland-darpasociometers-mining-reality.html. Greene, Kate, ‘Reality Mining’, MIT Technology Review, 19 February 2008. https://www.technologyreview.com/technology/tr10-reality-mining. The Guardian, ‘Israel Defence Forces’ Response to Claims about Use of “Lavender” AI Database in Gaza’, 3 April 2024. https://www.theguardian.com/world/2024/apr/03/israel-defence-forcesresponse-to-claims-about-use-of-lavender-ai-database-in-gaza. Haas, Michael-Carl, and Sophie-Charlotte Fischer, ‘The Evolution of Targeted Killing Practices’, Contemporary Security Policy, 38(2) (2017), 281–306. Hall, Bert, and Kelly DeVries, ‘Review of The Military Revolution: Military Innovation and the Rise of the West, 1500–1800, by G. Parker’, Technology and Culture, 31(3) (1990), 500–507.
Halpern, Sue, ‘The Coming Tech Autocracy’ [Review of AI Needs You: How We Can Change AI’s Future and Save Our Own, by Verity Harding; Taming Silicon Valley: How We Can Ensure That AI Works for Us, by Gary Marcus; The Mind’s Mirror: Risk and Reward in the Age of AI, by Daniela Rus and Gregory Mone; and Code Dependent: Living in the Shadow of AI, by Madhumita Murgia], The New York Review of Books, LXXI (17), 7 November 2024, 44–6. https://www.nybooks.com/articles/2024/11/07/the-coming-tech-autocracy-aineeds-you-verity-harding. Hambling, David, ‘AI is Helping the US Air Force to Decide Which Targets to Strike’, New Scientist, 28 September 2021. https://www.newscientist.com/article/2291586-ai-is-helping-us-air-force-todecide-which-targets-to-strike. ———, Swarm Troopers: How Small Drones Will Conquer the World (Archangel Ink, 2015). Harper, Jon, ‘Pentagon Requesting More than $3B for AI, JADC2’, DefenseScoop, 13 March 2023. https://defensescoop.com/2023/03/13/pentagon-requestingmore-than-3b-for-ai-jadc2. Harwell, Drew, ‘Ukraine is Scanning Faces of Dead Russians, then Contacting the Mothers,’ The Washington Post, 15 April 2022, https://www.washingtonpost.com/technology/2022/04/15/ukraine-facialrecognition-warfare. Hays, Constance, ‘What Wal-Mart Knows about Customers’ Habits’, The New York Times, 14 November 2004. https://www.nytimes.com/2004/11/14/business/yourmoney/what-walmartknows-about-customers-habits.html. Headquarters Department of the Army, Field Manual 3-0 Operations, October 2022. https://armypubs.army.mil/epubs/DR_pubs/DR_a/ARN36290-FM_3-0000-WEB-2.pdf. Henriksen, Rune, ‘Warriors in Combat—What Makes People Actively Fight in Combat?’, Journal of Strategic Studies, 30(2) (2007), 187–223. Hockenhull, James, ‘How Open Source Intelligence Has Shaped the Russia-Ukraine War’, Royal United Services Institute for Defence and Security Studies, speech, 7 November 2022. https://www.gov.uk/government/speeches/how-opensource-intelligence-has-shaped-the-russia-ukraine-war. Hoijtink, Marijn, ‘ “Prototype Warfare”: Innovation, Optimisation, and the Experimental Way of Warfare’, European Journal of International Security, 7(3) (2022), 322–36. Horn, Bernd, No Ordinary Men: Special Operations Forces Missions in Afghanistan (Toronto: Dundurn, 2014). Horowitz, Michael, ‘When Speed Kills: Lethal Autonomous Weapon Systems, Deterrence and Stability’, Journal of Strategic Studies, 42(6) (2019), 764–88. Horowitz, Michael, Gregory Allen, Elsa Kania, and Paul Scharre, Strategic Competition in an Era of Artificial Intelligence, Center for a New American
Security, 25 July 2018. https://www.cnas.org/publications/reports/strategiccompetition-in-an-era-of-artificial-intelligence. Hoskins, Andrew, and Pavel Shchelin, ‘The War Feed: Digital War in Plain Sight’, American Behavioral Scientist, 67(3) (2023), 449–63. Hsu, Jeremy, ‘AI Discovers New Class of Antibiotics to Kill Drug-Resistant Bacteria’, New Scientist, 29 December 2023. https://www.newscientist.com/article/2409706-ai-discovers-new-class-ofantibiotics-to-kill-drug-resistant-bacteria. Hume, David, A Treatise on Human Nature (Oxford: Oxford University Press, 1985). Hume, David, An Enquiry concerning Human Understanding (Cambridge: Hackett Publishing Company, 1977). Hunter, Cameron, and Bleddyn Bowen, ‘We’ll Never Have a Model of an AI MajorGeneral: Artificial Intelligence, Command Decisions, and Kitsch Visions of War’, Journal of Strategic Studies, 47(1) 2024, 128–9. https://www.tandfonline.com/doi/full/10.1080/01402390.2023.2241648. HYPR, ‘What is NotPetya? 5 Fast Facts’, accessed 29 October 2024. https://www.hypr.com/security-encyclopedia/notpetya. Ignatius, David, ‘How the Algorithm Tipped the Balance in Ukraine’ (opinion), The Washington Post, 19 December 2022. https://www.washingtonpost.com/opinions/2022/12/19/palantir-algorithmdata-ukraine-war. International Institute for Strategic Studies, The Military Balance 2024 (London: Taylor & Francis, 2024). Isaacson, Walter, Elon Musk (London: Simon & Schuster, 2023). Israel Defence, ‘The Future of Artificial Intelligence in the IDF’, 7 February 2017. https://www.israeldefense.co.il/en/node/30189. Jackson, Eric M., The PayPal Wars: Battles with eBay, the Media, the Mafia, and the Rest of Planet Earth (Los Angeles: World Ahead Publishing, 2004). Janes, ‘Janes at the Heart of the British Army’s First AI-driven Battlefield Operation’ (news release), 24 August 2021. https://www.janes.com/osintinsights/defence-news/defence/janes-at-the-heart-of-the-british-army’s-first-aidriven-battlefield-operation. Jensen, Benjamin, and Dan Tadross, ‘How Large Language Models Can Revolutionize Military Planning’, War on the Rocks, 12 April 2023. https://warontherocks.com/2023/04/how-large-language-models-canrevolutionize-military-planning. Jensen, Benjamin, Christopher Whyte, and Scott Cuomo, ‘Algorithms at War: The Promise, the Peril and Limits of Artificial Intelligence’, International Studies Review, 22(3) (2020), 526–50. Jewish Institute for National Security America (JINSA), ‘Gaza Conflict 2021: Observations and Lessons’, 28 October 2021.
https://jinsa.org/jinsa_report/gaza-conflict-2021-assessment-observations-andlessons. Johnson, David E., M. Wade Markel, and Brian Shannon, The 2008 Battle of Sadr City: Reimagining Urban Combat (Santa Monica, CA: RAND, 2013). Johnson, James, The AI Commander: Centaur Teaming, Command, and Ethical Dilemmas (Oxford University Press, 2024). ———, ‘Inadvertent Escalation’, European Journal of International Security, 7(3) (2022), 337–59. Johnson, Matthew, and Alonso Vera, ‘No AI is an Island: The Case for Teaming Intelligence’, AI Magazine, 40(1) (2019), 17–27. https://doi.org/10.1609/aimag.v40i1.2842. Johnson, Matthew, Jeffrey Mark Bradshaw, Robert Hoffman, and Paul J. Feltovich, ‘Seven Cardinal Virtues of Human-Machine Teamwork: Examples from the DARPA Robotic Challenge’, IEEE Intelligent Systems, 29(6) (2014), 74–80. Johnson, Matthew, Brandon Shrewsbury, Sylvain Bertrand, and Duncan Calvert, ‘Team IHMC’s Lessons Learned from the DARPA Robotics Challenge: Finding Data in the Rubble’, Journal of Field Robotics, 34(2) (2016), http://dx.doi.org/10.1002/rob.21674. The Joint Staff, Description of the National Military Strategy 2018. https://www.jcs.mil/Portals/36/Documents/Publications/UNCLASS_2018_Nation al_Military_Strategy_Description.pdf. Joshi, Shashank, ‘The War in Ukraine Shows How Technology is Changing the Battlefield’, special report, The Economist, 3 July 2023. https://www.economist.com/special-report/2023/07/03/the-war-in-ukraineshows-how-technology-is-changing-the-battlefield. Kania, Elsa, Battlefield Singularity: Artificial Intelligence, Military Revolution and China’s Future Military Power, Center for a New American Security, 28 November 2017. https://www.cnas.org/publications/reports/battlefieldsingularity-artificial-intelligence-military-revolution-and-chinas-future-militarypower. ———, Chinese Military Innovation in Artificial Intelligence: Hearing of the USChina Economic and Security Review Commission, Center for a New American Security, 7 June 2019. https://www.cnas.org/publications/congressionaltestimony/chinese-military-innovation-in-artificial-intelligence. Kilcullen, David, ‘Emerging Patterns of Adversary Urban Operations’, RUSI Urban Warfare Conference Session 3, 2 February 2018. King, Anthony, Command: The Twenty-first-Century General (Cambridge: Cambridge University Press, 2019). ———, ‘The Special Air Service and the Concentration of Military Power’, Armed Forces & Society, 35(4) (2009), 646–66. ———, Urban Warfare in the Twenty-first Century (Cambridge: Polity Press, 2021). ———, ‘What is Special about Special Operations Forces?’, in Jessica Glicken Turnley, Kobi Michael, and Eyal Ben-Ari (eds.), Special Operations Forces in the
21st Century: Perspectives from the Social Sciences (London: Palgrave, 2017), 273–84. Kinsey, Christopher, Corporate Soldiers and International Security: The Rise of Private Military Companies (London: Routledge, 2006). ———, Private Contractors and the Reconstruction of Iraq: Transforming Military Logistics (London: Routledge, 2009). Kinsey, Christopher, and Malcolm Patterson (eds.), Contractors and War: The Transformation of United States’ Expeditionary Operations (Stanford, CA: Stanford University Press, 2012). Kissinger, Henry, Eric Schmidt, and Daniel Huttenlocher, The Age of AI and Our Human Future (London: John Murray, 2021). Krahmann, Elke, States, Citizens and the Privatization of Security (Cambridge: Cambridge University Press, 2010). Kurth Cronin, Audrey, ‘Open Source Technology and Public-Private Innovation Are the Key to Ukraine’s Strategic Resilience’, War on the Rocks, 25 August 2023. https://warontherocks.com/2023/08/open-source-technology-and-publicprivate-innovation-are-the-key-to-ukraines-strategic-resilience. Kurzweil, Ray, The Singularity is Near: When Humans Transcend Biology (London: Duckworth: 2005). ———, The Singularity is Nearer: When We Merge with AI (Oxford: Bodley Head, 2024). Landau, Susan, ‘Cyberwar in Ukraine: What You See is Not What’s Really There’, Lawfare, 30 September 2022. https://www.lawfaremedia.org/article/cyberwarukraine-what-you-see-not-whats-really-there. Laricchia, Frederica, ‘Global Revenue of Apple from 2004 to 2023’, Statista, 21 May 2024. https://www.statista.com/statistics/265125/total-net-sales-of-applesince-2004. Larson, Erik, The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do (Cambridge, MA: Belknap Press, 2021). Lee, Kai-Fu, AI Superpowers: China, Silicon Valley, and the New World Order (Boston: Houghton Mifflin Harcourt, 2018), 71. Lele, Ajey, Disruptive Technologies for the Militaries and Security (Singapore: Springer, 2019). Lindsay, Jon R., ‘Stuxnet and the Limits of Cyber Warfare’, Security Studies, 22(3) (2013), 365–404. Longoni, Chiara, and Carey Morewedge, ‘AI Can Outperform Doctors. So Why Don’t Patients Trust It?’, Harvard Business Review, 30 October 2019. https://hbr.org/2019/10/ai-can-outperform-doctors-so-why-dont-patients-trustit. Lopez, Nick, and Kyle Atwell, ‘Artificial Intelligence in Counterterrorism and Counterinsurgency, with Retired Gen. Stanley McChrystal and Dr. Anshu Roy’, Modern War Institute, Irregular Warfare Podcast, 51:09, 1 January 2021.
https://mwi.usma.edu/artificial-intelligence-in-counterterrorism-andcounterinsurgency-with-retired-gen-stan-mcchrystal-and-dr-anshu-roy. Lovelock, James, The Novacene: The Coming Age of Hyperintelligence (London: Penguin Books, 2020). Luttwak, Edward, and Eitan Shamir, The Art of Military Innovation: Lessons from the Israel Defense Forces (Cambridge, MA: Harvard University Press, 2023). Machiavelli, Niccolo, The Prince, translated by G. Bull (London: Penguin Books, 1999). Mackenzie, Donald, Trading at the Speed of Light: How Fast Algorithms Are Transforming Financial Markets (Princeton, NJ: Princeton University Press, 2021). Macrotrends, ‘Alphabet Research and Development Expenses 2010-2024’, accessed 29 October 2024. https://www.macrotrends.net/stocks/charts/GOOGL/alphabet/researchdevelopment-expenses. ———, ‘Meta Platforms Research and Development Expenses 2010-2024’, accessed 29 October 2024. https://www.macrotrends.net/stocks/charts/META/meta-platforms/researchdevelopment-expenses. ———, ‘Microsoft Research and Development expenses 2010-2024’, accessed 29 October 2024, https://www.macrotrends.net/stocks/charts/MSFT/microsoft/researchdevelopment-expenses. Mallaby, Sebastian, The Power Law: Venture Capital and the Making of the New Future (London: Allen Lane, 2022). Manson, Katrina, ‘The US Army is Taking Generative AI out for a Spin’, Bloomberg, 5 July 2023. https://www.bloomberg.com/news/newsletters/2023-07-05/theus-military-is-taking-generative-ai-out-for-a-spin. Marcus, Gary, and Ernest Davis, Rebooting AI: Building Artificial Intelligence We Can Trust (New York: Pantheon Books, 2019). Marks, Robert, The Case for Killer Robots: Why America’s Military Needs to Continue to Development of Lethal AI (Seattle: Discovery Institute Press, 2020). Martelle, Michael, and Audrey Alexander, ‘Operation Glowing Symphony: The Missing Piece in the US Online Counter-ISIS Campaign’, in Michael Sexton and Eliza Campbell (eds.), Cyber War and Cyber Peace: Digital Conflict in the Middle East (London: I.B. Tauris, 2022), 119–136. Marx, Karl, Capital: A Critique of Political Economy, vol. 1, translated by Samuel Moore and Edward Aveling (Chicago: Charles H. Kerr and Co., 1932). Mayer-Schönberger, Viktor, and Kenneth Cukier, Big Data: A Revolution That Will Transform How We Live, Work, and Think (New York: Mariner Books, Houghton Mifflin Harcourt, 2014).
McChrystal, Stanley, with Tantum Collins, David Silverman, and Chris Fussell, Team of Teams: New Rules of Engagement for a Complex World (London: Penguin Books, 2015). Meijer, Albert, and Martijn Wessels, ‘Predictive Policing: Review of Benefits and Drawbacks’, International Journal of Public Administration, 42(12) (2019), 1031–9. Miller, Sergio, ‘The Battle for Bakhmut: Wagner Trench Warfare Tactics’, Wavell Room, 15 December 2022. https://wavellroom.com/2022/12/15/the-battle-forbakhmut. Mitchell, Melanie, Artificial Intelligence: A Guide for Thinking Humans (London: Pelican, 2019). Mollick, Ethan, Co-Intelligence: Living and Working with AI (London: WH Allen, 2024). Moran, Christopher, Joe Burton, and George Christou, ‘The US Intelligence Community, Global Security, and AI: From Secret Intelligence to Smart Spying’, Journal of Global Security Studies, 2(2) (2023), 1–18. The Mosul Study Group, ‘What the Battle of Mosul Teaches the Force’, US Army, 2018. https://www.armyupress.army.mil/Portals/7/Primer-on-UrbanOperation/Documents/Mosul-Public-Release1.pdf. Mueller, Hannes, Andre Groeger, Jonathan Hersh, Andrea Matranga, and Joan Serrat, ‘Monitoring War Destruction from Space Using Machine Learning’, Proceedings of the National Academy of Sciences of the United States of America, 118(23) (2021), e2025400118. Muldoon, James, Mark Graham, and Callum Cant, Feeding the Machine: The Hidden Human Labour Powering AI (Edinburgh: Canongate, 2024). National Cyber Security Centre, ‘UK and Allies Expose Russian Intelligence Services for Cyber Campaign of Attempted Political Interference’, 7 December 2023. https://www.ncsc.gov.uk/news/uk-and-allies-expose-cyber-campaignattempted-political-interference. National Security Commission on Artificial Intelligence (NSCAI), Final Report, 2021. https://nscai.wpenginepowered.com/wp-content/uploads/2021/03/Full-ReportDigital-1.pdf, accessed 11 October 2023. NATO, Draft Concept 0.8: Operating Concept for Multi-Domain Operations in the Urban Environment, January 2023. ———, ‘Summary of the NATO Artificial Intelligence Strategy’, 2022. https://www.nato.int/cps/en/natohq/official_texts_187617.htm. Neads, Alex, Theo Farrell, and David Galbreath, ‘Evolving towards Military Innovation: AI and the Australian Army’, Journal of Strategic Studies, 24 April 2023. https://doi.org/10.1080/01402390.2023.2200588. Neads, Alex, David Galbreath, and Theo Farrell, From Tools to Teammates: Human-Machine Teaming and the Future of Command and Control in the Australian Army, Australian Army Research Centre, 20 September 2021.
https://researchcentre.army.gov.au/sites/default/files/AARC%20Occasional%20 Paper%20No.7%20-%20From%20Tools%20to%20Teammates.pdf. Nowotny, Helga, In AI We Trust: Power, Illusion and Control of Predictive Algorithms (Cambridge: Polity Press, 2021). Overy, Richard, The Bombing War: Europe, 1939–1945 (London: Penguin Books, 2014). Parker, Geoffrey, The Military Revolution: Military Innovation and the Rise of the West, 1500–1800 (2nd edn, Cambridge: Cambridge University Press, 1996). Payne, Kenneth, ‘Artificial Intelligence: A Revolution in Strategic Affairs’, Survival, 60(5) (2018), 7–32. ———, I, Warbot: The Dawn of Artificially Intelligent Conflict (London: Hurst, 2021). Pekkanen, Saadia M., Setsuko Aoki, and John Mittleman, ‘Small Satellites, Big Data: Uncovering the Invisible in Maritime Security’, International Security, 47(2) (2022), 177–216. Pellerin, Cheryl, ‘Work: Human-Machine Teaming Represents Defense Technology Future’, US Department of Defense, 8 November 2015. https://www.defense.gov/News/News-Stories/Article/Article/628154/workhuman-machine-teaming-represents-defense-technology-future. Pentland, Alex, David Lazer, Devon Brewer, and Tracy Heibeck, ‘Using Reality Mining to Improve Public Health and Medicine’, Student Health Technology Information, 149 (2009), 93–102. Peterson, Magnus, ‘Small States and Autonomous Systems: The Scandinavian Case’, Journal of Strategic Studies, 44(4) (2022), 594–612. Pettyjohn, Stacie, Evolution not Revolution: Drone Warfare in Russia’s 2022 Invasion of Ukraine (Washington, DC: Center for a New American Security, February 2024). Pita, James, Manish Jain, Fernando Ordóñez, Christopher Portway, Milind Tambe, Craig Western, Praveen Paruchuri, and Sarit Kraus, ‘Using Game Theory for Los Angeles Airport Security’, Artificial Intelligence Magazine, 30(1) (2009), 43–57. Prier, Jarred, ‘Commanding the Trend: Social Media as Information Warfare’, Strategic Studies Quarterly, 11(4) (2017), 50–85. Raska, Michael, ‘The Sixth RMA Wave: Disruption in Military Affairs?’, Journal of Strategic Studies, 44(4) (2022), 456–97. Register Dynamics, ‘The Exponential Growth of Computational Power behind AI’, accessed 29 October 2024. https://www.register-dynamics.co.uk/blog/theexponential-growth-of-computational-power-behind-ai. Reilly, Briana, ‘Pentagon’s Priority on AI Spending Could Shield it from Cuts’, Roll Call, 5 March 2024. https://rollcall.com/2024/03/05/pentagons-priority-on-aispending-could-shield-it-from-cuts. Rid, Thomas, Active Measures: The Secret History of Disinformation and Political Warfare (London: Profile Books, 2021).
———, ‘Cyber War Will Not Take Place’, Journal of Strategic Studies, 35(1) (2012), 5–32. https://doi.org/10.1080/01402390.2011.608939. Ripley, Tim, Operation Aleppo: Russia’s War in Syria (Lancaster: Telic-Herrick Publications, 2018). Roberts, Michael, ‘The Military Revolution, 1560–1660’, in Michael Roberts, Essays in Swedish History (London: Arnold, 1967), 56–81. Rubin, Eliran, ‘Tiny IDF Unit is Brains behind Israeli Army Artificial Intelligence’, Haaretz, 15 August 2017. https://www.haaretz.com/israel-news/2017-08-15/tyarticle/tiny-idf-unit-is-brains-behind-israeli-army-artificial-intelligence/0000017fe35b-d7b2-a77f-e35fc8f40000. Russell, Stuart, ‘AI and Warfare’, Reith Lecture 2021. https://www.bbc.co.uk/programmes/m00127t9. ———, Human Compatible: Artificial Intelligence and the Problem of Control (London: Allen Lane, 2019). Ryan, Mick, Human-Machine Teaming for Future Ground Forces (Washington, DC: Center for Strategic and Budgetary Assessments, 2018). Sanders, General Sir Patrick, ‘Chief of the General Staff Speech at RUSI Land Warfare Conference’, 28 June 2022. https://www.gov.uk/government/speeches /chief-the-general-staff-speech-at-rusi-land-warfare-conference. Scharre, Paul, Army of None: Autonomous Weapons and the Future of War (New York: W.W. Norton and Co., 2019). ———, Four Battlegrounds: Power in the Age of Artificial Intelligence (London: W.W. Norton and Co., 2023). Schechner, Sam, and Daniel Michaels, ‘Ukraine Has Digitized its Fighting Forces on a Shoestring’, The Wall Street Journal, 3 January 2023. https://www.wsj.com/articles/ukraine-has-digitized-its-fighting-forces-on-ashoestring-11672741405. Schmidt, Eric, “Innovation Power: Why Technology Will Define the Future of Geopolitics”, Foreign Affairs, March/April 2023. https://www.foreignaffairs.com/united-states/eric-schmidt-innovation-powertechnology-geopolitics. ———, ‘I Used to Run Google. Silicon Valley Could Lose to China’ (opinion), The New York Times, 27 February 2020. https://www.nytimes.com/2020/02/27/opinion/eric-schmidt-ai-china.html. Shamir, Eitan, and Eyal Ben-Ari, ‘The Rise of Special Operations Forces: Generalized Specialization, Boundary Spanning and Military Autonomy’, Journal of Strategic Studies, 41(3) (2018), 335–71. Shanahan, John (Jack) N.T., ‘AI—a Game Changer and Decisive Edge’, Chief Digital and Artificial Intelligence Office, 11 September 2019. https://www.ai.mil/blog_09_11_19.html. ———, ‘Artificial Intelligence, Ubiquitous Sensors, and Human-Machine Teams: How AI Will Transform the Intelligence Cycle’, in William J. Lahneman and
Florina Cristiana Matei (eds.), Intelligence and Technology: Trends, Challenges, and Choices (Boulder, CO: Lynne Rienner Publishers, forthcoming). Sheridan, B., ‘A Trillion Points of Data’, Newsweek, 9 March 2009, 35–7. Singer, Peter, Corporate Warriors: The Rise of the Privatized Military Industry (Ithaca, NY: Cornell University Press, 2003). Smith, Brad, ‘Defending Ukraine: Early Lessons’, Microsoft on the Issues (blog), 22 June 2022. https://blogs.microsoft.com/on-the-issues/2022/06/22/defendingukraine-early-lessons-from-the-cyber-war. Smith, Eugene, ‘The New Condottieri and US Policy: The Privatization of Conflict and its Implications’, Parameters, 32(4) (2002), 104–19. Soare, Simona R., Pavneet Singh, and Meia Nouwens, Software-Defined Defence: Algorithms at War, International Institute for Strategic Studies, February 2023. https://www.iiss.org/globalassets/media-library---content-migration/files/research-papers/iiss_software-defined-defence_17022023.pdf. Soesanto, Stefan, ‘Ukraine’s IT Army’, Survival, 65(3) (2023), 93–106. Sparkes, Matthew, ‘DeepMind’s Protein-Folding AI Cracks Biology’s Biggest Problem’, New Scientist, 28 July 2022. https://www.newscientist.com/article/2330866-deepminds-protein-folding-aicracks-biologys-biggest-problem. Spearin, Christopher, Private Military and Security Companies and States: Force Divided (London: Palgrave, 2017). Special Competitive Studies Project (SCSP), ‘Conflict and Competition in the Digital Age: Lessons from Ukraine’, 2-2-2, 21 April 2022. https://scsp222.substack.com/p/conflict-and-competition-in-the-digital. ———, ‘Dr. Alex Karp of Palantir Reflects on the Lessons Learned from the War in Ukraine’, 2-2-2, 24 February 2023. https://scsp222.substack.com/p/specialguest-edition-dr-alex-karp. ———, Generative AI: The Future of Innovation Power, September 2023. https://www.scsp.ai/wp-content/uploads/2023/09/GenAI-web.pdf. Spencer, John, ‘The Battle of Bakhmut’, Modern War Institute, Urban Warfare Project Podcast, 35:38, 17 March 2023, https://mwi.westpoint.edu/the-battleof-bakhmut. ———, ‘Stealing the Enemy’s Urban Advantage: The Battle of Sadr City’, Modern War Institute, 31 January 2019, https://mwi.westpoint.edu/stealing-enemysurban-advantage-battle-sadr-city. Stanley-Lockman, Zoe, ‘From Closed to Open Systems: How the US Military Services Pursue Innovation’, Journal of Strategic Studies, 44(4) 2022, 480–514. Stanley-Lockman, Zoe, and Edward Hunter Christie, ‘An Artificial Intelligence Strategy for NATO’, 25 October 2021. https://www.nato.int/docu/review/articles/2021/10/25/an-artificial-intelligencestrategy-for-nato/index.html. Stone, Brad, Amazon Unbound: Jeff Bezos and the Invention of a Global Empire (London: Simon & Schuster, 2021).
———, The Everything Store: Jeff Bezos and the Age of Amazon (London: Corgi, 2018). Suchman, Lucy, ‘Algorithmic Warfare and the Reinvention of Accuracy’, Critical Studies on Security, 8(2) (2020), 175–87. Suleyman, Mustafa, with Michael Bhaskar, The Coming Wave: AI, Power and the Twenty-first Century’s Greatest Dilemma (London: The Bodley Head, 2023). Taylor, Paul, ‘Llamas, Pizzas, Mandolins: AI Doomerism’ [Review of The Coming Wave: Technology, Power and the Twenty-first Century’s Greatest Dilemma, by Mustafa Suleyman, and The World I See: Curiosity, Exploration and Discovery at the Dawn of AI, by Fei-Fei Li], London Review of Books, 46(6) (21 March 2024), 17–18. Temple-Raston, Dina, ‘How the U.S. Hacked ISIS’, NPR, 26 September 2019. https://www.npr.org/2019/09/26/763545811/how-the-u-s-hacked-isis. Thiel, Peter, with Blake Masters, Zero to One: Notes on Start-ups, or How to Build a Future (London: Virgin Books, 2014). Townsend, Stephen, ‘TRADOC Mad Scientist 2018 Megacities: 1.06 Scenarios, Foundations & Convergence / GEN Townsend’, TRADOC G-2 OE Enterprise, YouTube video, 36:31 (recording of Multidomain Battle in Megacities Conference, Fort Hamilton, New York, 3 April 2018). https://www.youtube.com/watch?v=ARz0l_evGAE. Tulumello, Simone, and Fabio Iapaolo, ‘Policing the Future, Disrupting Urban Policy Today; Predictive Policing, Smart City, and Urban Policy in Memphis (TN)’, Urban Geography, 43(11) (2021), 1–22. UK Foreign & Commonwealth Office, National Cyber Security Centre, and Lord (Tariq) Ahmad of Wimbledon KCMG, ‘Foreign Office Minister Condemns Russia for NotPetya Attacks’, 15 February 2018, https://www.gov.uk/government/news/foreign-office-minister-condemns-russiafor-notpetya-attacks. UK Ministry of Defence, Data Strategy for Defence, 27 September 2021. https://www.gov.uk/government/publications/data-strategy-for-defence/datastrategy-for-defence, accessed 11 October 2023. ———, Defence’s Response to a More Contested and Volatile World, CP 901, July 2023. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/a ttachment_data/file/1171269/Defence_Command_Paper_2023_Defence_s_resp onse_to_a_more_contested_and_volatile_world.pdf. ———, ‘Realising the Potential of Sustained Artificial Intelligence Development and Integration in Defence’, whitepaper by the Enhanced C2 Spearhead Team, Army Information Directorate, on behalf of the Defence Innovation Unit, 2021. University of Liverpool, Covid-SMART rapid antigen community testing evaluations, 23 December 2020. https://www.liverpool.ac.uk/research/researchthemes/infectious-diseases/coronavirus-research/covid-smart-pilot.
US Department of Defense, Summary of the Joint All-Domain Command and Control (JADC2) Strategy, March 2022. https://media.defense.gov/2022/Mar/17/2002958406/-1/-1/1/SUMMARY-OFTHE-JOINT-ALL-DOMAIN-COMMAND-AND-CONTROL-STRATEGY.PDF. ———, Summary of the 2018 National Defense Strategy of the United States of America: Sharpening the American Military’s Competitive Edge, 2018. https://dod.defense.gov/Portals/1/Documents/pubs/2018-National-DefenseStrategy-Summary.pdf. US Deputy Secretary of Defense, ‘Memo: Establishment of an Algorithmic Warfare Cross-Functional Team (Project Maven)’, 26 April 2017. https://dodcio.defense.gov/Portals/0/Documents/Project%20Maven%20DSD% 20Memo%2020170425.pdf. US Special Operations Command, Fiscal Year 2024, Budget Estimates, March 2023. https://comptroller.defense.gov/Portals/45/Documents/defbudget/fy2024/budg et_justification/pdfs/01_Operation_and_Maintenance/O_M_VOL_1_PART_1/SO COM_OP-5.pdf. Valdivia, Ana, Claudia Aradau, Tobias Blanke, and Sarah Perret, ‘Neither Opaque nor Transparent: A Transdisciplinary Methodology to Investigate Datafication at the EU Borders’, Big Data & Society, 9(2) (2022). https://doi.org/10.1177/20539517221124586. Vance, Ashlee, and Brad Stone, “Palantir: The War on Terror’s Secret Weapon,” Bloomberg, 22 November 2011. https://www.bloomberg.com/news/articles/2011-11-22/palantir-the-war-onterrors-secret-weapon. van Creveld, Martin, Command in War (Cambridge, MA: Harvard University Press, 1985). Van Puyvelde, Damien, Stephen Coulthart, and M. Shahriar Hossain, ‘Beyond the Buzzword: Big Data and National Security Decision-Making’, International Affairs, 93(6) (2017), 1397–416. Virilio, Paul, Speed and Politics (Los Angeles: Semiotext(e), 2006). Virilio, Paul, and Sylvere Lotringer, Pure War (Los Angeles: Semiotext(e), 1999). Visser, Matthew, ‘XVIII Airborne Corps Honors Outgoing Chief Technology Officer’, Defense Visual Information Service, 20 January 2023. https://www.dvidshub.net/news/437019/xviii-airborne-corps-honors-outgoingchief-technology-officer. von Clausewitz, Carl, On War, translated by Peter Paret and Michael Howard (Princeton, NJ: Princeton University Press, 1984). Waltzman, Rand, Lillian Ablon, Christian Curriden, Gavin S. Hartnett, Maynard A. Holliday, Logan Ma, Brian Nichiporuk, Andrew Scobell, and Danielle C. Tarraf, Maintaining the Competitive Advantage in Artificial Intelligence and Machine Learning (Santa Monica, CA: Rand, 2020). The Washington Post, ‘The Soldier with 100 Eyes’, accessed 11 October 2023. https://www.washingtonpost.com/sf/brand-connect/raytheon/soldier-with-100-
eyes. Watling, Jack, The Arms of the Future: Technology and Close Combat in the Twenty-first Century (London: Bloomsbury, 2023). Watling, Jack, and Nick Reynolds, Meatgrinder: Russian Tactics in the Second Year of its Invasion of Ukraine (London: RUSI, 2023). Weber, Max, ‘Politics as a Vocation’, in H. Gerth and C. Wright Mills (eds.), From Max Weber: Essays in Sociology (London: Routledge, 2007). Wilcox, Lauren, ‘Embodying Algorithmic War: Gender, Race, and the Posthuman in Drone Warfare’, Security Dialogue, 48(1) (2017), 11–28. Williams, John, ‘Locating LAWS: Lethal Autonomous Weapons, Epistemic Space and “Meaningful” Human Control’, Journal of Global Security Studies, 6(4) (2021), 1–17. Wilson, Peter, ‘Prussia as a Fiscal-Military State, 1640–1806’, in Christopher Storrs (ed.), The Fiscal-Military State in Eighteenth-Century Europe: Essays in Honour of P.G.M. Dickson (London: Routledge, 2009), 95–124. Wrobel, Sharon, ‘Israel Invests NIS 500m in R&D Infrastructure to Keep Pace in Global AI Race’, The Times of Israel, 18 September 2024. https://www.timesofisrael.com/israel-invests-nis-500m-in-rd-infrastructure-tokeep-pace-in-global-ai-race. Wyatt, Austin, ‘Charting Great Power Progress toward a Lethal Autonomous Weapon System Demonstration Point’, Defence Studies, 20(1) (2020), 1–20. Y.S. [Yossi Sariel], The Human-Machine Team: How to Create Synergy between Human and Artificial Intelligence that Will Revolutionize Our World (independently published, 2021). Yon, Michael, ‘Gates of Fire’, Michael Yon Online, 31 August 2005. ———, ‘Deuce Four: B. General Erik Kurilla Makes an Inspirational Speech Mentioning in Detail Heroes Like Scott Smiley’, Facebook, 1 January 2016. https://www.facebook.com/MichaelYonFanPage/photos/deuce-four-b-generalerik-kurilla-makes-an-inspirational-speech-mentioning-inde/10153342117445665. Zuboff, Shoshana, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (London: Profile Books, 2019). Zysk, Katarzyna, ‘Defence Innovation and the 4th Industrial Revolution in Russia’, Journal of Strategic Studies, 44(4) (2020), 1–29.
INDEX Adarga AI, 79, 83, 94 adversarial generative AI, 140 aerial combat, simulation of, 11, 16–17, 102 aerial systems, uncrewed: Project Maven and, 67, 116–17. See also drones Afghanistan: drone surveillance in, 117; IEDs in, 84; key leader engagement in, 125; Palantir software in, 86–87, 159; Special Operations Forces in, 90, 92; US military operations in, 19, 44, 50 Ahmad, Tariq, 135 AI (artificial intelligence): adversarial, 140; demanding more skilled human operators, 156; exponential progress of, 4–5; faulty data and, 175, 176; in financial markets, 4; first-wave, 26, 27; fragility of, 119, 151; generative, 4, 31– 35; human labour indispensable to, 153–57; investment in R&D, 64–65; making probabilistic inferences, 17; misattribution of agency to, 149–50, 152, 153, 157, 168, 169–70; origins of, 23–26; performative definition of, 23–24; requiring data, computing power, and expertise, 169; revolutionary powers of, 1–5; in scientific and medical research, 2–3; second-generation, 26–31; as a service requiring constant human effort, 165, 169; sub-fields of, 23; as a tool for human teams to use, 157. See also business and commercial applications of AI; military applications of AI Airbnb, 36, 42 airpower, 166–67 AI strategy. See policies for AI strategy; strategy AlexNet, 29, 32 algorithms: automating cyber operations, 146; in information and psychological operations, 147; optimised and updated, 119; predictive, 126–27; in secondgeneration AI, 27–28; social media using, 138–39, 140 Alibaba, 27 AlphaDogfight, 11, 16–17 AlphaFold, 3 AlphaGo, 2, 28, 29, 154–55 al Qaeda, 87, 90, 92, 93 Altmann, Jürgen, 10, 12 al Zarqawi, Abu Musab, 92–93
Amazon, 3, 37–40; based on explosion of data, 27, 38; computing facilities of, 29; establishing office near the Pentagon, 76; failed cloud computing contract and, 76; human workforce of, 38–40, 42; investment in research and development, 64–65; using algorithms to influence consumers, 139 Amazon Web Services, 38; aiding Ukraine, 172, 174; armed forces developing relations with, 81; in military-tech complex, 21; Royal Navy battle-management system and, 109; UK Ministry of Defence contract with, 78 Amicelle, Anthony, 155 Anduril: armed forces developing relations with, 21, 81; employing Special Operations veterans, 94; investment in research, 64; sensor system of, 108–9; Thiel’s investment in, 70; UK Ministry of Defence working with, 79 Antal, John, 11, 179 Anthropic, 32 Anthropocene, 1 antibiotic resistance, 3 Antonov, Anatoly, 173–74 Apple: based on explosion of data, 27; investment in research and development, 64–65; refusing to unlock iPhone, 67; start-up funding for, 63–64 Arkin, Ronald, 12 Armenian diaspora, 144–45, 147 Arquilla, John, 12 Aushev, Yegor, 137 Australian Army, 151–52 automation: in commercial workforce, 42; of data processing, 57; excessive at Tesla, 41; unlikely as fully functional, 155–56 automation of war: AI as we currently know it and, 23; armed forces seeking, 7; commentators believing in imminence of, 12, 41; data processing instead of, 57; fears regarding, 5–12; human-machine team and, 149–50; imagined future of, 166, 167; improbable with AI, 31, 35, 42, 167; with non-human decisionmaking, 7; overstated risks of cyberwar and, 132–33; sceptical views of, 15–17, 18, 55; scholars ignoring limitations of, 14; self-driving cars and, 36. See also military applications of AI Babbage, Charles, 24 back-propagation, 28, 29 Baidu, 27, 36 Baker, James, 11, 149–50, 152 Bakhmut, Battle of, 179–80 Banko, Michele, 26–27 bank software, for monitoring transactions, 155 battle-management systems, 106–9, 168; Anduril’s sensors and, 108–9; of Australian Army, 151; Israel’s Torch, 80–81, 107, 152; of Royal Navy, 109; Ukrainian, 108; US forces’ DCGS, 116; US forces’ JADC2, 101, 103
battle networks, digital, 46–47 Battle of Bakhmut, 179–80 Bawab, Muhammed, 127–28 Ben-Ari, Eyal, 91 Berman, Eli, 115 Bezos, Jeff, 37–39, 68, 76 bias, human, 151 bias or gap in data, 15–16, 30, 34, 175 Biden, Joseph, 174 big data, 36, 47, 58, 115, 150 Bigelow, Julian, 25 Bigley, Kenneth, 93, 112 Bing, 33 bin Laden, Osama, 87 biotechnology, 1, 48, 70 Bitzinger, Richard, 20 Blank, Julius, 62 bots, 139–40, 144, 147 Bowden, Mark, 87 Bowen, Bleddyn, 16 BRAWLER, 102 Brexit, 4, 54 Brill, Eric, 26–27 Brin, Sergey, 68 Brown, Jason, 117–18 Buchan, Iain, 120–21, 130 Buchanan, Ben, 10 Budanov, Kyrylo, 141–42, 147 Burton, Arran, 123 Bush, Vannevar, 62 business and commercial applications of AI, 3–4, 35–37, 42, 58. See also Amazon Cambridge Analytica, 83, 139 Cameron, James, vii Cameron, Lindy, 136 capitalism, Marx on, 153 Carter, Ashton, 73–74 cartography, military, 113 Cattler, David, 137 causation, 18, 30 centaur model, 149–50, 169 Chafkin, Max, 69 Chat-GPT, 4, 31–33, 34
Chernobrov, Daniel, 144–45 chess, 2, 26 China: Australia challenged by, 151; cyberespionage by, 132, 135–36; cyber specialists in, 133; drone swarms of, 13; emerging as power rival, 44–45; human labour force of tech companies in, 41; increasing use of defence software, 79; intending to have premier AI-enabled military, 12–13; military technologies and, 20; planning experiments and, 110–11; possible war between US and, 181–82; UK AI strategy and, 54; US AI strategy and, 47, 49; US tech sector and, 68, 70, 71 civil-military relations, transformation of, 170–71, 175, 183 Clausewitz, Carl von, 53, 132 cloud: Copilot on, 33–34; data servers on, 29 cloud-based training, 78 cloud computing: for the Pentagon, 76; for researchers and students, 71 Cohen, Jared, 70 Cohen, Stephen, 84 Cold War, 44, 53, 90; archaic procurement model of, 73; military-industrial complex in, 21, 81 command and control: battle-management systems and, 107, 108; UK’s Spearhead programme, 103–6 commanders: in civilian-military ensemble, 165; emotional commitments of, 7; functions supported by AI, 100, 114; substituted by AI, 7–8. See also decisionmaking by commanders; headquarters command hierarchies, changing, 20 commodity fetishism, 153 computer vision: drone surveillance and, 117; Project Maven and, 118; satellite images and, 103 computing power: exponential increase in, 29–30; for large language models, 32; required by AI, 168 conspiracy theories, 139 context, not recognized by AI, 30–31, 34, 36, 42 contracting: in business and commerce, 42. See also procurement convolutional neural network, 103 Copilot, 33–34 counterinsurgency operations, 86, 115; Liverpool as analogue for, 120 counterterrorism, and Special Operations Forces, 89–90, 91, 92 Covid testing in Liverpool, 116, 119–25, 130 creativity: in AlphaGo, 2; large language models and, 33 CrowdFlower, 118 Cukor, Drew, 72, 117–18, 160 Cuomo, Scott, 15–16 cybernetics, 24
cyber operations: AI used in, 58, 132–33, 168; autonomous, 146–47; at Battle of Bakhmut, 180; civilians participating in, 143–45; contrast with military actions, 145–46; human actors needed for, 147; severe limitations of, 145–46; three forms of, 132 cyberspace: Israeli intelligence in, 97; in multidomain operations, 51–52; in UK defence, 54; in US National Military Strategy, 48 cyberwar, 6, 12, 131–33 cyborgs, 1, 183 Daniels, Owen, 112 DARPA (Defense Advanced Research Projects Agency), 13, 155 Dartmouth seminar, 25, 62 data: bias or gap in, 15–16, 30, 34, 175; critical for UK’s Spearhead, 104; explosion in, 27; inability of AI to go beyond, 42; incomplete and inaccurate in war, 15; in military intelligence, 48–49; in second-generation AI, 26–27, 30; as series of binaries, 27; in servers of tech primes, 30; sources of, 58, 129; used as singular noun, ix data-wiper malware, 136 decision-making: AI’s lack of judgement for, 15; Amazon strategy and, 38–39 decision-making, military: by AI in warfare, 7; complexity of, 8; by drone swarms, 13; human-machine team and, 151; NATO’s AI capacities and, 57; not reducible to AI calculations, 15, 17; supported by AI, 59 decision-making by commanders: battle networks and, 46; complex process of, 15; generative AI and, 35; helped by UK’s Spearhead, 105, 106; in NATO exercise, 56–57; processes required for, 99–100; psychological inhibitions and, 7; of US Joint Force, 50. See also decision-making, military deepfakes, 140–41, 147 deep learning, 15, 23, 27, 29, 32 DeepMind, 1, 2–3, 29, 154 defence ministries, viii, ix Defense Innovation Unit (DIU), 73–74, 76, 118 Delta, 108, 173 deregulation, Schmidt’s advocacy of, 70 de Silva, Tom, 123 DeVries, Kelly, 61 diasporas, internet activism of, 143–45 DigitalGlobe, 118 Dillman, Linda, 3–4 disaster assessment, 75 disinformation, 139, 140, 141, 142 Distributed Common Ground System (DCGS), 84, 87, 88, 116 distributed denial of service (DDoS) attack, 135, 136 Donahue, Christopher, 117, 158–62, 164–65
Douhet, Giulio, 166–67 Dreyfus, Hubert, 26 drones: AI-enabled, 11; Amazon’s experiments with, 40; at Battle of Bakhmut, 180; big data from, 150; ground forces including, 150–51; of Hamas destroying Israeli surveillance, 176; land warfare not transformed by, 167; Maven processing video from, 74, 117; to replace humans, 12; in Russo-Ukraine War, 19, 142, 167; sensors on, 54; as ubiquitous weapon, 13. See also aerial systems, uncrewed drone swarms, autonomous: Australian Army and, 151; dominating future battlefield, 12; fears regarding, viii, 8, 9, 10; fictional assassination by, 9; potential of, 13, 167; supposedly speeding up conflict, 178 dynamic targeting, 127, 158 eBay, 84 Eisenhower, Dwight D., 44, 81 Elbit Systems, 77–78, 80–81, 107, 152 ENIAC (Electronic Numerical Integrator and Computer), 24 ESET, protecting Ukrainian interests, 172 Esper, Mark, 45 ethics: of automation, 57–58; of data-enabled operations, 178; of IDF’s targeting, 130; of lethal autonomous weapons, 175; outsourced to non-human agents, 8; of Project Maven, 119; supposedly made by AI, 12 Etzioni, Oren, 31 Evron, Yoram, 20 Facebook (Meta): applying AI to its market, 42; based on explosion of data, 27; computing facilities of, 30; election influenced by data from, 139; investment in research and development, 64–65; large language model of, 32 facial recognition programs, 29; bias in, 30; Ukrainian use of, 141 Fairchild, Sherman, 63 Fairchild Semiconductor, 62, 63 Farrell, Theo, 151 Fedorov, Mykhailo, 108, 137, 172–74 Ferguson, Niall, 173 fetishism: human-machine team as form of, 153–54; Marx’s concept of, 153 fire: in military history, 10. See also Prometheus fire perimeter model, 75 First Offset Strategy, 44 first-wave AI, 26, 27 fiscal-military state, 171 Flynn, Michael, 86–87 fog of war, 53 Fossey, Joe, 120–21, 130 Fourth Industrial Revolution, 5
France: behind the US in military application of AI, 76–77; defense budget of, 64; increasing use of defence software, 79; looking to employ AI, 20 Frantzman, Seth, 11 Frey, Carl Benedikt, 35 Galbreath, David, 151 Gallant, Yoav, 177 Garcia, Denise, 7, 12 Gates, Bill, 68 Gaza: current war in, 19, 97, 116, 128–29, 177, 181; security threat in, 80 generative AI, 4, 31–35; adversarial, 140; as lacking understanding, 112; likely job losses from, 35. See also large language models Gerasimov, Valery, 157–64, 177, 178 Germany: behind the US in military application of AI, 76–77; defense budget of, 64; increasing use of defence software, 79; looking to employ AI, 20 globalists, in tech sector, 67, 68, 69, 71 Go. See AlphaGo Gödel, Kurt, 25 GOFAI (good old-fashioned AI), 26 Goldfarb, Avi, 14–15 Gonzalez, Roberto, 7 Google (Alphabet): applying AI to its market, 36, 42; armed forces developing relations with, 81; based on explosion of data, 27; buying DeepMind in 2014, 2; computing facilities of, 29–30; contract for Maven and, 67–68, 116, 118, 119; investment in research and development, 64–65; large language model of, 32; in military-tech complex, 21; Schmidt’s executive roles in, 6, 70; self-driving car of, 36; supporting Ukraine, 21, 172, 174; Winograd schemas and, 31 Gorgon Stare, 117 Gospel (Habsora), 127, 128–29, 130 Gotham, 87–88 GPUs (graphical processing units), 29 Gray, Bill, 17 Green, Mark, 130 Grinich, Victor, 62 Guardian AI platform, 93 gunpowder revolution, 61 Hagel, Chuck, 44–45 Hall, Bert, 61 Hamas: attacking Israel on 7 October 2023, 175–76; IDF targeting of, 19, 97, 116, 126, 127–29, 130 Hambling, David, 10, 12 Hamilton, Tucker, 17 Hassabis, Demis, 2, 8, 154
Hawking, Stephen, 8 headquarters: civilian technicians integrated into, 160–61, 164–65, 169; human collaboration for targeting and, 130, 158, 164; human-machine team in, 150, 152; legal status of technicians in, 177; organisational changes in, 20, 160, 164; Spearhead in, 104; tech company employees in, 21, 171. See also commanders; staff officers Helsing, 79, 83 Hermes, 110–11 Hezbollah, 97 Hinton, Geoffrey, viii, 5, 6, 9, 12, 14, 22, 28, 52 Hockenhull, Jim, 19, 21–22 Hoerni, Jean, 62 Hsu, Jeremy, 3 Hui, Fan, 154 human-machine team, 149–53; agency and, 149–50, 152, 157, 169–70; alternative to concept of, 165, 169; Australian Army and, 151–52; brittleness of AI and, 155; excluding human labour from analysis, 153–57; as form of fetishism, 153–54 human-robot teams, 150–51 Hume, David, 18 Hunter, Cameron, 16 Hussain, Junaid, 142, 147 Huttenlocher, David, 5–7, 8, 17, 71, 131 Ignatius, David, 162–63 Igor software, 84, 89 Imbrie, Andrew, 10 improvised explosive devices (IEDs), 84, 102–3 India, and military technologies, 20 industrial revolutions, 5 information and psychological operations, 132, 138–43; in Russo-Ukraine War, 141–42; three elements of, 147 intelligence, military: activities included in, 57; for counterterrorism in Iraq, 92–94; Israel’s Unit 8200 and, 97; role of AI for, 15, 19, 48–49, 52–54, 58; sensors for, 46–47. See also targeting intelligence services, commercial, 105 intelligence tests, biased, 30–31 intelligent machines, 24–26 Iran: AI-enabled targeting in, 115; cyber specialists in, 131, 133 Iranian nuclear facilities, 97. See also Stuxnet Iraq, 44, 85–86, 90, 92–94, 118, 160 Isaacson, Walter, 173 ISIS, campaign against, 87–88, 90, 92, 118–19, 142–43, 147, 160, 162
Israel: accused of genocide in Gaza, 177; AI capabilities and, 13; attacked by Hamas on 7 October 2023, 175–76; cyber attacks by, 134; cyber capabilities of, 133; defense budget of, 64, 65; four pillars of security in, 57; military application of AI in, 20; military-tech integration in, 79–81, 96–97; procurement in, 77–78. See also Gaza Israel Aerospace Industries, 80 Israel Defense Forces (IDF): collateral damage caused by, 128; defence industries as partners to, 80; main mission of, 125; Special Operations Forces of, 96–97; targeting by, 116, 125–29, 130, 150; tech company integrated with, 78; Torch and, 80–81, 107–8, 152; in West Bank and Gaza, 80, 125–26. See also Gaza Jensen, Benjamin, 15–16 Jobs, Steve, 68 Johansson, Scarlett, 140 Johnson, James, 7–8 Johnson, Matthew, 155–57 Joint All-Domain Command and Control system (JADC2), 101, 103 Joint Artificial Intelligence Center (JAIC), 74–76 Joint Enterprise Defense Infrastructure (JEDI), 76 joint force commander, 51–52 Joint Special Operations Command (JSOC), 92–94, 159–61, 173 Joint Task Force Ares, 142–43 Karp, Alex, 13, 68, 84, 85, 163 Kasparov, Garry, 2 Kendall, Frank, 50–51, 174 key leader engagement, 125 Kissinger, Henry, 5–7, 8, 12, 14, 17, 71, 131 Kleiner, Eugene, 62, 63 Kochavi, Aviv, 127, 130 Krizhevsky, Alex, 29 Kropyva, 108 Kurilla, Erik, 159–60, 162 Kurzweil, Ray, 1 large language models, 31–35; GPUs used in, 29; hallucinations in, 111; limitations of, 34–35; limited intelligence of, 34; military planning and, 110–12. See also generative AI Last, Jay, 62 Lattice system, 108–9 Lavender, 128–29, 130, 150, 152 laws of armed conflict, 177–78 Legg, Shane, 2 Levchin, Max, 69, 83–84, 89
Levesque, Hector, 31 Lindsay, Jon, 14–15, 133 littoral combat ship, 155–56 Liu Guozhi, 44 Loebner, Hugh, 33 logic, in first-wave AI, 25–26 logical positivists, 25 Lovelace, Ada, 24 Lovelock, James, 1 Luckey, Palmer, 68, 108 Machiavelli, Niccolò, 171–72 machine intelligence, 24–26 machine learning: counterterrorism in Iraq and, 93; decision-making and, 15; fusing information with, 52, 54, 127; Hinton’s role in, 5; neural networks of, 30; Project Maven and, 117; self-driving car and, 36; of structural destruction in satellite images, 103; targeting by IDF and, 127; three methods of, 23, 28 MAD (mutually assured destruction), 44 Major Combat Operations Statistical Model (MCOSM), 101–2 malware, 131, 132, 133, 135; aimed at Ukraine, 172; autonomous defence against, 146; limitations of, 146; in Stuxnet, 134 ManTech, 102 mapping: Covid testing and, 122–23; in military history, 113 maritime warfare, 181–82 Martin, Ciaran, 146–47 Marx, Karl, 153–54 Mattis, James, 47–48, 74–75, 76 Maven, 52, 67, 72–76, 116–19, 152, 159, 160 McCarthy, John, 25 McChrystal, Stanley, 86, 92, 93–94, 160–61 McCord, Brendan, 118 McCulloch, Walter, 28 mercenaries, 21, 170, 171–72, 177, 180 Messenger, Gordon, 103 Metaconstellation, 87–88, 162 Microsoft: based on explosion of data, 27; cancelling biased program, 30; cloud of, 33–34; computing facilities of, 29–30; failed cloud computing contract and, 76; helping Ukraine, 21, 136, 137, 172, 174; investment in research and development, 64–65; large language models of, 32, 33–34; in military-tech complex, 21; North Korea attack on software of, 134; Royal Navy battlemanagement system and, 109; supplying US despite employee complaints, 68; Windows used in Stuxnet, 134 Microworld of UK’s Spearhead, 104–6, 152
military applications of AI: central to defence policy, 13; changing the nature of warfare, 7; data processing as prime function of, 55; as existential threat, 7, 13, 22; experts’ fears regarding, 5–12; historical evidence about, 18–22; human control of, 148, 151; less effective than military leaders believe, 7; major powers using, 20; as major technological transformation, 182–83; mostly in last five years, 22; as organisational story, 62; in recent wars, 19–20; scepticism about, 14–17, 18; slowing military operations, 178–80; specific functions helped with, 60; tech collaboration in, 148; today’s uses of, 12–14, 168; useful examples of, 15. See also automation of war; cyber operations; decision-making, military; military-tech complex; planning; targeting military-industrial complex: altered or displaced, 169; in Cold War, 21, 81; in Israel, 80; retired generals and admirals in, 85 military-tech complex, 183; emerging, 21–22, 76, 79, 81, 169, 170–71; introducing private political interests, 171–72, 174; lifeworld of cooperation in, 82–83; politicising national defense strategy, 174–75; as profound historic development, 170, 183; as revision of Weberian settlement, 171; Special Operations Forces and, 98. See also tech companies Milley, Mark, 19, 174 Minsky, Marvin, 25 misinformation, 140 missile defense, 46 missiles, in maritime warfare, 181–82 mission definition, 100 Mitchell, Billy, 166 models, in second-generation AI, 26–27, 29, 30 Moore, Gordon, 62 Mullen, Mike, 131 multidomain operations, 48, 51–52; in urban warfare, 56–57 Musk, Elon, 8, 37, 40–41, 68, 69, 162, 172–75 Nadella, Satya, 68 Nagorno-Karabakh wars, 11, 19, 143–45, 147, 179 Napoleon, 113 Nash, John, 25 National Security Commission on Artificial Intelligence (NSCAI), 6, 48–49, 52, 70, 75, 161 NATO: policies for AI strategy of, 56–57; Ukraine defence and, 163; US defence policies and, 44 naval warfare, 181–82 Ndungu, Tonee, 4 Neads, Alex, 151, 152 Netanyahu, Benjamin, 177 Netflix, 28
neural networks, 5, 28–29, 32 Newell, Allen, 25, 26 North Korea, cyber capabilities of, 133 NotPetya, 135, 136, 145 Novacene, 1 Noyce, Robert, 62, 63 nuclear facilities, cyberattacks against. See Stuxnet nuclear weapons, 6, 44, 134, 173 Obama, Barack, 45, 70 OpenAI, 4, 29, 31 Operation Glowing Symphony, 142–43, 147 Operation Guardian of the Walls, 116, 127–28, 176, 179 Operation Orchard, 134 Operation Swords of Iron, 116, 128 Oracle, 76 organisational transformation, 20–21, 183. See also military-tech complex Osborne, Michael, 35 Page, Larry, 68 Palantir Technologies, 83–89; armed forces developing relations with, 81; dataprocessing goals of, 83; founded by Thiel, 70, 83; human expertise in, 88–89; interpersonal networking in, 85, 87; investment in research, 64; security applications of software in, 84; selling software directly to military units, 85–87, 89; Special Operations Forces and, 94; suing the US Army, 88; supporting Ukraine, 21, 162–64, 174; targeting software of, 162–65; UK Ministry of Defence contract with, 78–79. See also Thiel, Peter Panetta, Leon, 131, 133, 145 Payne, Kenneth, 7, 11, 12, 14, 16–17 PayPal, 69, 83–85, 89 Pentland, Alex, 27 Petraeus, David H., 87 Philip II, 171 Piacentini, Diego, 39 Pitts, Warren, 28 planning: facilitated by AI, 58, 100–101, 168; large language models and, 110–12; predictive computer programs for, 102; process of, 100–101; UK Spearhead programme for, 103–6; and the use of AI in more complex operations, 112–13; US tools used for, 101–3. See also battle-management systems; decision-making by commanders policies for AI strategy, 43–44; of China, 44–45; data as support for, 57–59; ethics and, 57–58; of every major military power, 57–58; of NATO, 56–57; of UK, 53– 55; of US, 13, 44–53 Price, Roy, 38
Prigozhin, Yevgeny, 170, 180 private military and security companies, 170. See also mercenaries procurement: in British system, 77–79; in Israel, 77–78; by Pentagon, 72–76, 77, 79; system of Special Forces, 90–91 productivity: of Amazon’s pickers, 40; generative AI and, 4, 35; large language models and, 34 Project Maven. See Maven Prometheus, 1, 5 protein structure, 3 prototyping method of procurement, 74, 78 Putin, Vladimir, 12, 158, 179 quantum computing, 5, 22, 23, 70–71 Rafael Advanced Defense Systems, 80 Raman, Kal, 39 Reagan, Ronald, 69 Rebellion Defence, 77, 79 regulatory reform, 71–76, 78, 169 reinforcement learning, 23, 28, 29, 32; to identify IEDs, 103 Revolution in Military Affairs, 12, 107, 180 Rhombus Power, 83, 93 Rid, Thomas, 132, 139 Roberts, Sheldon, 62 robotics: assisting humans in specific functions, 155; military application of AI to, 57; US investment in, 13; in US National Defense Strategy, 48 robots: in Amazon fulfilment centres, 39–40; armed in future combat, 167; Australian Army using, 151; in automated warfare, 9, 11; in ground forces, 150– 51; in human-robot teams, 150–51; to replace humans in war, 12; Third Offset Strategy and, 45–46 Rock, Arthur, 63 rockets, in maritime warfare, 181–82 Ronfeldt, David, 12 Roosevelt, Franklin D., 62 Roper, Will, 117 Rosenberg, Jonathan, 70 Rosenblatt, Frank, 28 route planning, 105–6 Roy, Anshu, 93–94, 160, 161 Rumelhart, David, 28 Russell, Bertrand, 25 Russell, Stuart, viii, 9–10, 12, 14, 22, 52 Russia: aggressions of, 44, 47; cyberattacks by, 135; cyber specialists in, 133; information operations of, 139; US AI strategy and, 47
Russo-Ukraine War: Battle of Bakhmut in, 179–80; drones in, 19, 142, 167; fullscale invasion in, 136, 158; importance of AI in, 12–13, 19; information and psychological operations in, 141–42; military-tech complex in, 21–22; opensource data in, 162; and predictions by US’s MCOSM program, 102; as presage of the future, 180–81; Russian cyber operations in, 136–37, 140–41; Russian malware in, 136–37; and sinking of Russian ship Moskva, 181; Special Operations Forces in, 90; targeting in, 168; tech companies influencing US policy on, 174; tech primes’ support for Ukraine in, 172; Ukrainian cyber operations in, 137–38, 141–42; Ukrainian intelligence in, 54; US support for Ukraine in, 158–64; wounding of General Gerasimov in, 157–64, 177, 178 Ryan, Mick, 150–51, 152 sabotage, using cyberspace, 132, 133–35, 136 Sanders, Patrick, 132 Sandworm, 137 Sariel, Yossi, 150, 152 satellites: Battle of Bakhmut and, 180; Metaconstellation program and, 87; naval warfare and, 181; sensors on, 54, 55; surveillance data from, 118; Ukrainian military and, 22. See also Starlink Sauer, Frank, 10, 12 Scale AI, 110–11 Scharre, Paul, 11, 36 Schmidt, Eric, 5–7, 8, 9–10, 14, 17, 48, 68, 70–72, 110, 131 Schwarzenegger, Arnold, vii second-generation AI, 26–31; fallibility of, 30–31; improbability of full automation under, 40; as probabilistic and inductive, 26–27, 30, 31, 32, 42; three critical enablers of, 27. See also large language models Second Offset Strategy, 44 security studies: automation of war and, 10, 12, 41; cyber attacks and, 133; human-machine teams and, 151; ignoring organisational transformations, 97– 98; limitations of military AI and, 10; powers of AI and, viii; Project Maven and, 116; technological determinism in, 61 Sedol, Lee, 2, 154 self-driving cars, 36–37, 46 Sensity, 141 sensors: Anduril’s automated system of, 108–9; of battle networks, 46–47; of Israel in the Occupied Territories, 57, 126, 127; necessity of degrading enemy’s sensors, 179; overload of data from, 53; urban, 56; of US Joint Force, 50 Shamir, Eitan, 91 Shanahan, Jack, 52–53, 56, 72, 74–75, 117–19 Shockley, William, 62 Shotwell, Gwynne, 174
Silicon Valley, 61–66; DIU headquarters in, 73; Pentagon support for, 63; reoriented from libertarian to nationalist, 66–72, 174; venture capital and, 63– 64, 67 Simon, Herbert, 25, 26 Simonov, Andrei, 158 simulations, 16; of aerial combat, 11, 16–17, 102 situational awareness: across battlespace, 47; data processing for, 57, 58; in multidomain operations, 52; Palantir software and, 88; self-driving car and, 36; in UK intelligence policy, 55; of Ukrainian military, 22; in urban warfare, 103; in US intelligence policy, 49–50 Slaughterbots (film), 9 Smith, Brad, 136 Smith, Brian Cantwell, 25–26 Snowden revelations, 132 social media: algorithms used by, 138–39, 140; Armenian diaspora and, 144–45; bots and, 139–40, 144, 147; Budanov’s use of, 142; in information and psychological operations, 147; as open-source data, 162; Russian subversion and, 139–40 sociology, vii–ix; technological determinism and, 60–61 software: as main product of tech companies, 72–73; in second-generation AI, 27–28 Solomonoff, Ray, 25 Soviet Union, 44 space: in multidomain operations, 51; in UK defence, 54; in US National Military Strategy, 48 SpaceX: and travel to Mars, 40; in military-tech complex, 21; providing services in Ukraine, 173–74 Spearhead, 103–6, 152 Special Competitive Studies Project (SCSP), 110, 111–12 Special Operations Forces: counterterrorist role of, 89–90, 91; drowning in data, 95–96; of Israel Defense Forces, 96–97; large budgets of, 90; Palantir software used by, 86–87; procurement system of, 90–91; specific missions of, 92, 93; tech companies partnered with, 89–96, 98; transnational relations of, 91–92; unique in politico-military hierarchies, 91; unique status in US armed forces, 86, 90; in wounding of General Gerasimov, 157 spying, as cyber operation, 132, 135–36, 137 staff officers: becoming more essential, 113; data and, 101; large language models and, 110, 111; predictive calculations by, 102; relieved some of the burden of planning, 103, 105–6; supporting commanders’ decisions, 164–65. See also headquarters Starlink: armed forces developing relations with, 81; Ukraine and, 21, 162, 172–75 Stephens, Trae, 108 stock market, 4
Stone, Brad, 39 Stop Killer Robots campaign, viii, 8–9, 10 StormCloud, 109 strategic bombing, 166–67 strategic studies, vii strategy: experts’ fears about AI and, 6–7; made by AI, viii, 8; made by company executives, 39, 42; tech companies’ involvement in, 174–75. See also policies for AI strategy Stuxnet, 97, 134, 146 subversion: as cyber operation, 132; Russian active measures and, 139. See also information and psychological operations Suchman, Lucy, 10 Suleyman, Mustafa, 1–2, 33 Sullivan, Jake, 174 Summers, Jared, 160 Sunak, Rishi, 4–5 supervised learning, 23, 28, 29; NATO policy on targeting and, 56 surveillance systems, 12; drones in, 13; failure of Israeli system, 175–76; of Ukrainian military, 22. See also sensors Sutskever, Ilya, 29 Swift, Taylor, 140 Syria, 90, 103, 118, 134 Taiwan, imagined Chinese assault on, 182 Taliban, 90 targeting: agency of AI in, 149–50; AI requiring more people for, 130; for Covid testing, 116, 119–25, 130; of customers by companies, 115; dynamic, 127, 158; of General Gerasimov, 157–64, 177, 178; human-machine team and, 149–50; human teamwork in, 129–30; important role of AI in, 46–47, 58, 168; by Israel Defense Forces, 116, 125–29, 130; linking disparate data for, 129; in multidomain operations, 52; NATO policy and, 56; speed and precision of, 178; susceptibility of AI to errors in, 15–16; by UK forces, 55; by US Air Force, 51. See also intelligence, military; Maven Task Force Dragon, 159, 160, 161, 162–63, 164–65, 177 Tay (chatbot), 30 teamwork: of humans for targeting, 129–30; of military personnel and technicians, 169 tech companies: armed forces needing to collaborate with, 66; based on explosion of data, 27; competing for best talent, 65–66; competing with one another, 76; defence ministries and, viii, ix; dynamic market for software of, 72–73; helping Ukraine, 136, 137; integrated into military operations, 21; with offices near the Pentagon, 76; with offices near UK Ministry of Defence, 79; operating as part of state forces, 171; selling software directly to military units, 89; Special
Operations Forces partnering with, 89–96, 98; teams of experts in, 156–57; using algorithms to influence consumers, 139. See also military-tech complex; tech primes technological determinism, 60–62, 153 technology: of China in new century, 44; as a social product, 60–61; of US in Cold War, 44 tech primes: in disputes with US government, 67–68; influencing state interests, 172; investing in large language models, 33–34; and investments in research and development, 64–65; needing access to operational data, 169; RussoUkraine War and, 172; vast computing facilities of, 29–30; venture capital and, 63–64. See also tech companies Telegram, 141, 144 The Terminator (film), vii, 53 terrorist attacks on 11 September 2001, 69, 83, 90 terrorist cells in Gaza and West Bank, 125–26 terrorists, targeting of, 150 Tesla, 36, 37, 41 Thiel, Peter, 58, 68–70, 71, 72, 83–85, 87, 88–89. See also Palantir Technologies Third Offset Strategy, 13, 44–46, 57; autonomous systems in, 45–46, 47–48; procurement reform and, 73; targeting and, 116 Toolan, John, 87 Torch, 80–81, 107, 152 Tossell, Ben, 33 trace italienne, 62 Traitorous Eight, 62, 63 transistors, 24, 28 Trenchard, Hugh, 166 Trump, Donald, Thiel’s support for, 68 Trump administration, 45 Tunnell, Harry, 85–86 Turing, Alan, 24–25 Turing test, 25, 33 Twitter, bots on, 140 Uber, 36 UK: AI as opportunity for, 4–5; AI capabilities and, 13; behind the US in military application of AI, 76–77; cyber capabilities of, 133; defense budget of, 64, 65, 77; military application of AI in, 20; policies for AI strategy of, 53–55; procurement systems in, 77–79; Spearhead planning programme for, 103–6, 152; Special Operations Forces in, 96 Ukraine: AI-enabled intelligence in 2014, 47; battle-management systems of, 108; IT Army of, 137–38, 141, 147; Russian cyberattacks of 2014–2017 on, 135; US support for, 158. See also Russo-Ukraine War
understanding: lacked by generative AI, 112; not recognized by AI, 30–31, 36, 42; in US intelligence policy, 49 Unit 8200, 97, 116, 126, 127, 133, 134, 177 unsupervised learning, 23, 28, 29, 32 urban operations: planning of, 102–3; targeting in, 126, 128 urban warfare: AI-enabled, 182; building destruction in, 103; multidomain approach to, 56–57 US: cyber capabilities of, 133; defense budget of, 64, 65; drone swarms and, 13; as a pioneer in military application of AI, 20; possible war between China and, 181–82 US Air Force, 17, 50–51, 84, 117 US Army, 51–53 US Marine Corps, 48, 87, 110, 182 US Navy littoral combat ship, 155–56 US policies for AI strategy, 13, 44–53; joint forces in, 49–50, 51; massive datasets and, 48–49, 52; of separate services, 50–53 US presidential election of 2016, 139, 140 Varian, Hal, 27 venture capital, and Silicon Valley, 63–64, 67, 169 Vera, Alonso, 155–56 Vietnam, 44, 89 Virilio, Paul, 178 viruses, biological, 116, 119–25, 130 viruses, in computers, 131, 132, 133, 134, 137, 146 Wagner Group, 170 Walmart, 3–4, 36, 39 war: becoming slower, 178–80, 182; being reconfigured by AI, 20; between China and US, 181–82; as complex environment, 14–15, 16, 31; definition of, 132; incomplete and inaccurate data in, 15; influenced by private-sector tech companies, 21, 174; recent evidence from, 19–20, 22; trending towards attritional and positional war, 183. See also automation of war warbots, 11 war-gaming, 102 War on Terror, 70, 87, 90 wastewater, viral load in, 123–24 waterfall system, 72, 73, 77, 79 Waymo, 36, 37, 46 weapons, autonomous and lethal, 8–12; Australian Army and, 151; ethical concerns about, 175, 176; as eventual possibility, 46, 57–58, 167–68; inaccurate obsession with, 52–53; of Israel, 57; scholars concerned about, 59. See also drone swarms, autonomous weapons, nuclear, 6, 44, 134, 173
Weber, Max, 171 Weinberger, Sharon, 87 West Bank, 80, 125–26 Whyte, Christopher, 15–16 Wiener, Norbert, 24 Williams, John, 12 Winograd schemas, 30–31, 33 Wittgenstein, Ludwig, 25 Wong, Felix, 3 Work, Robert, 45–47, 48, 57, 72, 74, 83, 133 XVIII Airborne Corps, 158–64, 177 Yon, Michael, 159 Zambellas, George, 54 Zelensky, Volodymyr, 19, 140–41, 173, 174, 179