Review: Army of None

Army of None is a book about Autonomous Weapons and how the AI revolution could potentially play out for war fighting. The scope of the book is huge, ranging from the history of automatic weapons to the ethics behind the rules of warfare and an exploration of where automation is used in the wild to varying results.

The book is written by a former Army ranger, and so it goes into tedious details about stuff you may or may not be interested in, such as the intricacies behind the Pentagon policies about autonomous weapons and the various treaties involved in landmines.However, he points out several issues that you may or may not have thought of:

• AI spoofing is real, and in a situation where human life is on the line, there's no guarantee that an adversary can't make your image recognition/targeting intelligent system misidentify something.
• The place where fully automated agents are active in the wild is on Wall Street. On Wall Street, automation-driven flash crashes are so frequent that the trading systems now have back-stops and back-offs, and they get triggered on a regular basis.
• Highly complex systems such as nuclear reactors or highly automated weapon systems such as the Aegis Combat System are so complex that failures are "normal." To counteract such failures, the navy has developed an operation protocol around the Aegis such that the automation is always kept on a tight leash. At all times, navy personnel have their fingers on the button to turn off the system as soon as it has done its job. To a casual observer, the operators of an Aegis system exhibit no trust whatsoever as to the reliability of the system and the automation. But this protocol took more than 2 decades to develop, and before it was developed the system targeted a civilian aircraft and killed everyone on board, so this protocol was warranted.
• By contrast, the Patriot Surface-to-Air System introduced during the Persion Gulf War demonstrated 2 cases of fraticide. Both incidents were traced to the operators having too high a trust in the automation of the system. 
• Banning autonomous weapons won't work: history suggests that an effective weapon will be deployed as soon as it provides an advantage, and then a race to use that weapon most effectively will occur. Keeping humans in the loop when everything is moving at machine speed will cost you a war.

All in all, the book was long winded and took its time getting around to many points, but it was worth the read, pointing out interactions between systems that I didn't think of before. Recommended.