Maybe Quantoids are good for something after all

Alan Krueger and David Laitin have an article in the latest Foreign Affairs discussing the State Department’s infamous terrorism report for 2003. As you may recall, the report was unveiled with great fanfare, and purported to show that terrorist incidents were at a thirty-five year low, despite the fact that 2003 saw the most terrorist attacks of any year on record. Regrettably, the article is available by subscription only, but you can read the preview here.

Laitin and Krueger point out a number of problems with the terrorism survey. There are no clear criteria for determining whether an event is a terrorist attack or not. There are no clear criteria for determining whether a terrorist incident is international, in which case it counts, or domestic, in which case it doesn’t. Think for a moment about Chechen terrorism; domestic or international? The report itself is very hazy on the details of how it reached its conclusions. All of this would be bad enough to ruin a study if terrorism wasn’t politically relevant. Because it does matter politically, all sorts of folks have an interest in how it turns out, which means that they try to meddle with the results.

All of this is pretty bad, but genuinely unsurprising. This administration has a knack for combining mendacity and incompetence. However, there are larger issues that make messing with the terrorism survey particularly problematic. A few years back, a guy named Stephen Rosen demolished the myth that military organizations learn better during war time. He showed, using the examples of the U.S. Army in Vietnam and the RAF/USAAF in World War II, that military organizations only learn when they have accurate measures of effectiveness. Back in World War II, a bunch of generals in the RAF and the USAAF (United States Army Air Force) were convinced that bombing German cities out of existence would force the Germans to surrender. They also tried to destroy particular German industries, such as ball-bearing production. Problem was, they didn’t have any way to measure whether their bombing was effective; they knew they were blowing things up, but hadn’t a clue what this was doing to the German, and eventually the Japanese, war effort. Turns out that it wasn’t doing anything, and German arms production peaked at the very time the bombings were heaviest. The organizations were ineffective because they had no reliable means to determine whether what they were doing was working.

In the Pacific, political considerations intervened, making the situation a lot worse. The USAAF didn’t even try to bomb Japanese industry, and instead concentrated on firestorming Japanese cities. A firestorm is a real bad thing; it’s a fire that feeds upon itself. The USAAF wanted to be independent of the Army, and decided that if it won the war it could win its independence. So, the generals cooked up a bunch of measures of effectiveness that had no connection with reality, but that justified the continued incineration of Japan’s urban population. A lot of wasted effort, and a lot of dead people. Read Robert Pape, Bombing to Win, for more details.

Long story short, learning to fight terrorism, or even the Iraqi insurgency, is going to be really hard because good measures of effectiveness are extremely hard to come by. Spinning, washing, drying, and mangling the numbers, which is what this administration is doing, makes it impossible for organizations to learn. We continue bad strategies that we think are working, and eschew good strategies that might have an actual effect. But then, the administration isn’t really interested in winning anything that’s not an election, so no real problem.