Security Metrics: Replacing Fear, Uncertainty, and Doubt by Andrew Jaquith Book Review

Security Metrics: Replacing Fear, Uncertainty, and Doubt by Andrew Jaquith

5 Stars

Every Security Professional (or wannabe) should read this book

I'm not sure what I can write to sway you to buy or read the book if 5 star reviews from Ben Rothke and Richard Bejtlich don't sway you but I'll throw my likes and dislikes in here anyway. I'm not a "metrics guy" in fact, I'm still not , but I do think the book puts the concept of using them into perspective for the person that may not use any metrics in their security work.

I've been summing up the book to people at work by using the example (and I'll badly paraphrase) from the book of "if your spam gateway blocks 100,000 spam messages a day is that a good metric?" Initially you may say yes, that is a good metric. In fact most people at work said the same thing. But, as the author explains it is a poor metric. Better metrics are useful percentages like the percentage of missed spam or the percentage of false positives. Saying that 100,000 spam message are being stopped only tells us that you have a ton of spam on your network.

Some of the things I liked about the book were the author's discussions on how to make charts more readable and efficient at portraying information. I had to read the Tufte books in college and have to admit that I got more out of chapter 6 (visualization) than I feel I learned that whole semester of class. Chapter 2 discussing what makes good metrics was extremely useful, as well were chapters 3 & 4 because they gave good examples of metrics you can use to measure an organizations various defenses like perimeter security or application security. The discussion of using COBIT, ITIL and Security Frameworks in Chapter 4 was also good.

I only had two minor gripes. First was that toward the end of the book the author talks about colors of slides and charts which obviously doesn't do us any good since the book is in black and white and second, that he does use some big words throughout the book and I did find myself having to go back and reread things. Could he have put it into simpler terms, probably, but that doesn't make the book bad, just means I need to work on my vocab :-)

Overall it was a good entrance to the world of security metrics for me and took and away some of the perceived boredom of them. It definitely gave me some tools to look more critically at the numbers and stats that some of the vendors throw our way as well as how to deliver data and information in a more useful matter.