Title: Weapons of Math Destruction by Cathy O’Neil

Subtitle: How Big Data Increases Inequality and Threatens Democracy

Why: A co-worker recommended it.

Review: Five stars for the content. Maybe deduct a half star for the organization. The first few chapters were powerful, laying out the case for how these data algorithms have gotten out of control. Some of the later chapters got repetitive, but I think the repetition could have been really useful, if it drilled into the readers heads exactly how to distinguish a bad use of math from a good one. Readers should be encouraged to analyze things in the wild on their own. I was also hoping for stronger calls to action, but was left just feeling kinda hopeless at the end of the book.

All that said, everyone should read this book. I love math and big data and its potential for good, but this book clearly lays out how those tools can lead to misery for all of us, but especially for the least fortunate and most oppressed among us. At scale, they turn into positive feedback loops which never correct. Just as a simplistic example, if a college is rated by how selective it is, and a high rating leads more people to apply, then it will, by definition get more selective over time, which will lead to an even higher rating, leading to a vicious cycle. Even more perniciously, if jobs use low credit scores to weed out applicants, then those most in need of a good job will be least likely to be able to secure one, which will lead to more financial difficulty, leading to a worse credit score and an even more vicious cycle. The worst part of these algorithms, is that they are opaque to everyone involved so it is impossible to fight back against their decisions. Just as another ridiculous example, drivers with clean accident records, but a poor credit score were quoted higher rates than convicted drunk drivers with good credit scores. The users of “big data” get so enamored with their algorithms that a poor proxy for risk (credit score) is given more weight than an actual risk (drunk driving).

Hopefully, society will begin to regulate this area and guide our use of these algorithms in a way that is beneficial to all members of society, especially the most disenfranchised portions.