With the excess free time that is quickly becoming our norm given the quarantine measures, I’ve dedicated much of that time to whittling down my reading list, including many titles related to the vast field of data science and technological innovation. Near the top of that list, I found Weapons of Mass Destruction by Cathy O’Neil, a journey through the various aspects of our society - from insurance policies to incarceration sentences, college admissions to online advertising - now dictated by machine learning algorithms in a way that exposes the inequities and injustices that these weapons of math destruction (or WMDs, as O’Neil refers to them throughout the book) tend to reproduce.
With regards to the content of the book, there remains little to say that O’Neil does not say herself. She is frank, direct, and unreserved in her scathing analysis of the ways that we as a society utilize mathematics, statistics, and progamming in order to expedite injustice and effectively extract money from the lower classes for the benefit of corporations and those at the pinnacle of our social hierarchy. Not to mention that these consequences occur in the name of science and objectivity, by-products of a “mathematically-sound” system that asserts a separation from human bias. However, when biased humans are the ones programming these WMDs, the product is a machine that operates with the same imbued bias and discrimantory tendencies.
Whether we are considering futuristic TV shows such as Westworld and Black Mirror, despairing movies like Blade Runner and Alita, or the era of dystopian novels stemming from George Orwell’s 1984, the possibilities for our future seem unlimited. But they are all rooted in common themes: surveillance, technology, and the reduction of human free will. Across all forms of media we gain a glimpse into a future where machinery rules the world as we know it today, a result of the unyielding technological progress that will ultimately become our downfall. But in her book, O’Neil dares to ask the question: what if that day is already here?
By revealing the oftentimes subtle ways (and, other times, very explicit ways) that various scores and rankings (such as your income level and credit score) allow algorithms to opaquely dictate what opportunities and scams we are exposed to or shielded from, O’Neil demonstrates that we no longer have control over or the ability to change the ways in which we navigate the game of life. Computers increasingly have the ability to predict our decisions and preferences, correlating these with our chances of success in a way that make these “chances” self-fulfilling prophecies. Sound familiar? Maybe you’ve seen this on Westworld, where each person’s “loop” or life trajectory is predetermined by a machine named Rehoboam. Or maybe it’s Black Mirror, on an episode where a score dictating each person’s likability score hovers above their head. Even further, it could by 1984, in a society where we no longer know who to trust, as any wayward action results in almost certain imprisonment and death. Are these futures a certainty, or do we have the power to prevent these “utopias” from becoming our reality?
In her final chapter, and the afterword published in the most recent edition, O’Neil offers some advice with regards to how to rework our preexisting WMBs and prevent the negative consequences they currently prepetuate. After 200 pages of dismal revelations about our society, I have to admit that I was hoping for more. That’s not at all to say that O’Neil fails to deliver existing solutions. But instead, that solutions that we must adhere to are not simple. And they’re certainly not easy. It means critically examining each and every algorithm we put into practice. It means building this deeper analysis into our pre-existing processes. It means sacrificing at times the glorious optimatization and efficiency that we so crave in machine learning. It means knowing when to prioritize our goals as a company or organization, and when to prioritize instead the lives and livelihoods of those our systems consequently affect. Sound easy to you? Yeah, me neither.
The frustration I felt reading those final two chapters reminded me of the feelings I have when I encounter injustice in my daily life or when I hear of others’ stories in person or through social media platforms. When our entire system is built on oppression and discrimination based on race, class, and other aspects of our identities, how can we truly dismantle these structures to replace them with something more just? Like O’Neil emphasizes, it starts will small acts in each direction we are able to look, no mater how small and seemingly insignificant. Whether is our current project at work, the ways we interact with strangers at the grocery store, or how we share the stories of others, these small acts work together to transform the system one piece at a time. In an era characterized by increased transparency and the rapid distribution of information across the internet, it is all too easy to become overwhelmed by the status of our society and impending doom that seems to permeate every aspect of our newsfeeds. But it’s essential to remember that we have impact and the ability to enact change. O’Neil certainly doesn’t forget that, and neither should the rest of us.