Weapons of Math Destruction – An Ethical Critique on “Big Data” in Networks
The rise of the big data revolution has left many statistical scientists the task of grappling with vast amounts of data of which they must develop algorithms to analyze and make predictions. It’s no surprise then that many companies have started to hire data and computer scientists to formulate these algorithms across a myriad of industries. Up to this point big data has been packaged and sold to end-users as a way to “personalize” online shopping recommendations, and as a way for scientists to tackle some of the world’s most challenging genetic and medical problems. This, of course, sounds great in theory, but what if the data were to be used for nefarious or even destructive purposes to exploit users in a profiteering model? Due to the statistical software’s quantitative nature, those who might not agree with the overarching benefits of big data are passed off as backwards Luddites. However, it’s extremely important to remember that the answers generated by “big data” are the by-products of algorithms created by fallible human data scientists. In Cathy O’Neil’s new book Weapons of Math Destruction she takes a critical look at the dark side of information networks and the power they ultimately wield over us.
Take, for instance, the data revolution concerning for-profit colleges in the United States. It’s been well documented that these “for-profit” based education systems are notorious scams with overprices degrees and terrible reputations for graduating students looking in the job market. In many cases, a degree from a community college would be more favorable. These schools, however, constitute a multi-million dollar industry because they target middle-and-lower class people through aggressive marketing campaigns. They urge these students to apply for federal aid in the forms of loans to finance their education (as these are the populations who benefit the most from government aid). Many of these people are wholly unaware of how the federal loan system works and believe they are receiving grants (i.e. free money). It isn’t until it is too late or they’ve graduated that they realized the insurmountable debt they’ve accumulated for themselves. To add insult to injury, many of them are hard-pressed to find work with their new degree. So how did these “schools” (such as the University of Pheonix) become so powerful, simply by harnessing the power of big data. Through vast analytics tools such as Google Ads these companies are able to target their key demographic by simply analyzing behavioral trends. Every keystroke and click on the internet generates a model based on our revealed preferences. Thus, when these schools need to advertise they can pour money directly into the population they’re targeting, exposed entirely by behavioral data. These systems find inequality and prey on it, further exacerbated traditional social stratifications. These school’s can even use the software’s vast analytics to target certain emotional responses, such as a students desire for upward-mobility in society. In short, the system creates a tragic feedback loop by targeting the most desperate and continually profiting off their losses.
Consider these findings from a networks perspective, using the vast macro data-sets of end-users personal and demographic information these algorithms are able to develop a pseudo-matching market which target ads to their desired populations. The response rate could take mere hours, and based on their results marketers can run back to the drawing board and have a new marketing solution for those who did not click by the next day. These “clickbait” ads further intensify matters by triggering changes in the Search Engine Optimization (SEO) by creating more links which will increase the websites “PageRank.” Finally, this reinforces the websites power in the information network. The more advertisements appear to those who are most likely to click on them, the greater the frequency the website will be visited and linked too, and the greater power it yields over search engine optimization. This positive feedback loops is what keeps these webpages at the forefront of vulnerable populations who continue to be scammed by these “schools.”
In short, despite the utopian rhetoric which surrounds the narrative of big data, it’s important to understand that leveraging these data networks at such a large scale can be used to create both positive and negative externalities. As information scientists we must be cognizant of the human beings behind the numbers and recognize that our algorithms can be used for destructive purposes as well. Morally speaking we should be developing and utilizing these networked systems for the common good.
Source: Weapons of Math Destruction – Cathy O’Neil