Former President Franklin D. Roosevelt once said, “Great power involves great responsibility.” What he might not have imagined is how powerful models designed to address pressing societal issues end up punishing those they are designed to help. Virginia Eubanks describes the negative impacts of high tech tools in her book Automating Inequality using examples from Indiana, Los Angeles, and Allegheny County, warning citizens of this digital age that the reprehensible legacy of poorhouses is still continued today in another form.

During our discussion of this book, we touched on the topic of increasing digital surveillance. We see ads liked to our search activity on Facebook page, we receive emails from previously visited websites that we didn’t provide an email address to, and we get targeted spam calls from strangers. Digital surveillance is ubiquitous — and it’s acting as a foundation of those digital models that are implicitly controlling our lives. By obtaining data about its target audience, those models are able to make “informed” decisions and provide different “services”. High tech tools compromise our privacy, but their true damage lies in their impact on people in the lower end of the social spectrum. Eubanks describes how automated decision making systems inefficiently and problematically allocate resources spanning from child welfare to housing for the homeless. Instead of providing the most needed essential resources, those systems leave out the impoverished and further widen the social gap. I have been reading a book called Weapons of Math Destruction by Cathy O’Neil for my writing seminar and was surprised by the similarities in main arguments of these two books. Weapons of Math Destruction takes the topic of this book further by showing the negative feedback loop caused by the digital models. People from poor neighborhoods are charged with a higher automobile insurance regardless of their driving behavior, increasing their financial burdens; applicants lacking job experience get screened out by resume screeners, stripping them of needed employment opportunities; lower-ranked colleges struggle to attract top students and faculty, thus their high acceptance rates continue lowering their rankings… It seems that those digital models are like a mire, trapping the needed in a downward spiral that they cannot escape.

Eubanks warns her readers that although the impact is primarily “felt” by poorer individuals, the models’ predatory reach extends far beyond a limited group, and those tools can be turned to target on us. For ordinary citizens, we cannot change the models or escape from their control, but we need to keep our eyes wide open — to look out for those model mires lurking around us, and to be aware when we are being targeted.