Cathy O'Neil QUOTES

101 " I post a petition on my Facebook page. Which of my friends will see it on their news feed? I have no idea. As soon as I hit send, that petition belongs to Facebook, and the social network’s algorithm makes a judgment about how to best use it. It calculates the odds that it will appeal to each of my friends. Some of them, it knows, often sign petitions, and perhaps share them with their own networks. Others tend to scroll right past. At the same time, a number of my friends pay more attention to me and tend to click the articles I post. The Facebook algorithm takes all of this into account as it decides who will see my petition. For many of my friends, it will be buried so low on their news feed that they’ll never see it. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

102 " In both cultures, wealth is no longer a means to get by. It becomes directly tied to personal worth. A young suburbanite with every advantage—the prep school education, the exhaustive coaching for college admissions tests, the overseas semester in Paris or Shanghai—still flatters himself that it is his skill, hard work, and prodigious problem-solving abilities that have lifted him into a world of privilege. Money vindicates all doubts. They’re eager to convince us all that Darwinism is at work, when it looks very much to the outside like a combination of gaming a system and dumb luck.
In both of these industries, the real world, with all of its messiness, sits apart. The inclination is to replace people with data trails, turning them into more effective shoppers, voters, or workers to optimize some objective. This is easy to do, and to justify, when success comes back as an anonymous score and when the people affected remain every bit as abstract as the numbers dancing across the screen. More and more, I worried about the separation between technical models and real people, and about the moral repercussions of that separation. In fact, I saw the same pattern emerging that I’d witnessed in finance: a false sense of security was leading to widespread use of imperfect models, self-serving definitions of success, and growing feedback loops. Those who objected were regarded as nostalgic Luddites. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

103 " Justice cannot just be something that one part of society inflicts upon the other. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

104 " Someone who takes the trouble to see her file at one of the many brokerages, for example, might see the home mortgage, a Verizon bill, and a $ 459 repair on the garage door. But she won’t see that she’s in a bucket of people designated as “Rural and Barely Making It,”or perhaps “Retiring on Empty. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

105 " Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

106 " By the end of the meeting, one conscientious human being had cleared up the confusion generated by web-crawling data-gathering programs. The housing authority knew which Catherine Taylor it was dealing with. The question we’re left with is this: How many Wanda Taylors are out there clearing up false identities and other errors in our data? The answer: not nearly enough. Humans in the data economy are outliers and throwbacks. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

107 " Dios los cría y, estadísticamente hablando, ellos efectivamente se juntan. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

108 " Giant sectors of the economy are organized and optimized to make us feel horrible. "

― Cathy O'Neil , The Shame Machine: Who Profits in the New Age of Humiliation

109 " The primary purpose of shame is to enforce conformity. This, I know, is a problematic word. It signals spinelessness, sheeplike behavior, the sacrifice of one’s individuality for the collective. Worse, the conventions of the group we conform to can be flawed or unjust. "

― Cathy O'Neil , The Shame Machine: Who Profits in the New Age of Humiliation

110 " This was the Big Data economy, and it promised spectacular gains. A computer program could speed through thousands of résumés or loan applications in a second or two and sort them into neat lists, with the most promising candidates on top. This not only saved time but also was marketed as fair and objective. After all, it didn’t involve prejudiced humans digging through reams of paper, just machines processing cold numbers. By 2010 or so, mathematics was asserting itself as never before in human "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

111 " To create a model, then, we make choices about what’s important enough to include, simplifying the world into a toy version that can be easily understood from which we can infer important facts and actions. We expect it to handle only one job and accept that it will occasionally act like a clueless machine, one with enormous blind spots. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

112 " This was the Big Data economy, and it promised spectacular gains. A computer program could speed through thousands of résumés or loan applications in a second or two and sort them into neat lists, with the most promising candidates on top. This not only saved time but also was marketed as fair and objective. After all, it didn’t involve prejudiced humans digging through reams of paper, just machines processing cold numbers. By 2010 or so, mathematics was asserting itself as never before in human affairs, and the public largely welcomed it. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

113 " Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that’s something only humans can provide. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

114 " In WMDs, many poisonous assumptions are camouflaged by math and go largely untested and unquestioned. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

115 " There’s a paradox here. If we return one last time to that ’50s-era banker, we see that his mind was occupied with human distortions—desires, prejudice, distrust of outsiders. To carry out the job more fairly and efficiently, he and the rest of his industry handed the work over to an algorithm. Sixty years later, the world is dominated by automatic systems chomping away on our error-ridden dossiers. They urgently require the context, common sense, and fairness that only humans can provide. However, if we leave this issue to the marketplace, which prizes efficiency, growth, and cash flow (while tolerating a certain degree of errors), meddling humans will be instructed to stand clear of the machinery. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

116 " Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

117 " Predictive models are, increasingly, the tools we will be relying on to run our institutions, deploy our resources, and manage our lives. But as I’ve tried to show throughout this book, these models are constructed not just from data but from the choices we make about which data to pay attention to—and which to leave out. Those choices are not just about logistics, profits, and efficiency. They are fundamentally moral. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

118 " The trouble was that the rankings were self-reinforcing. If a college fared badly in U.S. News, its reputation would suffer, and conditions would deteriorate. Top students would avoid it, as would top professors. Alumni would howl and cut back on contributions. The ranking would tumble further. The ranking, in short, was destiny. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

119 " The trouble is that profits end up serving as a stand-in, or proxy, for truth. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

120 " If language and child care issues posed problems for otherwise solid candidates, the solution was not to reject those candidates but instead to provide them with help—whether English classes or onsite day care—to pull them through. This is a point I’ll be returning to in future chapters: we’ve seen time and again that mathematical models can sift through data to locate people who are likely to face great challenges, whether from crime, poverty, or education. "

― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy