Home > Author > Cathy O'Neil
61 " Poverty, in much of the world, is so tainted by shame that people who are poor go to great lengths not to associate themselves with it, and to contrast their position, favorably, with those who are even poorer. "
― Cathy O'Neil , The Shame Machine: Who Profits in the New Age of Humiliation
62 " A model, after all, is nothing more than an abstract representation of some process, be it a baseball game, an oil company’s supply chain, a foreign government’s actions, or a movie theater’s attendance. Whether it’s running in a computer program or in our head, the model takes what we know and uses it to predict responses in various situations. All of us carry thousands of models in our heads. They tell us what to expect, and they guide our decisions. "
― Cathy O'Neil , Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
63 " The input to my internal cooking model is the information I have about my family, the ingredients I have on hand or I know are available, and my own energy, time, and ambition. The output is how and what I decide to cook. I evaluate the success of a meal by how satisfied my family seems at the end of it, how much they’ve eaten, and how healthy the food was. Seeing how well it is received and how much of it is enjoyed allows me to update my model for the next time I cook. The updates and adjustments make it what statisticians call a “dynamic model. "
64 " Seven years after A Nation at Risk was published with such fanfare, researchers at Sandia National Laboratories took a second look at the data gathered for the report. These people were no amateurs when it came to statistics—they build and maintain nuclear weapons—and they quickly found the error. Yes, it was true that SAT scores had gone down on average. However, the number of students taking the test had ballooned over the course of those seventeen years. Universities were opening their doors to more poor students and minorities. Opportunities were expanding. This signaled social success. But naturally, this influx of newcomers dragged down the average scores. However, when statisticians broke down the population into income groups, scores for every single group were rising, from the poor to the rich. "
65 " This part of the analysis, like any collection of human opinion, was sure to include old-fashioned prejudice and ignorance. It tended to protect the famous schools at the top of the list, because they were the ones people knew about. And it made it harder for up-and-comers. "
66 " Ill-conceived mathematical models now micromanage the economy, from advertising to prisons. "
67 " They draw statistical correlations between a person’s zip code or language patterns and her potential to pay back a loan or handle a job. These correlations are discriminatory, and some of them are illegal. "
68 " A young suburbanite with every advantage—the prep school education, the exhaustive coaching for college admissions tests, the overseas semester in Paris or Shanghai—still flatters himself that it is his skill, hard work, and prodigious problem-solving abilities that have lifted him into a world of privilege. Money vindicates all doubts. "
69 " A model’s blind spots reflect the judgments and priorities of its creators. After all, a key component of every model, whether formal or informal, is its definition of success. This is an important point that we’ll return to as we explore the dark world of WMDs. In each case, we must ask not only who designed the model but also what that person or company is trying to accomplish. "
70 " A model’s blind spots reflect the judgments and priorities of its creators. "
71 " What’s different here is the focus on the proxy when far more relevant data is available. I cannot imagine a more meaningful piece of data for auto insurers than a drunk driving record. It is evidence of risk in precisely the domain they’re attempting to predict. It’s far better than other proxies they consider, such as a high school student’s grade point average. Yet it can count far less in their formula than a score drawn from financial data thrown together on a credit report (which, as we’ve seen, is sometimes erroneous). "
72 " According to a survey by the Society for Human Resource Management, nearly half of America’s employers screen potential hires by looking at their credit reports. Some of them check the credit status of current employees as well, especially when they’re up for a promotion. "
73 " The national drugstore chain CVS announced in 2013 that it would require employees to report their levels of body fat, blood sugar, blood pressure, and cholesterol—or pay $600 a year. "
74 " Los puntos ciegos de un modelo reflejan las opiniones y prioridades de sus creadores. "
75 " No obstante, el modelo seguiría cometiendo errores, ya que todo modelo es, por su propia naturaleza, una simplificación. Ningún modelo puede incluir toda la complejidad del mundo ni los matices de la comunicación humana. Es inevitable que parte de la información importante se quede fuera. "
76 " But a crucial part of justice is equality. And that means, among many other things, experiencing criminal justice equally. People who favor policies like stop and frisk should experience it themselves. Justice cannot just be something that one part of society inflicts upon the other. "
77 " My point is that police make choices about where they direct their attention. Today they focus almost exclusively on the poor. That’s their heritage, and their mission, as they understand it. "
78 " Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that’s something only humans can provide. We have to explicitly embed better values into our algorithms, creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit. In "
79 " Note that there’s no option to answer “all of the above.” Prospective workers must pick one option, without a clue as to how the program will interpret it. And some of the analysis will draw unflattering conclusions. If you go to a kindergarten class in much of the country, for example, you’ll often hear teachers emphasize to the children that they’re unique. It’s an attempt to boost their self-esteem and, of course, it’s true. Yet twelve years later, when that student chooses “unique” on a personality test while applying for a minimum-wage job, the program might read the answer as a red flag: Who wants a workforce peopled with narcissists? "
80 " If it was true during the early dot-com days that “nobody knows you’re a dog,” it’s the exact opposite today. We are ranked, categorized, and scored in hundreds of models, on the basis of our revealed preferences and patterns. "