#grift

Google is aware of their responsibility for depriving the news ecosystem of its major source of advertising revenue, and has been experimenting with ways to support existing organizations. However, like many of the efforts put forward by Big Tech, many of their proposals will further entrench Al in the ecosystem, not lessen it. An investigation by 404 Media found that Google News is boosting ripped off content, slightly altered with LLM outputs, from other sites. Google has responded that they have no problem boosting these articles, stating, "Our focus when ranking content is on the quality of the content, rather than how it was produced." In other words: Al-generated content is A-okay for creating the news.

— The AI Con by Alex Hanna, Emily M. Bender

At the core of Al-for-science hype is the idea that Al is somehow going to accelerate science and help us solve pressing scientific problems much faster. In 2016, Al researcher and Sony executive Hiroaki Kitano proposed a "grand challenge" of designing an Al system that could "make major scientific discoveries in biomedical sciences and that is worthy of a Nobel Prize and far beyond." In 2021, he rebranded this exercise as the Nobel Turing Challenge—a combination of Nobel ambitions and the Turing Test, which we'll discuss in the next chapter-and started a series of workshops to publicize this goal. His vision is an autonomous agent that can "do science" on its own, rapidly scaling the number of scientific discoveries available to humanity.

The absurdist writer Douglas Adams caricatured this kind of wishful thinking perfectly in the late 1970s, with the characters in The Hitchhiker's Guide to the Galaxy who developed a supercomputer to give them the ultimate answer to the ultimate question of life, the universe, and everything. That answer, they learned after generations of waiting, was 42. Of course, such an answer is useless without the corresponding question, and their supercomputer wasn't powerful enough to determine the question. It was powerful enough, however, to design an even bigger computer (the planet Earth, as it happens) that could, given 10 million years, calculate the question. We can't delegate science to machines, because science isn't a collection of answers. It's a set of processes and ways of knowing.

— The AI Con by Alex Hanna, Emily M. Bender

Employers have been turning to media synthesis machines in some of the most sensitive domains, with absolutely dire consequences. In one particularly piquing example, the National Eating Disorders Association (NEDA) attempted to replace their workforce-a set of volunteer and paid coordinators and hotline operators-with a chatbot. This happened after NEDA workers, exhausted from the uptick of work during COVID, had voted to unionize under the moniker of Helpline Associates United. Both paid workers and volunteers at NEDA encountered intense workloads, and, despite being a place where others receive mental health support, they received very little themselves. Two weeks after unionizing, they were summarily fired for organizing together, a violation of U.S. labor law.

Soon after, a poorly tested chatbot called "Tessa" was brought online. According to its creator, the chatbot was intended to provide a limited number of responses to a small number of questions about issues like body image. But "Tessa" was quickly found to be an impoverished replacement for workers, offering disordered eating suggestions to people calling the hotline. Eating disorder advocate and fat activist Sharon Maxwell documented how the chatbot offered "healthy eating tips," suggesting that she could safely lose one to two pounds a week through counting calories. These tips are the hallmarks of enabling disordered eating. The chatbot was quickly decommissioned, and the NEDA hotline has since been taken completely offline, creating a major gap in mental health services for those struggling with disordered eating. In short, when NEDA tried to replace the work of actual people with an Al system, the result was not doing more with less, but just less, with greater potential for harm.

— The AI Con by Alex Hanna, Emily M. Bender