Can Artificial Intelligence Avoid the Enshittification Trap?
I recently went on vacation In Italy as one does these days, I scoured my itinerary from GPT-5 for sightseeing suggestions and restaurant recommendations. The bot reported that the best choice for dinner near our hotel in Rome was a short walk down Via Margotta. It was one of the best meals I remember. When I got home, I asked the model how she chose that restaurant, which I hesitate to reveal here in case I want a table in the future. Among these factors were rave reviews from locals, notices in food blogs and the Italian press, and the restaurant’s renowned blend of Roman and contemporary cuisine. Oh, and short walks.
Something was also needed from my end: trust. I had to accept the idea that GPT-5 was an honest broker and would choose my restaurant without prejudice. That the restaurant won’t show me as sponsored content and won’t receive my check. I could have done some deep research on my own to double check the recommendation (I searched the website), but the purpose of using AI is to bypass this friction.
This experience strengthened my confidence in the results of AI, but also made me wonder: As companies like OpenAI become more powerful and try to compensate their investors, will AI be prone to the erosion of value that seems to be the case for the technology applications we use today?
Play with words
Technology writer and critic Corey Doctoroff calls this erosion “depression.” His premise is that platforms like Google, Amazon, Facebook, and TikTok start out with the goal of satisfying users, but once the companies beat the competition, they intentionally become less useful for making big profits. After WIRED republished Doctorow’s pioneering 2022 paper on the phenomenon, the term entered the vernacular, largely because people realized it was downright dangerous. Enshittification was chosen as the word of the year 2023 by the American Dialect Association. The concept has been invoked so much that it has gone beyond slang and appears in places where such a word would normally be encountered. Doctorow has recently published a book with the same name in this field. The cover image is an emoji for… guess what.
If chatbots and AI agents get worse, it could be worse than Google search being underutilized, Amazon results plagued with ads, and even Facebook showing less social content in favor of infuriating clickbait.
Artificial intelligence is on its way to being a constant companion and providing instant answers to many of our requests. People now rely on it to help interpret current events and get advice on all kinds of purchasing choices — and even life choices. Due to the huge costs of creating a full-fledged AI model, it is fair to assume that only a few companies will dominate this field. They all plan to spend hundreds of billions of dollars over the next few years to improve their models and get them into the hands of as many people as possible. Right now, I’d say AI is in what Doctorow calls the “good for users” phase. But the pressure to recoup huge investments will be enormous – especially for companies with a locked-in user base. As Doctorow writes, these conditions allow companies to exploit their users and business customers to “reclaim the entire value for themselves.”
When one imagines AI mischief, the first thing that comes to mind is advertising. The nightmare is that AI models make recommendations based on what companies have paid for placements. This is not happening now, but AI companies are actively exploring the advertising space. In a recent interview, OpenAI CEO Sam Altman said, “I believe there’s probably an interesting ad product we can do that’s a net win for the user and kind of positive for our relationship with the user.” Meanwhile, OpenAI recently announced a deal with Walmart to allow the retailer’s customers to shop from within the ChatGPT app. You can’t imagine the conflict there! AI search platform Perplexity has a program where sponsored results appear in clearly labeled searches. But, it promises, “these ads don’t change our commitment to maintaining a reliable service that provides direct and unbiased answers to your questions.”
