Review: Empire of AI

 Empire of AI is about OpenAI, with the inside story about how a couple of years ago over a weekend, Sam Altman got fired by the non-profit board and then got himself brought back. It's got all the juicy details behind that backstory, and describes a high level of corporate politics that I'm glad I've never had to witness first hand, despite having various toxic managers in the course of my long career in Silicon Valley.

The book is also about a lot more, such as the environmental and social costs of the scaling up of modern LLMs. The problem with the author's approach, however, is that she appears to be writing for a non-technical audience, giving you abstract ideas of the environmental costs of serving a gemini query vs a regular google search. I would have rather gotten absolute numbers. There's an entire section on how the human reinforcement folks in 3rd world countries working for scale.ai or one of those analogues suffered from mental health problems. But any skeptical reader would see that the fact that those workers continued working indicated that they didn't actually have any better jobs available, and that the AI sweatshops were actually the best jobs they could find!

Ultimately, I came away with an understanding of how Sam Altman could get so much resources to build massive data centers and consumer 40% of the world's RAM production. But if you had a computer scientist's mind, you'd also realize that the current LLM approach is already hitting their limits --- when you've already added all of reddit and 4chan into your training corpus and is already using video to speech to create more text for your LLM to train on, where's the next order of magnitude of scaling up going to come from? Sooner or later you're going to hit diminishing returns (assuming you haven't already), and I'm guessing that eventually the lawsuits about OpenAI and other companies illegally scraping content they didn't have permission to use will get to the point where those creators would have to be compensated.

Anyway, the book is good reading (but you need to read it with skepticism), and I'm guessing the OpenAI employees are getting paid so much so they will happily put up with a level of toxic politics that I've never encountered.