Ero

AI Progress Will Be Unevenly Distributed

Cover Image for AI Progress Will Be Unevenly Distributed

(rearing my head after a hectic 7 months)

Technology adoption is not limited by the rate of discovery, but by the pace of societal adoption.

And we seem to be adapting well to LLMs.

Lots of words are being generated now.

Making something easy to do doesn’t necessarily mean more of it will be done. People still need to care about doing the thing. For example, top universities began publishing their entire course catalogs online in the late 2010s. The cumulative effect is unlike what I would've expected if someone told me 20 years ago that "Harvard is now free online." (I've updated my world model accordingly since)

What is working?

There have been some great use-cases: 1) Note-taking (expected), 2) Popular Media (unexpected1), and 3) Coding (obvious in hindsight). This led me to believe enterprises would be able to immediately realize value. Yes and no.

Note Taking

LLMs are great at summarization. The best meeting notes are summaries. So it follows that AI note takers have seen high adoption because people don't really read meeting notes2, but note taking has always seemed like important work.

If you throw a big blob of AI-generated text at people, they'll pick what they need from it and ignore the rest, assuming their first thought isn't to send it to another AI for summarization. Less is more.

In Popular Media

I met a guy recently who runs an interesting business. He helps people build social proof for their businesses by getting them backlinks3 in popular media. How does it work?

Considering airplanes are acting out these days. It could be profitable to be known as an aviation expert. You get TV airtime the more planes continue to act out.

How do you get on the TV?

Someone has to vouch for you being an aviation expert. Maybe some small online magazine in need of all the traffic they can get. Remember, bad news drives clicks.

But in order to give your piece credibility, you need an expert to comment on it.

Here's where our guy comes in.

Anytime an airplane misbehaves, our guy AI generates a bunch of quotes and sends them to as many reporters as he can.

While they're prepping their article, they may pick a quote and link back to the profile of the up and coming expert. The up and coming expert pays monthly until they get enough links to prove their expertise.

When said expertise is established, they can go and monetize it however they want.

If popular media is not truth-seeking, then truth is not a barrier to adopting new tech.

Coding

Code is abundant in LLM training data. In fact, increasing the volume of code in the training data generally improves model quality. Additionally, good code arises from intent split into logical steps. And LLMs do well on tasks where they are provided high-context + specificity, and narrow scope.

And in the enterprise?

AI adoption for work will be take a long time for two reasons:

  1. Organizations still have trouble organizing and searching their data. Most tasks are downstream of well-organized, easily-retrievable data.
  2. In order to prompt a machine, you have to provide clear instructions. Most middle managers aren't good at this. In fact, the reason most organizations (with a lot of middle management) function is that their subordinates have good incentives to figure out what their bosses actually mean. Call it managing up.

...

Thanks to Surakat and Mohammad for reading early drafts of this.

Footnotes

  1. Until I realized how things really worked, I assumed most news media was truth-seeking. But the news is always presented from a perspective -- it has to be. Alignment is determined by your frame of reference. In fact, my current thinking is that the current, most aligned form of news media is financial news seeing as the incentives generally flow in the same direction for most major financial events. ↩

  2. I was once in meeting with a person who had 3 AI note takers join -- he was very proud of it too. When the meeting was done, he shared walls of text that his notetakers produced. The next time we met, he had a very poor grasp on the details of our last meeting. What was the point? After reading a draft of this, my friend said "We’ve banned it from meetings - Only the Meeting host can have it, and then they circulate the notes... Sometimes, I’ve had meetings with more Note takers sent to it than Humans 😂." Touche. ↩

  3. Backlinks are an important part of search engine optimization. More backlinks increase your chances of ranking higher in search results. This is very important when trying to establish expertise. ↩