Authority Marketing

Authority Marketing

Share this post

Authority Marketing
Authority Marketing
The Biggest Websites in the World are Blocking Large Language Models
Copy link
Facebook
Email
Notes
More

The Biggest Websites in the World are Blocking Large Language Models

The big boys don't want their content scraped by AI

Tetra's avatar
Tetra
Sep 21, 2023
∙ Paid
9

Share this post

Authority Marketing
Authority Marketing
The Biggest Websites in the World are Blocking Large Language Models
Copy link
Facebook
Email
Notes
More
3
Share

AI companies built their large language models on the backs of other people’s content.

That’s just a fact.

You can decide for yourself whether or not that’s ethical (I think it isn’t), but the facts are the facts. It’s the world we live in and we all have to decide how to react to it.

The big boys of the internet are reacting by signaling that they DON’T want their content scraped by LLMs.

A new study from Orginality.AI goes into the details.

Let’s take a look and find out what it means for your site. I’ll also go over the steps you should take if you want to do the same to protect your content.

Results of the study

About 18.6% of the top 1000 websites are blocking at least one AI bot.

Broken down by crawler, the data shows that:

  • 111 sites blocked GPTbot

  • 32 blocked ChatGPT-User (plugin bot)

  • 63 blocked CCBot

  • None blocked Anthropic-AI

The number of major sites blocking GPTbot is increasing.

Rapidly.

Some of the biggest sites blocking ChatGPT are:

  • Amazon

  • New York Times

  • Shutterstock

  • Wikihow

  • CNN

  • Quora

  • Business Insider

  • Washington Post

  • Dictionary.com

  • Tumblr

  • IKEA

Here’s the spreadsheet with the full list: https://docs.google.com/spreadsheets/d/1KR7PXruxpMBMZoUkPNNY8foUwGx1GjgMAiVWseGVHGw/edit#gid=0

Why are sites blocking some but not all AI bots?

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 BowTiedTetra
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More