Glad you are coming around to our POV. Better late than never. Different types of content have different purposes. Of course you want product documentation consumed by all types of AI, but if you are a creator that is monetizing content with ads probably not.
Guillermo Rauch
Guillermo Rauch16.8. klo 04.23
The three types of AI traffic: 1️⃣ AI training crawlers 2️⃣ AI engine grounding 3️⃣ AI source referrals Each needs to be understood and measured differently. It might or might not make sense to block some of these, depending on your business. Any default or broad stokes blocking decision by your infrastructure provider is likely to be harmful. Here's how to think about it:
33,37K