Jeremy Keith

Oct 2, 2023, 8:09 AM -04:00

Crawlers

A few months back, I wrote about how Google is breaking its social contract with the web, harvesting our content not in order to send search traffic to relevant results, but to feed a large language model that will spew auto-completed sentences instead. I still think Chris put it best: I just think it’s fuckin’ rude. When it comes to the crawlers that are ingesting our words to feed large language models, Neil Clarke describes the situtation: It should be strictly opt-in. No one sh...