What should be in this site’s robots.txt as it relates to scraping/training for AI agents? It would seem hypocritical if you opened up this site’s robots.txt and see AI agents explicitly denied, no?
OR, would it be prudent to disallow AI agents based on the fact much of the content found here was AI generated or AI assisted? Should I try to prevent “model collapse” by not let AI train on itself?