The New York Times and Other Top News Sites Block SearchGPT Web Crawling Bot Amidst AI Concerns - TechStory

Major news publishers, including The New York Times and Condé Nast titles, have blocked OpenAI’s SearchGPT crawler, signaling a growing resistance to AI integration in content distribution. This move stems from deep-seated concerns regarding content misuse and the erosion of direct web traffic. Publishers fear that despite OpenAI’s assurances that the bot only indexes data for search results rather than training models, their valuable journalistic content may still be exploited without permission or compensation. The skepticism is rooted in past experiences where OpenAI collected data for model training, leading to legal action and strained relationships. Publishers are particularly wary of the shift from traditional search engines, which drive traffic to original sources, to AI-powered tools that summarize information and keep users within the platform. This dynamic threatens the traditional business model of news organizations, which rely on visitor engagement and advertising revenue rather than being treated merely as data feeders for artificial intelligence systems. This conflict is highly relevant to the open_data community as it highlights the tension between data accessibility and copyright protection. It underscores the urgent need for clear, ethical frameworks governing how web content can be harvested for AI development versus search indexing. As technology evolves, establishing transparent consent mechanisms and fair value exchanges will be critical to ensuring that open data initiatives do not undermine the economic viability of content creators who provide the foundational material for these innovations.

Source: techstory.in
Published on 2024-08-06