News organisations are forced to accept Google AI crawlers, says FT policy chief
Matt Rogerson argues that Google’s shifting use of publisher content for AI development has fundamentally broken the long-standing social contract of the open web. By scraping articles to fuel AI Overviews and selling data to third-party large language models, Google extracts significant commercial value without compensating creators or driving necessary user engagement. This undermines the economic model of news organizations, which rely on web traffic to sustain their operations and quality journalism. The article highlights a coercive dynamic where publishers lack a genuine choice. To protect their intellectual property from unauthorized AI use, sites must opt out of Google’s search crawler entirely, effectively becoming invisible to the majority of online users. This dilemma exposes the fragility of current digital infrastructure, where dominance in search provides tech giants with unparalleled access to content, forcing publishers to choose between financial survival and the protection of their IP rights. This situation is critical to open data because it illustrates how proprietary control over information ecosystems can erode the transparency and accessibility that open principles champion. When incumbent platforms scrape content without fair licensing or reciprocity, it hinders the development of equitable data markets. Establishing consistent, compensated licensing agreements is essential to ensure that the value generated by open data and high-quality content is shared fairly, preventing the monopolization of digital resources by a few dominant technology corporations.
Source: pressgazette.co.ukPublished on 2024-11-07