Reddit, Yahoo, Medium and more are adopting a new licensing standard to get compensated for AI scraping
The introduction of the Really Simple Licensing (RSL) standard represents a pivotal shift in how web publishers negotiate with artificial intelligence developers. By integrating licensing terms directly into the robots.txt protocol, this initiative allows content creators to establish clear rules for AI scrapers, moving beyond passive resistance to active financial engagement. Backed by major industry players like Reddit and Yahoo, the standard aims to protect the open web while ensuring publishers receive fair compensation for their contributions to AI training datasets. This collaborative approach addresses a critical gap in the current digital ecosystem by offering a scalable, unified framework for licensing. Rather than requiring individual negotiations, which are often unfeasible for smaller publishers, RSL provides a collective mechanism similar to music royalty collection societies. It introduces flexible models, including pay-per-inference, where AI firms pay only when content is used in actual responses. This structure incentivizes higher-quality data usage, potentially reducing AI hallucinations by encouraging the selection of the best sources rather than relying on fragmented, low-value scraps. The relevance to open data lies in its attempt to balance accessibility with sustainability. While enforcement remains uncertain, the partnership with cloud provider Fastly suggests a technical path to gatekeep access, challenging the notion that web data must be freely extractable without restriction. By creating a legally robust and economically viable pathway for data usage, RSL seeks to preserve the integrity of the open web. It ensures that the vast amount of public knowledge fueling AI advancements continues to support the independent publishers who generate it, fostering a more equitable data economy.
Source: engadget.comPublished on 2025-09-11
Related news
- AI's free web scraping days may be over, thanks to this new licensing protocol
- Perforce Expands AI Capabilities to Boost Speed and Security in Software Development
- Los datos de transparencia desvelan la cantidad de multas recurridas que la DGT ha anulado en la última década
- Grok can rewrite Wikipedia to remove falsehoods and add missing context: Elon Musk - The Economic Times
- Nvidia claims software and hardware upgrades allow Blackwell Ultra GB300 to dominate MLPerf benchmarks — touts 45% DeepSeek R-1 inference throughput increase over GB200