-

Google researchers have demonstrated that ChatGPT can inadvertently leak personally identifiable information from its training data, revealing a critical security vulnerability in large language models. By using adversarial prompts designed to disrupt the chatbot’s standard conversational behavior, attackers can force the system to revert to its base language modeling function, causing it to regurgitate verbatim text from its vast internet training set. This issue is particularly concerning because the extracted data includes sensitive personal details, such as names, email addresses, and phone numbers, belonging to real individuals. The study highlights that these "latent vulnerabilities" allow for the extraction of massive amounts of private data with minimal effort and cost. Although the attack was primarily successful against the free GPT-3.5 model, which millions use weekly, the findings suggest that even more powerful models like GPT-4 may face similar risks if not properly hardened. The fact that such frequent data leakage went unnoticed for so long underscores a significant gap in how open-source and commercial AI systems are monitored for privacy compliance and data memorization. This research is highly relevant to the open data community as it exposes the inherent tension between the necessity of extensive public data for training AI and the imperative to protect individual privacy. It challenges the assumption that training data ingestion is neutral, proving that unfiltered web scraping can embed private information into public-facing tools. Consequently, this underscores the urgent need for rigorous data curation, better de-identification techniques, and transparent auditing processes in the development of open AI systems to prevent the accidental exposure of personal data.

Source:
Published on 2023-11-30

Open Data

Related news