AIs get worse at answering simple questions as they get bigger
Large language models become increasingly unreliable when scaled up or fine-tuned for human feedback, showing improved performance on complex tasks but worsening accuracy on simple queries. This divergence suggests that as models attempt to answer every question, their refusal rates drop while the probability of providing incorrect answers rises. Consequently, users may encounter confident but wrong responses to basic inquiries, undermining the perceived competence of these systems. The study highlights a critical lack of self-awareness in AI, which fails to recognize the boundaries of its own knowledge unlike humans. This limitation creates a dangerous environment where developers often portray AI as omniscient, encouraging an unhealthy level of user trust and overreliance. The discrepancy between the system’s confidence and its actual error rate poses a significant risk to decision-making processes that depend on accurate information. This finding is crucial for open_data because it underscores the necessity of rigorous, transparent evaluation protocols. Open data initiatives can help identify these systematic failures by providing diverse, standardized benchmarks for testing model reliability. By making evaluation data accessible, the community can develop better metrics for truthfulness and uncertainty, ensuring that AI transparency leads to safer and more accountable technologies.
Source: newscientist.comPublished on 2024-10-03