ChatGPT Struggles With Scientific Facts in New Study

Researchers at Washington State University found that ChatGPT often fails to correctly identify the validity of scientific hypotheses ^[1][2]. In a study involving over 700 hypotheses, the AI performed only slightly better than random chance after adjusting for guessing ^[2][3]. The system particularly struggled when tasked with identifying false statements as incorrect ^[1]. The research highlights a "fluency trap" where the AI uses convincing language to mask significant errors in scientific reasoning ^[1][2]. This inconsistency poses risks for users relying on generative tools for technical information ^[3]. Experts suggest that while the AI appears confident, its underlying logic remains unreliable for complex scientific validation ^[2].