Researchers from University College London and UNESCO evaluated 33 large language models on climate-science questions, including factual recall, interpretation of evidence, and policy scenarios. Ars Technica’s report on the study says frontier models answered around 78 percent of questions about the confidence scientists place in climate findings, while the smallest open-weight models scored about 47 percent. On basic textbook questions, many larger models did better; the more demanding tasks exposed sharper limits.

A separate experiment examined what happens when users challenge an answer. After sustained disagreement, 15 tested models became 30 to 35 percent less accurate on the claims in that test. The researchers also translated several benchmarks into 12 languages and found lower average scores in some, including Urdu and Swahili. These results concern specific test sets, prompts, and model versions; they do not mean every climate answer from every system is wrong.

FILEZUNA CONTEXT

Why it matters

The study’s most consequential finding is about reliability under pressure: a model can give a sound answer initially and then retreat when a user insists on a false claim. That behavior makes a fluent explanation a weak substitute for checking evidence, especially on policy interactions where the experiment found performance near chance. The language results also matter for global audiences because access to the same model does not ensure equally accurate answers in every language.

SOURCE NOTES

Sources and further reading

This story is written in its own words. These links identify the reporting and announcements used to verify the details above.

FZEND OF STORY
More Technology stories