climate policy: ChatGPT VS ACADEMICS

A new paper “Comparing advice on climate policy between academic experts and ChatGPT” on Ecological Economics compares the results from a recent global expert survey on climate policy with answers to the same survey by the online artificial-intelligence chatbot ChatGPT.

The comparison provides insights about performance criteria, policy instruments, and use of information from distinct academic disciplines.

With a few exceptions, responses by ChatGPT are informative and of high quality. We find that ChatGPT answers questions with less bias than experts from various scientific disciplines. The latter may also be a disadvantage as it seems to weight all the information available equally without accounting well for relevance, which arguably may require human rather than artificial intelligence.

On the other hand, experts from distinct disciplines show difference in average responses, with some even expressing opinions inconsistent with objective evidence, meaning there is no consistent and unbiased expert opinion on climate policy.

As a new way of synthesizing large amounts of academic and grey literature, ChatGPT can serve policymaking. However, since the procedure that it follows for collecting and summarizing information remains a black box, it is best regarded as a complement rather than a substitute to traditional literature reviews and expert surveys.

• Read the full paper on Ecological Economics, Volume 226, December 2024, 108352