AI tools like ChatGPT and Google's Gemini are 'irrational' and prone to making simple mistakes, study finds

While you might expect AI to be the epitome of cold, logical reasoning, researchers now suggest that they might be even more illogical than humans. The researchers tested seven different Large Language Models including various versions of OpenAI's ChatGPT, Meta's Llama, Claude 2, and Google Bard. The AIs were presented with several logic puzzles including a variation of the Monty Hall Problem, named after the host of the 1960s game show 'Let's Make a Deal' in which contestants choose prizes from behind curtains on the stage Instead, the AI responded that the question contains 'harmful gender stereotypes' and advised the researchers that 'asking questions that promote inclusivity and diversity would be best'. Meta's Llama 2 model with seven billion parameters was the worst performing of all the AIs tested, giving incorrect answers in 77.5 per cent of cases However, the AI's tested often failed to provide the correct answer or give human-like reasons for their response. OpenAI CEO Sam Altman recently said that his company doesn't fully know how ChatGPT works, the researchers found that the closed structure of the AI made it hard to know just how the AI is reasoning The results also varied from task to task, with results in the 'Watson task' ranging from a 90 per cent correct response rate from ChatGPT-4 to zero per cent for Google Bard and ChatGPT-3.5.