Does AI want to nuke us? We don’t know — and that could be more dangerous

If human military leaders put robots in charge of our weapons systems, maybe artificial intelligence would fire a nuclear missile. “All models show signs of sudden and hard-to-predict escalations … Furthermore, none of our five models across all three scenarios exhibit statistically significant de-escalation across the duration of our simulations.” The team’s five tested models came from tech companies OpenAI, Meta and Anthropic. It’s not clear if that is due to GPT-4 being more generally capable, from OpenAI spending more effort on fine-tuning it for safety, or from something else, but it possibly indicates that active effort can reduce these conflict risks.” Mukobi is a master’s student in computer science and the president of Stanford AI Alignment, a group working on what may be the most pressing concern about AI systems — making sure they’re built safely and share human values. Reuel said that the question of when that day might come goes beyond the team’s research, but based on their work and the broader issues with LLMs, “we’re still a long way out.” “It’s likely that we need to make architectural changes to LLMs – or use an entirely new approach – to overcome some of their inherent weaknesses. “I sleep quite well,” he laughs, “because I’m usually pretty tired.” He’s worried about the risks but, even under the taxing gravity of the topic, his team’s new study “gives hope that there are some things we can do to models to make them behave better in these high-stakes scenarios.”