As OpenAI unveils o1-preview, its new series of “enhanced reasoning” models, this has prompted warnings from AI pioneer Professor Yoshua Bengio about the potential risks associated with increasingly capable artificial intelligence systems.
According to the company, these new models are designed to “spend more time thinking before they respond,” allowing them to tackle complex tasks and solve harder problems in fields such as science, coding, and math.
This preview version of o1, which went by the codename ‘Project Strawberry’, is now available for ChatGPT Pro subscribers to try out and is also available through OpenAI’s API.
According to OpenAI, the performance of these models is notable:
- In qualifying exams for the International Mathematics Olympiad (IMO), the new model correctly solved 83 percent of problems, compared to only 13 percent solved by its predecessor, GPT-4o.
- In coding contests, the model reached the 89th percentile in Codeforces competitions.
- The model reportedly performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.
OpenAI also introduced o1-mini, a pared down but faster reasoning model it says is particularly effective at coding. It’s 80 percent cheaper than o1-preview, positioning it as a more affordable option for applications requiring reasoning but not broad world knowledge.
However, this advancement has raised concerns among experts in the field. Bengio, a professor of computer science at the University of Montreal and a renowned figure in AI research, issued a warning about the potential dangers of these new models.
“If OpenAI indeed crossed a ‘medium risk’ level for CBRN (chemical, biological, radiological, and nuclear) weapons as they report, this only reinforces the importance and urgency to adopt legislation like SB 1047 in order to protect the public,” Bengio said in a comment sent to Newsweek, referencing the AI safety bill currently proposed in California.
SB 1047 aims to establish safety requirements for “frontier” AI models—advanced systems that could potentially cause catastrophic harm.
He said, “The improvement of AI’s ability to reason and to use this skill to deceive is particularly dangerous. We must put in place regulatory solutions like SB 1047 to ensure developers prioritize public safety at the frontier of AI.”
Patrick T. Fallon/AFP/Getty Images
Newsweek contacted OpenAI via email for comment.
Commenting on the new OpenAI release, Dan Hendrycks, director of the Center for AI Safety, said it “makes one thing clear: serious risk from AI is not some far-off, science-fiction fantasy.”
“The model already outperforms PhD scientists most of the time on answering questions related to bioweapons. OpenAI continues to increase their own estimates of the risks of these models, and further increases are sure to come,” Hendrycks told Newsweek.
“Critics of SB 1047 have asked for evidence before requiring safety testing and guardrails—what would they say is enough? We need to stop with the delay tactics and make SB 1047 law,” he said.
Abigail Rekas, copyright and access law and policy scholar at the University of Galway, Ireland, explained that while the bill is not targeting current AI systems, it sets a framework for future, more advanced models.
Models that meet specific criteria for causing severe harm include those used to create or deploy chemical, biological, radiological, or nuclear weapons resulting in mass casualties or at least $500 million in damages, or those causing similar damage through cyberattacks on critical infrastructure.
To qualify, the model must have cost over $100 million to train and require substantial computing power (10^26 FLOP or equivalent). Additionally, the AI model must be the sole means by which the harm could have occurred, and the developer must have failed to exercise reasonable care to prevent such unreasonable risks.
“The obligations imposed by the bill essentially detail what taking reasonable care looks like, including creating a kill switch and developing a means to determine whether it is likely that the AI will behave in a way likely to cause the harms mentioned above,” Rekas told Newsweek.
But what legal challenges might arise in determining causation between an AI model and “catastrophic harm” in potential lawsuits? “It will require proving essentially that, but for the frontier AI model, the harm (specified) would not have occurred. I have no idea how difficult this will be, as these systems do not yet exist and how that harm might occur remains purely speculative,” said Rekas.
OpenAI says it has taken steps to address safety concerns with its new models. The company states it has developed a new safety training approach that leverages the models’ reasoning capabilities to better adhere to safety and alignment guidelines. In one of their hardest “jailbreaking” tests, the o1-preview model scored 84 out of 100, compared to GPT-4o’s score of 22.
The company says it has also ramped up its safety work, internal governance, and federal government collaboration. This, it says, includes rigorous testing and evaluations using its ‘Preparedness Framework’, best-in-class red teaming, and board-level review processes.
OpenAI has recently formalized agreements with the U.S. and U.K. AI Safety Institutes, granting them early access to a research version of this model. This collaboration, it says, is an important step in establishing a process for research, evaluation, and testing of future models prior to and following their public release.
