Gov. Gavin Newsom issued an artificial intelligence executive order on Sept. 18 to strengthen oversight of powerful AI systems. The order increases the speed of safety oversight and requires independent verification of their AI organization, according to Newsom’s website.
“Governor Newsom’s executive order is a rare example of government action to address key threats posed by AI,” said Patrick Lin, a Cal Poly philosophy professor.
The executive order is already in effect. It directs state agencies to accelerate implementation of existing laws, such as x and y. It additionally requires the agencies to develop recommendations for potential changes to state law. It does not yet impose Newsom’s proposed requirements for embedded independent oversight or emergency shutoff systems on AI developers.
“Given all the breathless hype around AI, national governments naturally see it as a technology that can potentially supercharge productivity and knowledge production — boosting their economy, military, and everything in between — and so they’re reluctant to do anything that would prematurely slam the brakes on AI progress,” Lin said. “Yet at the same time, it’s clear to most people who are paying attention that there’s no responsible driver behind the wheel of AI, and it may very well drive us off a cliff.”
The order calls for recommendations in four areas:
- It would speed up independent oversight of the organization, meaning an outside group would assess AI safety.
- It would require the safety frameworks, transparency reports and filed risk assessments to be independently verified. This would work to study stronger safety requirements inside major AI labs and check their safety reports.
- They would explore a “kill switch,” which is for emergencies and a way to stop an advanced AI system if it becomes dangerous or people lose control of it. It would also be verified by an independent organization.
- The order calls for recommendations to expand which safety incidents AI companies must report, including incidents involving a loss of control over their systems.
Newsom emphasizes the importance of “updating the definition of critical safety incidents that AI companies are required to report.”
Yet, Newsom is working to bring in outside national experts, who will help advise California on how to strengthen oversight of advanced AI. These experts will work to examine the ordered safeguards.
Companies are currently not required to install kill switches. Advisers have until Nov. 16 to turn in final AI recommendations and full guidelines.
“So, this executive order is at least forcing the hard conversation needed about addressing AI risks, many of which are clear and known,” Lin said. “It might not be the perfect solution, but that’s not its goal — it’s only to stop the bleeding, until we can figure out something better.”
The order aims to have a long-term impact with stronger oversight on AI’s usage, testing, and release.
“Perhaps we’ll see more technical guardrails, such as AI that refuses to respond to certain requests, [like] when it’s inappropriate in a learning environment,” Lin said.“If people are using AI inappropriately in those ways, they might have reason to be worried that they can’t keep doing that in the future, but I wouldn’t expect ordinary users to see much change if any.”
Clara Fulks, a digital ethicist and AI safety specialist, adds her advice to students.
“Students are training for a workforce that is going to look completely different by the time they graduate. My advice would be to not focus on how to use AI at all, and instead focus on how to use your brain. Sharp, cunning critical thinking skills will be the most valuable asset a person can have coming into this new world.”
Talia Rabkin, a freshman, affirmed the idea of creating safety mechanisms.
“I think that’s a good idea because you can never be too careful with new technology and artificial intelligence,” she said. “Better to be safe than sorry.”
“But remember, this is also a people problem — most of technology ethics is really just about human ethics in a wired world,” Lin said.