A photo of the dome is taken at the State Capitol in Sacramento, California, on Monday, Aug. 5, 2024. (AP Photo/Juliana Yamada)
Copyright 2024 The Associated Press. All rights reserved.
A bill to regulate artificial intelligence companies in California has cleared a significant hurdle in the California Assembly's Appropriations Committee, but passage is far from assured. The bill passed the California Senate by a 32-1 vote earlier this year and will now go to the full state Assembly for a vote. A date for that vote has not yet been set, but the state legislative session ends on August 31, leaving a short window of time for the bill to pass the Assembly.
If it passes the legislature, the bill would then go to California Governor Gavin Newsom (D), who would have until the end of September to decide whether to sign or veto the bill. Newsom has not commented on the bill but has previously said that if the state “over-regulates,” California “may cede this territory to other states or other countries.”
California Senator Scott Wiener's (Democrat) proposal would require safety testing of cutting-edge AI models, defined as models that cost more than $100 million to develop or meet a certain level of computing power. Such safety testing would be required before models are made publicly available. Additionally, the California Attorney General would have the power to sue companies if their technology is found to have caused significant harm.
As reported by The New York Times and TechCrunch, the bill has drawn pushback from many technology companies, including major tech companies such as Google, Meta, Anthropik, and OpenAI, as well as venture capital funds such as A16z and Y Combinator. The companies' concern is that while the bill is a potential state law, it could become a de facto national standard if other states copy the bill or if it becomes easier to comply with the law nationwide than to lay out specific rules for California users. Critics also argue that if too restrictive, the bill could stifle innovation or push AI development out of California, mainly due to the amount of liability companies would have to deal with under the requirements.
Notably, some of the criticism of the bill has come from California Democrats, including former House Speaker Nancy Pelosi (D-CA). In a statement, Pelosi said the bill is “well-intentioned but ill-informed.” She supported establishing protections and guardrails for AI, but did not offer any specific proposals herself. Other California Democrats who criticized the bill include Reps. Zoe Lofgren (D-CA) and Ro Khanna (D-CA), who have urged Governor Newsom to veto the bill if it passes.
The state Assembly's Appropriations Committee amendments included industry-friendly changes such as removing a provision that would have established a state-run Frontier Models Division to certify AI developers, eliminating civil penalties for violations that do not pose harm or risk, and adjusting language to require companies to use “reasonable care” rather than “reasonable assurance” to prevent harm from AI systems. Enforcement power remains in the hands of the state attorney general.
Many of these changes mirror those proposed by Anthropological Society President Buffy Wicks in a letter published in July. The most significant appears to be an adjustment of the language from “reasonable assurance” to “reasonable care.”
Wiener accepted the amendments, but it is unclear whether they will be enough to quell opposition from AI companies, investors, and lawmakers. As the bill heads to the California Assembly, further comments are likely in the coming days, indicating how these amendments change the bill's chances of passing the Assembly. Newsom's silence on the issue suggests that if frustration remains but is not enough to block the bill in the Assembly, he could be persuaded to veto the bill, primarily if opponents can convince him that the bill does more harm than good for the AI industry.
