Critical infrastructure: automated decision systems: human oversight.

Senator McNerney's proposal to regulate artificial intelligence systems in California's critical infrastructure establishes new oversight requirements for state agencies managing essential services like transportation, energy, and emergency response networks.

The legislation requires state agencies to implement human monitoring of AI systems in real-time and obtain human approval before executing AI-proposed actions. The Department of Technology would provide specialized training in AI safety protocols and risk management for oversight personnel. However, the human oversight requirement includes an exemption for existing automated systems where immediate implementation would destabilize critical operations.

Under the bill, agencies must conduct annual assessments examining their AI systems' compliance, performance, safety, and potential vulnerabilities - including risks that could lead to mass casualty events. These evaluations would be submitted to the Department of Technology and coordinated with existing risk analysis requirements. The measure defines critical infrastructure as systems whose disruption would severely impact public health, safety, or economic security, encompassing sectors like communications, financial services, and food and agriculture.

The proposal creates specific parameters around automated decision systems, which it defines as computational processes that generate simplified outputs like scores or recommendations that substantially affect individuals. Common tools like spam filters, firewalls, and calculators are explicitly excluded from these requirements.