The myriad deployment of artificial intelligence across professions has allowed industry leaders and specialists to do work considerably smoother. Neela Paykel, who serves as a tech general counsel in residence with corporate law firm Mayer Brown, views the technology as a great everyday tool.
“You can create personal assistants for yourself,” she said in an interview with Mustang News. “I’ve seen people do amazing things with AI for their daily schedule, and I use it all the time to help me draft better emails and better letters, and get past writer’s block”
Paykel has previously served as general counsel for Hyperfine, Inc., Waymo and other legal advisorship roles with Blue Shield of California, CVS Caremark and Genentech.
“It’s amazing that it’s available, as long as it is used as a tool,” Paykel added.
Yet, the role of AI remains heavily debated by experts across notable areas and sectors. Its presence within the health care industry has stirred worry over patient privacy and technical efficiency and led to staunch retaliations by labor unions, such as the California Nurses Association and legislators alike.
Members of the California Assembly are currently debating the role of AI within the California health care system, with the introduction of AB 1979, according to the CalMatters Digital Democracy database.
The bill, entitled “Health care services: artificial intelligence,” serves as a continuation of AI-related legislation authored by Assemblymember Bonta, according to the CalMatters Digital Democracy database. Bonta’s AB 489, signed into law by Governor Newsom last year, prohibits AI systems from misrepresenting themselves as licensed nurses, doctors or psychologists.
The danger of AI persists in its unregulated use, according to Bonta. AB 1979 will protect the privacy of health care consumers by ensuring professionals’ better judgement is not “replaced or circumvented” by artificially generated outputs.
An April 1 press release emphasized the need for “compassion, empathy, and real-world judgment” in patient care that cannot be captured in patterns and algorithms.
“Technology should assist human clinicians, not replace them. AB 1979 draws that line,” Bonta said in the press release.
Some of the 24 organizations with positions recorded in the bill analysis, who stood against adopting the bill, were Adventist Health, the Advanced Medical Technology Association, the California Hospital Association, Kaiser Permanente and the Civil Justice Association of California.
Out of these organizations, only four stood in support of adopting the bill. These include the California Labor Federation, California Nurses Association, Consumer Watchdog and the California Peer Watch.
What kind of AI technology does the bill concern?
The technology in question concerns the usage of AI-integrated electronic health records that guide health care professionals in a “clinical decision support” capacity. Clinical decision support systems may operate as components of or plug-ins to electronic health records.
As of April 23, the bill directly requires health care entities to exercise “independent professional judgment” when approving a clinical decision that is based on the output of these systems.
What are the concerns regarding patient confidentiality?
The integration of AI into electronic health records, the California Nurses Association argues, risks connecting medical records to large language models that “analyze, sell, or otherwise utilize health data outside the traditional privacy frameworks,” which guard patients’ data.
“If an individual uploads their medical record to ChatGPT Health for advice, existing health privacy laws may not protect that information from being used, stored, or shared for purposes beyond the individual’s request,” the nurses association claimed in their legal fact sheet.
The bill would subject any business that offers a “health care chatbot” to a consumer, for the purpose of allowing them to manage sensitive medical information, to the requirements of California’s Confidentiality of Medical Information Act. The act outlines any “provider of health care, health care service plan, a contractor, or a corporation and its subsidiaries and affiliates” cannot share nor sell medical information for purposes external to the health care services of a patient.
In instances that an AI company or business abuses patient data for purposes outside of providing immediate health care relief laws, such as the Confidentiality Medical Information Act, directly apply.
How could AI improve the health care system?
Electronic health records enable clinical decision support systems to help with diagnoses and track for negative drug interactions with a more complete image of the patient’s data. Under a clinical quality and safety overview by the Office of the National Coordinator for Health IT, these systems help reduce errors and net costs and ease cognitive burdens on providers.
Studies, which include one referenced by the California Nurses Association in their legal fact sheet, demonstrate results that suggest integrating large language models into emergency room workflows could maximize the number of surviving patients, while maintaining the same quality.
“[AI] could be really transformative. It could be a really great way to have some efficiency in the system and help our really busy health care professionals, especially in the emergency room,” Paykel said. “They’re not paid well, and you don’t have a lot of people going into emergency medicine. If we can make their lives easier, it’d be great.”
Jill Speece, a professor of industrial and manufacturing engineering at Cal Poly, previously worked in the health care industry and served as a consultant. She believes AI could greatly benefit the health care system.
“It’s not reasonable to prohibit it when it’s something that could improve health care efficiency, so what we have to do is use it properly,” Speece said.
Annalise Mui, public health junior, plans to become a physician’s assistant. Her research has covered deploying AI for health communication purposes.
Mui considers AI in health care appropriate as a cautious supplement to everyday tasks but emphasizes not falling reliant on it. This is due to adverse impacts it can have on the patient-health care worker relationship.
“AI essentially eliminates that,” she said.
Additionally, the threat of a data breach is “very real,” with more pieces of data becoming connected to technology. However, she deems that the threat does not extend to AI-integrated electronic health records.
“Electronic health records are already built to protect patient privacy,” Mui said. “There’s no need to automate or use AI for those kinds of data storage, especially because of the amount of time it would take to train a model to think outside of textbook scenarios.”
The bill’s status is currently in the California Assembly. If signed into law, the bill would go into effect on Jan. 1.