CA could require tech companies to disclose AI training data

Democratic Rep. Jackie Irwin, a former tech industry insider, is taking on the industry with a sweeping bill that would require artificial intelligence developers to disclose what data they use to “train” their systems.

“Consumer trust in AI systems has not grown as rapidly as industry adoption,” Irwin said at a hearing last month. “Many consumers have legitimate questions about how these AI systems and services are created.”

It's notable that Rep. Irwin has voiced concerns about AI because she has been a leading voice and occasional advocate for the tech industry in Congress.

As a young engineer at the Johns Hopkins University Applied Physics Laboratory, she was tasked with troubleshooting the launch of the U.S. Navy's Trident II nuclear missiles, making her the only working rocket scientist in the state legislature.

She is a former engineer at Teledyne Technologies, a global aerospace and technology conglomerate headquartered in Irwin's hometown of Thousand Oaks.

She co-chairs the California Assembly Technology and Innovation Caucus and the National Legislative Task Force on AI, Cybersecurity and Privacy.

She's also a favorite of big tech companies, having once authored a bill that was criticized for weakening digital privacy protections in California to serve the tech industry, where her family has deep ties. At the time, her husband was chief operating officer of Amazon-owned Ring.

The prominent lobbying group TechNet named her “Legislator of the Year” in 2017.

But now TechNet and nearly every lobbying group representing big tech companies oppose her latest bill, Assembly Bill 2013. The influential California Chamber of Commerce also opposes the bill, and last week the state Assembly voted 56-8 to send the measure to the Senate.

Last month, Chamber of Commerce Commissioner Ronak Daylami told the Assembly's Privacy and Consumer Protection Committee, on which Irwin sits, that Irwin's bill could expose closely guarded trade secrets held by tech companies.

“While it may not be obvious on the surface, the expertise and judgment and actual selection of data and datasets chosen to train a particular AI model is unique in and of itself,” Daylami said.

Can disclosure help avoid AI bias?

But Irwin said the move would give consumers a powerful tool to better understand emerging technology that has raised privacy concerns after revelations that tech companies were using copyrighted material such as facial recognition, social media posts, artwork and news articles to train artificial intelligence software.

Irwin said mandatory disclosure of training data could also help prevent potential bias in AI software's decision-making.

She said she became intrigued by the issue after hearing a physician group discuss using AI for drug dispensing at a recent National Conference of State Legislatures. The problem, she said, is that companies aren't required to disclose the data they used to train their systems, so it's not clear whether such systems have inherent bias.

What if, she wondered, it was like a clinical drug trial, testing a drug only on suburban, white men, rather than a diverse patient population whose bodies might respond differently?

“With these AI medical devices, you really need to know what groups they were trained on,” she said.

More broadly, Haley Tsukayama, a legislative advocate at the Electronic Frontier Foundation, likened the disclosure requirements to being able to read the list of ingredients in a meal.

“Sometimes it's a lot easier to parse an ingredients list than it is to taste the dish at the end and figure out what's in it,” she said.

Irwin owns Amazon and tech stocks.

The AI ​​disclosure bill is not Irwin's first foray into technology regulation since joining the House in 2014.

Irwin's office released a “non-exhaustive” list of 13 tech- and cybersecurity-related bills he has authored, most of which have passed. Several of those bills have also been opposed by the tech industry, which has donated at least $288,000 to his campaigns over the years, according to the Digital Democracy Database.

According to an analysis by Digital Democracy, since 2015, Irwin's votes have aligned with TechNet's stance on the bill 28 percent of the time.

But her most controversial tech bill was a 2019 bill that critics said would weaken the state's landmark California Consumer Privacy Act, which gives Californians legal power to order tech companies to tell them what personal information they've collected about them, and allows customers to ask the companies to delete that information and not sell it.

At the time, Irwin's husband, John, was chief operating officer of Amazon-owned Ring, raising questions about a conflict of interest given the company was subject to privacy laws.

Irwin denied that this was the case: She told Politico at the time that it was disrespectful to infer that she worked for her husband's company, given her professional background and expertise.

Jon Irwin has since left Amazon to become COO of Centegix, a technology company that makes wearable emergency alerts and security systems for schools and other facilities, according to his LinkedIn page.

Last year, he told state ethics officials that the family had sold between $300,000 and $3 million worth of Amazon stock, which state ethics officials allow lawmakers to report the value of a wide range of stock portfolios when filing annual financial disclosure forms.

Irwin made at least $60,000 last year from investments in cryptocurrency, AI and semiconductors, according to his disclosure documents.

In an interview with CalMatters last week, Irwin declined to provide a more precise figure for how much she sold her Amazon shares or discuss her recent investments in technology companies. She said she was following state ethics disclosure requirements and that her and her husband's investments did not factor into her decision-making process.

“I make every decision based on what's best for my constituents,” she said. “No one questions anything I make, so we always take great care in every decision.”

Rather, she said she became interested in technology and cybersecurity law because her background made it a natural fit, and she also places a premium on educating her fellow lawmakers about cybersecurity.

“You can talk to any of my colleagues, and I've taken most of their phones and said, 'Oh my gosh, you know, your phone is tracking you. These apps are tracking you. Turn off your location device, turn on two-factor authentication,'” she said. “At caucus meetings, I stand up and tell people how to make their phones more secure.”

Tsukayama, the legislative advocate at the Electronic Frontier Foundation, asserts that Irwin knows as much, if not more, about complex technology issues than anyone else in Congress, even if digital consumer rights groups sometimes oppose her bills.

“We haven't always agreed with her,” Tsukayama says, “but it's rarely because she misunderstands how technology works.”

CalMatters Economic Reporter Levi Sumagaisay Data Reporter Jeremiah Kimmelman contributed to this story.