Automated decision tools, or ADTs, are algorithms that are derived from personal data and are used to make important decisions in peoples' lives.

These complex systems are traditionally used for things like credit decisions, health care risk predictions, insurance eligibility and employment screening.

But as these algorithms are being used more and more to assess eligibility for benefits or penalties, local Assemblymember Rebecca Bauer-Kahan (D-Orinda) is now calling for more regulations on ADTs due to the harm it is causing marginalized communities.

"As the use of decision-making via algorithm becomes more prevalent in our daily lives, it is crucial that we take steps to ensure that it is used ethically and responsibly," Bauer-Kahan stated in a press release on Monday. "Without quick, thoughtful regulation, we face a future where decision making is heavily biased without any protections from the devastating impacts."

One example of why ADTs are harmful to marginalized communities that Bauer-Kahan highlighted was when it referenced a 2019 study published in the American Association for the Advancement of Science magazine called: Dissecting racial bias in an algorithm used to manage the health of populations.

According to the study, health care risk-prediction algorithms used across hospitals in the U.S. demonstrate racial bias against Black patients because it relies on faulty metrics to determine need.

"Black patients assigned the same level of risk by the algorithm are sicker than White patients. The authors estimated that this racial bias reduces the number of Black patients identified for extra care by more than half," the 2019 study states. "Bias occurs because the algorithm uses health costs as a proxy for health needs. Less money is spent on Black patients who have the same level of need and the algorithm thus falsely concludes that Black patients are healthier than equally sick White patients."

That is why Bauer-Kahan says she is introducing Assembly Bill 331.

AB 331, according to the press release, will require ADT developers and users to "conduct and record an impact assessment including the intended use, the makeup of the data and the rigor of the statistical analysis."

"The data reported must also include an analysis of potential adverse impact on the basis of race, color, ethnicity, sex, religion, age, national origin, or any other classification protected by state law," the press release states.

The first-of-its-kind bill will look to crack down on these issues to ensure that decisions, which significantly impact people's lives, are made with the consent and knowledge of those people.

"ADT is rapidly evolving and it is imperative that we take action now to ensure that it is used in a way that benefits everyone," Bauer-Kahan said. n