Who’s to Blame When Algorithms Discriminate?

85

If disparate impact becomes a less viable legal tool, civil rights groups counter that it will be almost impossible to curb policies and decisions that reinforce segregation and widen the racial wealth gap. If plaintiffs must prove that someone, somewhere, explicitly intended to discriminate, they’ll never be able to police city officials who keep that intent silent — or algorithms that have no “intent” at all.

“People don’t just say the things they used to say,” said Myron Orfield, a law professor at the University of Minnesota who directs the Institute on Metropolitan Opportunity there.

But some statistical patterns speak just as loudly.

“A black household that makes $167,000 is less likely to qualify for a prime loan than a white household that makes $40,000,” Mr. Orfield said, citing analysis of public mortgage data by the institute. “That looks funny. What the banks say in these cases is, ‘It’s the credit histories, and our models explain the differences.’ But you can’t look at those models. They’re proprietary.”

The proliferation of data has made it easier to identify such patterns. But HUD suggests that in all this data, the threat of legal challenges under the Fair Housing Act has grown, too.

The new rule “frees up parties to innovate, and to take risks to meet the needs of their customers, without the fear that their efforts will be second-guessed through statistics years down the line,” Paul Compton, HUD’s general counsel, said in a phone call with reporters.

The new rule, which enters a 60-day public comment period, rewrites a disparate impact standard published by the Obama administration in 2013 that put more onus on defendants to explain any practices that appear discriminatory. Under the new rule, plaintiffs will hold the burden of showing that any discrimination could have been averted by a different policy.

Where algorithms or models are accused of bias, the new rule also lays out arguments companies can use to defend them, including showing that a model is the standard in an industry or was created and maintained by a third party. Those defenses may be insurmountable, civil rights groups say.