Clik here to view.
(AP Photo/Mark Lennihan)
As more cities embrace big data, algorithms play a growing role in the day-to-day activities of civic life, determining things like which school a child will attend or whether a person is likely to be allowed out of jail with bail. But all data comes from human sources, and humans are, of course, biased. A new bill passed Monday by New York City Council aims to correct some of those innate biases and make sure the city’s computerized systems are, in fact, fair.
The bill would establish a task force to monitor the algorithms used by municipal agencies, Tech Crunch reports. It currently awaits the mayor’s signature, and is being supported by the New York ACLU.
“Flawed code can … further entrench systemic inequalities,” the organization wrote in a recent release. “The algorithms used in facial recognition technology, for example, have been shown to be less accurate on black people, women and juveniles, putting innocent people at risk of being labeled crime suspects.”
A 2016 ProPublica study found that software used across the country to make bail and sentencing decisions were biased against black people. The tools are designed to determine the likelihood of future criminal activity, but the predictions were correct only 20 percent of the time.
The task force will write a report that explores whether given systems disproportionately impact certain groups, like the elderly, immigrants or the disabled. It will also examine how people should be informed as to whether they’re being assessed algorithmically and how those systems should be documented and archived, among other things. It would need to be formed within three months of the bill’s signing, and according to the bill include “persons with expertise in the areas of fairness, accountability and transparency relating to automated decision systems and persons affiliated with charitable corporations that represent persons in the city affected by agency automated decision systems.”
As Tech Crunch’s Devin Coldewey puts it, “this wouldn’t just be a bunch of machine learning experts and a couple of lawyers.”
The bill sets a welcome precedent for other cities, where leaders often struggle with how to regulate their data systems. It could also dovetail with efforts like Data for Black Lives, which seeks to build bridges between chief data officers and the black community regarding issues like displacement and economic opportunity.
