Documents obtained by Motherboard show that a key vendor of school-based facial recognition tools lied to school officials about the accuracy rate and racial bias of its surveillance product. The records reveal that SN Technologies' AEGIS system misidentifies black students at alarmingly high rates and mistakes objects like broom handles for guns. Despite these errors, at least one New York school district has the system configured to automatically alert police when it detects a weapon or an individual on the district’s watchlist. The use of face surveillance systems in schools increases unnecessary interactions between police and students and can accelerate the school-to-prison pipeline. SN Technologies' algorithm was included in the 2019 NIST study that showed extensive racial bias in face surveillance systems. EPIC advocates for a moratorium on facial recognition technologies and urges policymakers to increase algorithmic accountability and transparency around the adoption and use of these tools.
During oral argument this week in Van Buren v. United States, a case concerning the scope of the Computer Fraud & Abuse Act, several Justices of the U.S. Supreme Court emphasized the need to protect sensitive personal data from both hackers and insiders who could abuse their access privileges. Van Buren, a police officer, was prosecuted under the CFAA for improperly accessing personal data in a government system for financial gain. He argued that he didn't violate the law because he had credentials to access the system. EPIC filed an amicus brief in the case, arguing that the CFAA was enacted "to protect personal information stored in recordkeeping systems" and the scope of the law "should be co-extensive with its data protection purpose." At oral argument, many of the justices questioned Van Buren's attorney about the impact of his interpretation on the privacy of sensitive personal information, and a majority seemed to agree that the conduct at issue in this case should be criminalized. Justice Alito said that insiders who abuse their access can do "enormous damage" to personal privacy and referenced EPIC's amicus brief. In the brief, EPIC explained that government databases "hold vast quantities of some of the most sensitive personal data imaginable" and that "we need the CFAA, now more than ever, to be an extra check against abuse by the people entrusted to access sensitive data and systems." EPIC also argued that the Court need not limit CFAA liability to those who bypass a login system to avoid criminalizing the activity of ordinary internet users. During argument, several justices were interested in alternative ways to limit the statute to better align the law with its data protection purpose. EPIC has also participated as amicus in another CFAA case before the Court, LinkedIn v. hiQ Labs. The petition for review in LinkedIn is currently pending.
A recent patent application reveals Microsoft is developing a “meeting insight computing system” that would monitor body language, facial expressions, and other features of participants in order to assign a “quality score” to workplace meetings. According to the filing, the system could be applied both to in-person and remote meetings. Microsoft also introduced a "Productivity Score" last month which would have allowed organizations to monitor employees' use of Microsoft products. The company quickly backtracked in response to public outcry and eliminated the individualized tracking feature. Worker surveillance has rapidly increased with the transition to remote work due to COVID-19, and many organizations with on-site workers are instituting surveillance systems with the stated goal of protecting public health. EPIC advocates against social scoring and has filed a complaint with the FTC about HireVue, which similarly evaluates facial expressions and vocal patterns in the context of hiring.
