Results of the summarizing process for the arXiv paper: 2403.13004v1

In this paper, we focus on the challenges faced by public defenders in scrutinizing AI systems in the U.S. criminal legal system. Our study involved interviews with 17 individuals from the public defense community, including public defenders, technologists, and legal experts. Through their insights and experiences, we explore the reliability of computational forensic software (CFS) tools such as facial recognition, gunshot detection, and probabilistic genotyping. Our findings reveal that public defenders encounter difficulties in navigating how CFS is developed and used, overcoming non-critical perceptions of CFS by judges and jurors, and accessing expertise on CFS technology. To address these challenges, we provide recommendations that consider the technical, social, and institutional context to support contestability in practice. Additionally, our study references related research on trust in AI models based on stated and observed accuracy. We also examine communication of uncertainty information to decision-makers and jurors' perceptions of error rates associated with forensic evidence. These studies highlight the importance of understanding downstream stakeholders' needs for performance information and how they interpret it. Our methodology section details participant recruitment strategies using convenience and purposive sampling methods to ensure diversity among participants from different regions in the U.S. Interviews were conducted over video calls or phone calls between December 2022 and August 2023. Recordings were transcribed and anonymized for analysis. We acknowledge the power imbalances within the criminal legal system that disproportionately impact low-income and marginalized communities. While our focus is on public defenders' perspectives, we recognize the need for engagement with criminal defendants and broader communities served by public defenders to provide a comprehensive understanding of the impacts of AI systems. Overall, our study aims to shed light on the challenges faced by public defenders in scrutinizing AI systems in high-stakes settings. We propose recommendations to enhance contestability through a nuanced understanding of technical, social, and institutional factors.

• - Challenges faced by public defenders in scrutinizing AI systems in the U.S. criminal legal system
• - Difficulties encountered by public defenders in navigating the development and use of computational forensic software (CFS) tools
• - Recommendations provided to address challenges, considering technical, social, and institutional contexts
• - Importance of understanding downstream stakeholders' needs for performance information and interpretation of it
• - Methodology involving participant recruitment strategies using convenience and purposive sampling methods
• - Acknowledgment of power imbalances within the criminal legal system impacting low-income and marginalized communities
• - Recognition of the need for engagement with criminal defendants and broader communities served by public defenders to understand impacts of AI systems

SummaryPublic defenders in the U.S. have a hard time checking AI systems used in the legal system. They struggle with using computer tools for forensic work. To help, experts suggest ways to deal with problems in different areas like technology and society. It's important to know what people need from data and how to explain it well. Researchers find participants for studies by choosing them on purpose or based on convenience. Definitions- Public defenders: Lawyers who help people who can't afford their own lawyer. - Scrutinizing: Carefully examining or looking closely at something. - Computational forensic software (CFS) tools: Computer programs used for investigating crimes. - Recommendations: Suggestions or advice on what to do. - Stakeholders: People or groups affected by a decision or situation. - Methodology: A way of doing things, like a plan or strategy. - Participant recruitment strategies: Methods for finding and selecting people to take part in a study. - Power imbalances: Unequal distribution of control or influence among different groups. - Marginalized communities: Groups of people who are treated unfairly or left out of important decisions. - Engagement: Involvement or connection with others in meaningful ways.

Title: Challenges Faced by Public Defenders in Scrutinizing AI Systems in the U.S. Criminal Legal System Introduction: Artificial intelligence (AI) has become increasingly prevalent in various industries, including the criminal legal system. With the use of computational forensic software (CFS) tools such as facial recognition, gunshot detection, and probabilistic genotyping, there are concerns about their reliability and potential biases. In this research paper, we focus on the challenges faced by public defenders in scrutinizing AI systems within the U.S. criminal legal system. Methodology: Our study involved interviews with 17 individuals from the public defense community, including public defenders, technologists, and legal experts. To ensure diversity among participants from different regions in the U.S., we used convenience and purposive sampling methods for participant recruitment. Interviews were conducted over video or phone calls between December 2022 and August 2023. Recordings were transcribed and anonymized for analysis. Challenges Faced by Public Defenders: Through our interviews with public defenders, we identified three main challenges they face when it comes to scrutinizing AI systems: 1) Navigating CFS Development and Use: Public defenders encounter difficulties in understanding how CFS is developed and used due to its complex technical nature. This lack of understanding can hinder their ability to effectively challenge its use as evidence. 2) Overcoming Non-Critical Perceptions of CFS: Judges and jurors may have a non-critical perception of CFS due to its perceived objectivity and accuracy. This can make it challenging for public defenders to raise doubts about its reliability or potential biases. 3) Accessing Expertise on CFS Technology: Public defenders often do not have access to experts who can provide insights into how CFS works or identify potential flaws in its algorithms. This lack of expertise can limit their ability to effectively challenge its use as evidence. Recommendations for Enhancing Contestability: To address these challenges, we provide recommendations that consider the technical, social, and institutional context. These include: 1) Technical Recommendations: a. Transparency: CFS developers should be transparent about their algorithms and data sources to allow for better understanding and scrutiny. b. Independent Audits: Regular independent audits of CFS tools should be conducted to identify potential biases or errors. c. Explainability: CFS tools should be designed with explainability in mind to make it easier for public defenders to understand how they work. 2) Social Recommendations: a. Education: Public defenders should receive training on AI technology and its potential biases to enhance their ability to scrutinize it effectively. b. Collaboration: Collaboration between public defenders and technologists can help bridge the gap in understanding of AI systems. c. Community Engagement: Engaging with criminal defendants and broader communities served by public defenders can provide a comprehensive understanding of the impacts of AI systems. 3) Institutional Recommendations: a. Standards for Admissibility: Standards for admissibility of evidence from AI systems should be established to ensure its reliability is thoroughly examined before being presented in court. b. Resources for Public Defenders: Adequate resources should be allocated towards providing public defenders with access to experts who can assist them in scrutinizing AI evidence. Related Research on Trust in AI Models: Our study also references related research on trust in AI models based on stated and observed accuracy, communication of uncertainty information, and jurors' perceptions of error rates associated with forensic evidence. These studies highlight the importance of understanding downstream stakeholders' needs for performance information and how they interpret it. Conclusion: In conclusion, our study sheds light on the challenges faced by public defenders in scrutinizing AI systems within high-stakes settings such as the U.S. criminal legal system. By providing recommendations that consider technical, social, and institutional factors, we aim to enhance contestability when it comes to using AI evidence in court. However, we also acknowledge the power imbalances within the criminal legal system that disproportionately impact low-income and marginalized communities. Therefore, engagement with criminal defendants and broader communities served by public defenders is crucial to provide a comprehensive understanding of the impacts of AI systems.