Towards A Global AI Auditing Framework: Assessment and Recommendations

The report analyzes the current AI audit ecosystem, identifying critical gaps in auditor independence, data access, and the failure to account for global sociotechnical impacts. It uses New York City's Local Law 144 as a case study to illustrate how regulatory-driven audits often prioritize transparency over actual harm mitigation. To address these issues, the authors propose a comprehensive Global AI Auditing Framework that emphasizes multidisciplinary teams, participatory stakeholder engagement, and the inclusion of labor and environmental criteria to ensure accountability across the AI supply chain.

• The current AI audit ecosystem is fragmented and lacks standardized quality controls, making it difficult to determine which audits effectively capture system errors.
• Existing AI audits often focus on documenting risks and harms rather than assessing whether the developer has actually mitigated those harms.
• Auditor independence is frequently compromised by financial dependencies, as third-party firms are often paid by the entities they are auditing, creating a potential conflict of interest.
• A case study of New York City's Local Law 144 reveals that while it provides clear scope and criteria for bias audits, it primarily functions as a transparency regime that does not necessarily ensure the remedy of identified biases.
• The proposed Global AI Auditing Framework recommends that audit scope be determined by the AI system layer and stage of the life cycle, including the labor and material supply chains.
• Effective auditing requires the inclusion of criteria for environmental sustainability, such as minimizing greenhouse gas emissions and energy and water consumption.
• The framework emphasizes the need for 'Indigenous data sovereignty' and the engagement of communities from whom data is collected to ensure data rights are respected.
• To ensure rigor, the framework suggests that auditors should not rely solely on developer-provided tests but should conduct their own performance and scenario tests.
• The report argues that audit reports should be made publicly available through searchable repositories to foster trust and enable external accountability.

APA

Faveri, B., Johnson-León, M., Sylvester, P., Chun, W. H. K., Hannák, A., Mendoza, M., Broussard, M., Enikolopov, R., Nelson, A., Sandvig, C., Sesan, ‘., Sporle, A., Srihari, R., Srinivasan, J., Wilson, C., & Zou, M. (2024). Towards A Global AI Auditing Framework: Assessment and Recommendations. International Panel on the Information Environment. https://www.ipie.info/research/sr2024-3

Chicago

Faveri, Benjamin, Maureen Johnson-León, Prem Sylvester, Wendy Hui Kyong Chun, Anikó Hannák, Marcelo Mendoza, Meredith Broussard, et al. Towards A Global AI Auditing Framework: Assessment and Recommendations. International Panel on the Information Environment, 2024. https://www.ipie.info/research/sr2024-3.

Wikipedia

{{cite report |last1=Faveri |first1=Benjamin |last2=Johnson-León |first2=Maureen |last3=Sylvester |first3=Prem |last4=Chun |first4=Wendy Hui Kyong |last5=Hannák |first5=Anikó |last6=Mendoza |first6=Marcelo |last7=Broussard |first7=Meredith |last8=Enikolopov |first8=Ruben |last9=Nelson |first9=Alondra |display-authors=etal |title=Towards A Global AI Auditing Framework: Assessment and Recommendations |publisher=International Panel on the Information Environment |date=2024 |url=https://www.ipie.info/research/sr2024-3 |access-date=23 October 2026 |via=Climate Insights Directory}}

BibTeX

@techreport{faveri2024towards, author = {Faveri, Benjamin and Johnson-León, Maureen and Sylvester, Prem and Chun, Wendy Hui Kyong and Hannák, Anikó and Mendoza, Marcelo and Broussard, Meredith and Enikolopov, Ruben and Nelson, Alondra and Sandvig, Christian and Sesan, ‘Gbenga and Sporle, Andrew and Srihari, Rohini and Srinivasan, Janaki and Wilson, Christo and Zou, Mimi}, title = {{Towards A Global AI Auditing Framework: Assessment and Recommendations}}, institution = {International Panel on the Information Environment}, year = {2024}, url = {https://www.ipie.info/research/sr2024-3}, urldate = {2026-10-23}, note = {Indexed by Climate Insights Directory} }