AI Frameworks,

Guidelines,

Toolkits

Frameworks, Guidelines, Toolkits

ACLU, Algorithmic Equity Toolkit

AI4People (Atomium - European Institute for Science, Media and Democracy), Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations

AI4People (Atomium - European Institute for Science, Media and Democracy), On Good AI Governance 14 Priority Actions, a S.M.A.R.T. Model of Governance, and a Regulatory Toolbox

AI Now Institute, Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability

AI Now Institute, Algorithmic Accountability Policy Toolkit

AISP, A Toolkit for Centering Racial Equity Throughout Data Integration

Alan F. Winfield ; Katina Michael ; Jeremy Pitt ; Vanessa Evers: Machine Ethics: The Design and Governance of Ethical AI and Autonomous Systems, Published in: Proceedings of the IEEE ( Volume: 107 , Issue: 3)

The Alan Turing Institute and the Information Commissioner’s Office (ICO), Project ExplAIn

AlgoritmWatch, Ethics and algorithmic processes for decision making and decision support

Ambacher, B. u. a. , Trustworthy Repositories Audit & Certification: Criteria and Checklist (TRAC), CRL Center for Research Libraries, Chicago, (2007)

Arogyaswamy, B. Big tech and societal sustainability: an ethical framework. AI & Soc (2020).

B. d’Alessandro, C. O’Neil, T. LaGatta, Conscientious classification: A data scientist’s guide todiscrimination-aware classification, Big data 5 (2) (2017)

The Barcelona City Council Open Digitisation Plan

Barocas, Solon and Hood, Sophie and Ziewitz, Malte, Governing Algorithms: A Provocation Piece, (2013)

BASIC - A Toolkit and Ethical guidelines for Applying Behavioural Insights in Public Policy, OECD, (2018)

Ben Shneiderman (2020) Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy, International Journal of Human–Computer Interaction, 36:6

Brown University, A Framework for Making Ethical Decisions

Brundage, Miles et al. “Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims.” ArXiv abs/2004.07213 (2020)

Canada Algorithmic Impact Assessment

​Center for Democracy & technology (CDT), DD Tool

Christian Sandvig, Kevin Hamilton, Karrie Karahalios and Cedric Langbort. When the Algorithm Itself Is a Racist: Diagnosing Ethical Harm in the Basic Components of Software, Int’l. J. Comm. 10: (2016)

Christian Sandvig, Kevin Hamilton, Karrie Karahalios and Cedric Langbort. “Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms.” In Data and Discrimination: Converting Critical Concerns into Productive: A preconference at the 64th Annual Meeting of the International Communication Association. Seattle, WA, (2014)

CIGREF, Digital Ethics

Dafoe, Allan. "AI Governance: A Research Agenda", 2018

DataEthics. White Paper on Data Ethics in Public Procurement of AI-based Services and Solutions, 2020

Data for Children Collaborative with UNICEF, Ethical Assessment

David Freeman Engstrom, Stanford University Daniel E. Ho, Stanford University, Catherine M. Sharkey, New York University, Mariano-Florentino Cuéllar. "Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies"

Dawson D and Schleiger E*, Horton J, McLaughlin J, Robinson C∞, Quezada G, Scowcroft J,and Hajkowicz S†(2019) Artificial Intelligence: Australia’s Ethics Framework. Data61 CSIRO, Australia.

Dent, Kyle and Dumond, Richelle and Kuniavsky, Mike, A Framework for Systematically Applying Humanistic Ethics when Using AI as a Design Material (July 1, 2019). Temes de Disseny 35

Diakopoulos, N., “Algorithmic Accountability Reporting: On the Investigation of Black Boxes”, (2014)

doteveryone, Consequence Scanning Kit 

Driven Data, Deon ethics checklist

Humanitarian Data Science and Ethics Group (DSEG), A Framework for the Ethical Use of Advanced Data Science Methods in the Humanitarian Sector

Ethics Canvas

 

Expert Group Data Ethics,  Ethical Codex for Data- Based Value Creation, (2019)

European Commission High Level Expert Group on AI, Ethics Guidelines for Trustworthy AI, 2019

European Commission High-Level Expert Group on AI, Trustworthy AI Assessment List

European Commission High-Level Expert Group on AI, Towards a European strategy on business-to-governmentdata sharing for the public interest

European Commission, White Paper on Artificial Intelligence -A European approach to excellence and trust

European Commission & ADAPT, EnTIRE (Mapping Normative Frameworks for EThics and Integrity of REsearch)

European Council Committee of Ministers, Recommendation CM/Rec(2020)1 of the Committee of Ministers to member States
on the human rights impacts of algorithmic systems
, 2020

EU Privacy Seals Project

Facets, Analyzing machine learning datasets by visualization

FactSheets: Increasing Trust in AI Services through Supplier’s Declarations of Conformity

F. Kamiran, T. Calders, Data preprocessing techniques for classification without discrimination, Knowledge and Information Systems 33 (1) (2012)

Floridi, L. (2020). Ethical Foresight Analysis: What it is and Why it is Needed. Minds and Machines

Floridi, L. (2020). How to Design AI for Social Good: Seven Essential Factors

GitHub, CodeSearchNet

Google, Playing with AI Fairness: What-if Tool

Google, Dataset Search Beta (aiEthicist.org does not suggest that the datasets are free of bias, but only provides link to this Google tool)

Google, Explainable AI

Google, Perspectives on Issues in AI Governance

Google, ML-fairness-gym: A Tool for Exploring Long-Term Impacts of Machine Learning Systems

GovEx, the City and County of San Francisco, Harvard DataSmart, and Data Community DC, Ethics and Algorithms Toolkit

GovLab, Re-imagining Governance in PracticeBenchmarking British Columbia’s Citizen Engagement Efforts, 2013

GovLab, The Open Policy Making Playbook

GovLab, Mapping and Comparing Responsible Data Approaches, 2016

Judy Goldsmith, Emanuelle Burton, Why Teaching Ethics to AI Practitioners Is Important, AAAI Conference on Artificial Intelligence
Thirty-First AAAI Conference on Artificial Intelligence, 2017

Hallensleben, Sebastian and Hustedt, Carla. From Principles to Practice: An interdisciplinary framework to operationalise AI ethics, Bertelsmann Stiftung 2020

Harvard University - Berkman Klein Center for Internet and Society, Ethics and Governance of AI

IAPP, Building Ethics into Privacy Frameworks for Big Data and AI

IBM, AI Fairness 360 Open Source Toolkit

IBM, IBM Watson OpenScale

IBM, Everyday Ethics for Artificial Intelligence

IBM, Advancing AI ethics beyond complianceFrom principles to practice

IBM, Adversarial Robustness Toolbox

ICO Guidance on the AI auditing framework

IDEO, AI Ethics Cards

IEEE, Ethically Aligned Design, Version 1 (EADv1)

IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human
Well-being with Autonomous and Intelligent Systems
, Version 2 - Request for Input. IEEE, 2017.

IFTF & Omidyar Network, Ethical OS Framework

Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell,Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. In Conference on Fairness, Accountability, and Transparency (FAT* ’20), January 27–30, 2020, Barcelona,Spain .ACM, New York

 

The Institute for Ethical AI & ML,  AI-RFX Procurement Framework

Integrate.ai, Trusted Signals Engine

interpretML

ISO/IEC JTC 1/SC Standardization on Artificial Intelligence

The Japanese Society for AI (JSAI), Ethical Guidelines

Jessica Morley, Luciano Floridi, Libby Kinsey, Anat Elhalal. From What to How: An Overview of AI Ethics Tools, Methods and Research to  Translate Principles into Practices

Jobin, A., Ienca, M. & Vayena, E. ,The Global Landscape of AI Ethics Guidelines, Nat Mach Intell 1, (2019)​

Kaminski, Margot E. and Malgieri, Gianclaudio, Algorithmic Impact Assessments under the GDPR: Producing Multi-layered Explanations. U of Colorado Law Legal Studies Research Paper No. 19-28. (2019)

Kat Zhou, Design Ethically Toolkit

​Kathy Baxter (Salesforce Research), How to Build Ethics into AI — Part I Research-based recommendations to keep humanity in AI

Kathy Baxter (Salesforce Research): Building Ethics into AI: Lessons Learned from Pioneers in the Trenches

Kiritchenko, Svetlana and Mohammad, Saif M., Equity Evaluation Corpus (EEC)

Leidner, Jochen L. and Vassilis Plachouras. Ethical by Design: Ethics Best Practices for Natural Language Processing

Leighton Andrews, Bilel Benbouzid, Jeremy Brice, Lee A. Bygrave, David Demortain, Alex Griffiths, Martin Lodge, Andrea Mennicken, Karen Yeung. Algorithmic Regulation. The London School of Economics and Political Science, (2017)

Lepri, Bruno et al. “Fair, Transparent, and Accountable Algorithmic Decision-Making Processes.” Philosophy & Technology 31, 4(December 2018): 611–627 © 2017 Springer Science+BusinessMedia

Linux Foundation, Apache NiFi ‹› AI Fairness 360 (AIF360) Integration – Trusted AI Architecture Development Report 1

 

Lütge, Christoph. AI Ethics and Governance “Building a Connected, Intelligent and Ethical World”, 2020

Madaio, Michael A. and Jennifer Wortman Vaughan. “Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI.” (2020).

Markkula Center for Applied Ethics, A Framework for Ethical Decision Making

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). Association for Computing Machinery, New York

The Ministry of Economy, Trade and Industry (METI) of Japaan: Contract Guidelines on Utilization of AI and Data

Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity

NetHope Solutions Center, Artificial Intelligence (AI) Suitability Toolkit for Nonprofits

NIST, A Plan for Federal Engagement in Developing Technical Standards and Related Tools

PARC, AI Ethics Review

Platform for the Information Society, Artificial Intelligence Impact Assessment

Office of the Privacy Commissioner for Personal Data, Ethical Accountability Framework for Hong Kong

Open Data Institute, The Data Ethics Canvas

Open Robo Ethics, AI Ethics Assessment Toolkit

ProPublica, Data Store

PWC, Responsible AI Toolkit

R. Benjamins, A. Barbado, D. Sierra, Responsible AI by design

Responsible Research and Innovation, Self-Reflection Tool


Responsible Innovation Compass, Self-Check

​The Institute and Faculty of Actuaries (IFoA) and the Royal Statistical Society (RSS), A Guide for Ethical Data Science

Ruf, B., Boutharouite, C., & Detyniecki, M. Getting Fairness Right: Towards a Toolbox for Practitioners.

Sabou, Marta & Bontcheva, Kalina & Derczynski, Leon & Scharl, Arno. (2014). Corpus Annotation through Crowdsourcing: Towards Best Practice Guidelines. Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC'14).

​​Santa Clara University, Markkula Center, An Ethical Toolkit for Engineering/Design Practice

Sendak, M.P., Gao, M., Brajer, N. et al. Presenting machine learning model information to clinical end users with model facts labels. npj Digit. Med. 3, 41 (2020).

Singapore, Infocomm Media Development Authority & Personal Data Protection Commission: Model Artificial Intelligence Governance Framework, 2nd Edition

Singapore, Infocomm Media Development Authority & Personal Data Protection Commission: Implementation and Self-Assessment Guide for Organizations

Smart Dubai, AI System Ethics Self-Assessment Tool

Stark, Jeannette and Nicholas Diakopoulos. “Algorithm Tips : A Resource for Algorithmic Accountability in Government.” (2017)

TensorFlow, Fairness Indicators

TensorFlow, Model Analyzer

Thilo Hagendorff, The Ethics of AI Ethics -- An Evaluation of Guidelines

Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford, "Datasheets for Datasets," (2018)

Timnit Gebru, Emily Denton, FATE/CV Tutorial (Fairness, Accountability, Transparency, Ethics / Computer Vision)

UC Berkeley Center for Long-Term Cybersecurity (CLTC), Decision Points AI Governance

UK Government: UK Data Ethics Framework

UK Government: Artificial Intelligence and Public Standards Report

UNDP, UN Global Pulse, ‘A Guide to Data Innovation for Development: From Idea to Proof of Concept,’ 2016

University of Chicago Center for Data Science and Public Policy, Aequitas Bias & Fairness Audit

University of Washington, Lime

Utrecht University, Data Ethics Decision Aid (DEDA)

Value Sensitive Design and Information Systems

Vidgen, R., Hindle, G., and Randolph, I., (2020). Exploring the ethical implications of business analytics with a business ethics canvas. European Journal of Operational Research, 281(3)

Vollmer, Sebastian & Mateen, Bilal & Bohner, Gergo & Király, Franz & Ghani, Rayid & Jonsson, Pall & Cumbers, Sarah & Jonas, Adrian & McAllister, Katherine & Myles, Puja & Granger, David & Birse, Mark & Branson, Richard & Moons, Karel & Collins, Gary & Ioannidis, John & Holmes, Chris & Hemingway, Harry. (2018). Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness.

World Economic Forum (WEF) with Centre for the Fourth Industrial Revolution Fellows from Accenture, BBVA, IBM, Suntory Holdings, Australian Institute of Company Directors, Best Practice AI, Latham & Watkins, and Splunk, with contributions from AI4All, AI Board Toolkit

World Economic Forum (WEF) Unlocking Public Sector AI Toolkit: AI Procurement in a Box

10 Simple Rules for Responsible Big Data Research

510, F.A.C.T Score for Responsible AI

 

© 2020 by aiEthicist.org

  • LinkedIn