The Right Tool for the Job: Open-Source Auditing Tools in Machine Learning

Introduction

As a society, we are tossed around on a sea of competing ideas, backed by statistics, data, and algorithms. The decisions made by algorithms rely on the data used as input and the manner in which the algorithms are configured and programmed by humans. Ultimately, however, all decisions are human decisions, regardless of the manner or means by which the data are filtered, manipulated and wielded. Humans select the filters, manipulate the data, and wield the algorithms. That means that humans can also ultimately be held responsible for the outcome of those decisions, whether directly made by a human brain or by extension, an algorithmic matrix. It is widely known that algorithms can encompass, express, and unveil intentional or unconscious bias, including bias intrinsically present in the data, and manipulation by competing interests or those with agendas. The critical need for fairness, accountability, transparency, and ethics (FATE) in machine learning has increased dramatically in recent years as the output of systems has shown the disparities in data and algorithms in every field of endeavor. This increase in awareness has also triggered legal concerns that have evolved into rubrics for ethical use of algorithmic tools, but real-world legal concerns are being raised and governmental regulators around the world are being educated on remedies that may be needed if the industry does not better police itself.

Audits are evaluations with expectations of accountability. Algorithmic auditing is not a one-size-fits all solution. Different methods of auditing are required in order to search for and test both the data and the algorithms for underlying or hidden bias and attributes. Methods encompassing anomaly detection and pattern recognition are required. No single auditing method will likely uncover all potential issues. The breadth and scope of auditing methods necessitates creating an algorithmic auditing toolkit that one may use to deeply examine data and algorithms with various degrees of granularity.

Why Auditing is Important

Fairness is an important concern in terms of trust and reliability of the decision-making process involving machine learning models. Each model has specific requirements for the type of data it will operate over. These necessitate different pre-processing techniques. However, in order to provide a consistent comparison across models, each model must receive similar input. Data are manipulated by pre-processing it to generate similar types of inputs that models can handle. However, this data manipulation can also create bias in datasets as it is a deliberate method of focusing the analytical lens on a particular set of data. Data may also be biased because it is a collection of human-generated information. There is an evidence-based assumption that bias is bad. There is a societal need to be accountable, transparent, and fair, especially because the outputs of algorithms are not always able to be understood or interpreted by humans. Algorithmic output and decisions are also susceptible to inaccuracies, may be adversarial to human rights or other fundamental rights, and may present legal challenges such as data privacy and unauthorized use or sale of data by third-parties. Auditing algorithms may not resolve all issues of concern, but audits are part of the solution in providing more ethical accountability for very powerful machine learning software.

Model Standards

The implementation of reproducible and standardized techniques has led industry to establish guidelines for fairness, accountability, and transparency in machine learning models. However, the implementation of these guidelines requires tools. The main objective of these tools is to provide a mechanism by which bias and other forms of concerning algorithmic output can be detected, measured, and reduced.

Various auditing tools and open-source libraries have been developed to provide a high level of functionality in machine learning and deep learning models.

Different models of analysis are used to extract different features from data. These features, their patterns, and anomalies are then analyzed to determine whether there are irregularities in the underlying data or in the algorithm used that need to be explored further. The expectation of finding bias of some type in any given generic dataset should be neutral. While none is overtly expected, history shows that humans have both overt and unconscious bias and that bias may be reflected in unexpected places.

Pervasive use of high performing, ‘black-box’ models in sensitive areas like healthcareand criminal justice systems raises ethical concerns and a critical need to audit these models for fairness. Recent worksandhave revealed disturbing bias in some machine learning models against minority groups.

One way to audit these algorithms is to look inside the inner-workings of the algorithm, model, and datasets to understand how the underlying algorithms, data, data pre-processing, input features, feature engineering, tuning, and optimization are combined to arrive at the model output. However, due to proprietary reasons, the inner workings of machine learning models may not be publicly available. One possible solution to this dilemma is that post-hoc interpretability techniques can serve as useful auditing tools.

The Law

There are also growing legal reasons to be concerned about fairness in machine learning. Legal standards differ in different countries and what may be considered fair on one country’s legal system may not be fair in another. Much of the law related to computer science is related to privacy and access to data and systems. In the United States employment laws related to disparate impact protect the public from the harm posed by biased algorithms, in some cases, but only once that disparate impact is discovered and challenged. There are, however, important legal distinctions. Disparate impact, by itself, is not unlawful. Disparate treatment may be unlawful in certain situations, particularly when individuals with a legally recognized protected status are harmed. However, even in this circumstance, the burden of proof is very high, is is very fact-specific, and will be limited to the specific context in which it arises. There are anti-discrimination laws that apply to access to public services, housing, lending, and the like, but US law is a negative limitation on government, not the private sector.

In other areas of the world, the laws are very different. The laws of the European Union tend to be more stringent than laws in the United States. The General Data Protection Regulation (GDPR) protects privacy rights as well as rights related to disparate treatment and discrimination. In Asia, some laws related to machine learning systems are more lax than those of the United States. The laws in many South America and African countries are limited insofar as machine learning is concerned. Many countries have data protection laws, but their degree of protections vary.

The lack of regulation at the present time is an opportunity for the field to place a greater emphasis on self-policing.The more upset the global public becomes with algorithms that show unfairness, bias, disparate impact, disparate treatment, and differential classification that discriminates against one group over another, the more attention will be drawn to the field and the calls for governmental regulation will increase.

These laws and regulations may be sector-specific, such as requiring computer vision algorithms to comply with safety standards of other vehicles driven on public roadways. Or they may be broader in scope, such as algorithms related to credit scoring and the manner in which they affect lending interest rates as a surrogate of access to or denial of financial or real estate services.

Fairness in machine learning is generally related to fairness of decision outcomes (distributive fairness), under the law. This is why the area is often examined under the disparate impact and disparate treatment legal rubrics. However, fairness in machine learning is also related to procedural fairness, especially in settings where there may be a deprivation of a legally-guaranteed right or privilege, such as a criminal, employment, or financially-based setting.This also applies to computer vision algorithms related to biometrics, many of which simply apply Euclidean distance metrics and classifiers. The manner in which facial recognition software operates is important because a 1:1 identification has far less risk of discrimination or misidentification that a 1:N or 1:many analysis where an unknown face is compared to a large database of known faces to determine the unknown person’s identity.

In a private setting, legal concerns about fairness in machine learning are tied directly to concerns about risk-management and public perception. A corporation can be sued or can easily lose brand status because of using machine learning algorithms that are publicly considered to be unfair, even if that unfairness does not rise to the level of unlawfulness in a civil context or illegality in a criminal context.

Educating the public by publishing algorithmic auditing assessments and proof of the use of auditing tools will help mitigate some of the concerns about fairness in machine learning algorithms. Taking responsibility for the outcome of the algorithm and its use will promote public good will.

A worst-case scenario is for industry to do nothing until a legislative body decides to regulate machine learning technologies in a strict manner similar to other product liability laws. This is what is slowly occurring in the autonomous driver space. When vehicle manufacturers cannot control damage to property and lives due to faults in their algorithms, the entities supplying the software, the executives, and engineers responsible may be held legally accountable.

Exemplary Areas of Concern

At the 2021 Neural Information Processing Systems (NeurIPS) conference, the organizing committee instituted an Ethics Review process for papers submitted to the conference. According to the conference Retrospective publication, 9122 total papers were submitted and reviewed and 265 were flagged for additional ethics review in one or more areas of concern.

Areas of concern included: Discrimination/Bias/Fairness Concerns, Inadequate Data and Algorithm Evaluation, Inappropriate Potential Applications and Impact (e.g. human rights concerns), Legal Compliance (e.g. GDPR, copyright, terms of use), Privacy and Security (e.g. consent), Responsible Research Practice (e.g. IRB, documentation, research ethics), and research integrity issues (e.g. plagiarism).

The ACL, CVPR, ICLR, and EMNLP conferences have also taken steps to establish ethical expectations for the research community including introducing ethics review processes. Including fairness and ethics assessments in the underlying work and research methodologies and reporting the same in the papers, reviews, and conference presentations will boost confidence from industry, academia, government, business, the public, and end-users who have concerns about AI and machine learning fairness and safety.

In January 2022, the United States Internal Revenue Service (IRS) announced that it would soon require facial recognition technology in order to access its systemsand. The IRS contracted with a third-party company, ID.me, to provide the service. The company also provides services to the US Department of Veterans Affairs and the US Social Security Administration.However, the agency reversed its decision after large public outcries over the practice.

Even more chilling data privacy concerns were uncovered following the public outcry that showed the IRS had consented to allowing ID.me to “disclose or share Non-Personally Identifiable Information with Third-Party Service Providers and Affiliates”. This implies that the company was or intended to be a data broker and many data privacy advocates have become rightly concerned.

Initially, ID.me initially stated that it was only going to be using 1:1 facial recognition, but later clarified that the company also uses 1:N (1:many) technology, prompting privacy advocates to raise public concerns.The CEO founder of ID.me stated that the 1:many is only used during initial enrollment and that afterwards, 1:1 is used. However, a New Jersey man was indicted by the US Department of Justice in January 2022 for deliberately circumventing the security measures of ID.me in order to further a scheme to defraud the California Employment Development Department (EDD), filing at least 78 fraudulent unemployment insurance claims with EED seeking Pandemic Unemployment Assistance and other benefits under the federal CARES Act. EDD has partnered with ID.me since at least October 2021.

How much better off would ID.me be if they could show that they or a neutral third-party entity has conducted an algorithm audit of their systems and processes and be able to provide proof of the same? In situations where the algorithmic model and data are proprietary, the use of something like zero-knowledge proofs may be a way to affirmatively show the results of an algorithmic and data audit without actually opening the proprietary system for analysis to the public or in a public forum.

As issues of fairness in machine learning and AI ethics continue to expand, it becomes imperative for the community to be aware of the potential risk and legal implication and the need to be familiar with open-source and free auditing tools that can be used to insure integrity of data and algorithms used.

Techniques

In recent years several machine learning interpretability techniques have emerged that can be used to explain the machine learning model results in terms of individual input feature contributions. Current popular interpretability techniques can be categorized into post-hoc interpretability and intrinsic interpretability. Post-hoc methods provide explanations of the model output after the model is trained. Some of the current state-of-art methods include feature attributions (LIME, SHAP), model and data analysis tools (What-If Tool, Fairness Indicators), gradient and concept based testing (TCAV, GRAD-CAM) and data point inspection methods (partial dependence plots, ablation testing). Intrinsic interpretability is achieved by using inherently interpretable machine learning models.

In many instances, deep learning models have been shown provide a higher degree of accuracy than the traditional machine learning models when the input consists of large, unstructured data. Similarly, sophisticated ensemble techniques like bagging and boosting are preferred to simpler, more interpretable decision trees as they achieve better performance with structured tabular data. However, there exists a trade-off between accuracy and interpretability. The complex architecture of these models make it hard for the end-user to interpret how the input features have contributed to the model output prediction.

Post-hoc interpretability techniques serve as viable model auditing tools as they enable us to understand individual feature contributions to the model output.

In certain scenarios the explanations provided by the post-hoc methods are unreliable. Using inherently interpretable models can overcome this drawback.

Tools

Data and algorithms come in a variety of flavors. This is also true of auditing tools. It is important to know and understand which tool is appropriate to use at each phase of the machine learning continuum. The main objective with these tools is to detect, measure, and reduce unwanted bias. Some of these tools also incorporate simulations and data visualization models as graphical representations that can be used to discern differences in datasets in order to help identify patterns or anomalies. The open-source framework facilitates a direct comparison of algorithms.

A sampling of a wide variety of tools and toolkits with links to papers, models, tutorials, instructions for use, and code repositories are as follows.

AI Fairness360

The AI Fairness 360 toolkit (AIF360) is a comprehensive Python open-source package developed at IBM that contains over 70 fairness metrics and 10 bias mitigation algorithms.It is designed to translate algorithmic research from the lab into the actual practice of domains as wide-ranging as finance, human capital management, healthcare, and education.In some cases, AI360 toolkit requires datasets to be transformed into Binary-Label-Dataset. The protected attributes can the be selected for observational analysis. These attributes partition a data population into groups that have a difference in terms of parity received.

Prior to applying fairness metrics to the selected attributes, standard statistical analysis can also be applied to determine the target attribute.

Code and instructions can be found at,and.

LIME

LIME (Local Interpretable Model-Agnostic Explanations)is a popular perturbation-based interpretability technique that provides local explanations to black-box models. LIME trains a linear classifier that locally approximates the behavior of the black-box machine learning model. A perturbed dataset is created by feeding in variations of a single data point to the ML model and noting the corresponding outputs. LIME then trains an interpretable model (for example, a decision tree) on this new dataset of permuted samples, where the samples are weighted by their proximity to the sample of interest. The local interpretable model is then used to explain the predictions made by the machine learning model to be audited.

The LIME results and explanations can serve as a helpful model auditing tool when the inner workings of the machine learning model are not revealed. The user can feed in data samples permuted around a sample of interest to the black-box model and store the outputs as a new dataset to train a more interpretable model. However, LIME suffers from a drawback of unclear coverage. The explanations provided by LIME are local in nature and the boundary to which these explanations apply is not well defined.

Code and instructions can be found at.

ANCHORS

ANCHORS (High-Precision Model-Agnostic Explanations) is a model-agnostic system proposed by the authors of LIME that generates a set of ‘if-then’ rules as explanations for model output predictions and overcomes the drawbacks of LIME by providing a clear coverage of the data instances to which the explanations apply.

Code and instructions can be found at,and.

What-If Tool

The “What If Tool” (WIT) is open-source auditing tool that provides an interactive visualization platform that allows users to probe, visualize, and analyze their data against different classifiers.Visualization of hypothetical scenarios can be made against different subsets of inputs and analysis can be assessed related to the importance of different data features over different models.

WIT converts datasets to a numerical format with a graphical display. Each data point is displayed in the graph are editable and can be set to different values to observe counterfactual points in the data. What if tool can be useful in identify bias using hypothetical situations against different features using linear classifier and neural network classifier. Models of classification can be assessed against any input features. A ground truth feature can also assess binary categories.

WIT also has a visualization tool called Facets Overviewthat uses subsets of the data to compare fairness metrics and assess overall model performance.

WIT with SHAP (SHapley Additive exPlanations), as a technique to audit a black-box machine learning model used in the healthcare domain.

Code and instructions can be found atand.

FairSearch

FairSearch is the fair open-source search API to apply fairness in ranked search results. Its ranked group fairness definition extends group fairness using the standard notion of protected groups, based on ensuring that the proportion of protected candidates in every prefix of the top-k ranking remains statistically above or indistinguishable from a given minimum.

Code and instructions can be found atand.

FairTest

The FairTest tool helps to identify unwanted associations, unfair, discriminatory, or offensive user treatment in data-driven applications. It examines and biases between potential outcomes and sensitive attributes. It has been used to address disparate impact, offensive labeling, and uneven rates of algorithmic error in a variety of datasets.

Code and instructions can be found at,and.

Aequitas

Aequitas is an open-source Python library with many fairness metrics to detect bias. To use Aequitas one must upload their data to the tool. Some may not be comfortable in using their data on other sites. If there are data leakage concerns, you should use another tool. Aequitas uses a Python library and a command line interface. Users upload their data from the system being audited, configure bias metrics for protected attribute groups of interest as well as reference groups, and then the tool generates bias reports. Bias assessments can be made prior to model selection, evaluating the disparities of the various models trained based on whatever training data was used to tune it for its task. The audits can be performed prior to a model being operationalized or it can be used to determine bias in an A/B testing environment using limited trials and revising the algorithm accordingly.

Code and instructions can be found at,and.

Themis-ML

Themis-ML is a tool that makes model agnostic predictions using reject object recognition (ROC) and discrimination aware ensemble classification (DAEC). The concept is that it that looks for bias by altering the training data by reweighting and relabeling the classes using sampling. Themis-ML will modify your dataset, so be sure to save your original model and make one or more copies to run on Themis-ML.Code and instructions can be found atand.

Fairness Test-Bed

The “Fairness-Aware” Machine Learning interface, Fairness Test-Bed, is an open-source framework that facilitates a direct comparison of multiple algorithmsand. The framework for the tool focuses on factors like pre-processing if multiple algorithms are going to be compared to each other. The Fairness Test-Bed paper shows that that a combination of class-sensitive error rates and either DI or CV is a good minimal working set. To account for training instability, it also shows that the performance of an algorithm in a single training-test split appears to be insufficient. Accordingly, they recommend reporting algorithm success and stability based on a moderate number of randomized training-test splits. They use the standard accuracy measures of uniform accuracy, the true positive rate (positive predictive value) and the true negative rate (negative predictive value). They also consider the balanced classification rate, which is a version of accuracy that is unweighted per class.

Code and instructions can be found atand.

ML-Fairness Gym

ML-fairness gym is dynamic tool implemented in the environment API from OpenAI Gym that builds simple simulations that explores potential long-run impacts of deploying machine learning-based decision systems in social environments.and. This tool is more unique than some of the other tools because the decisions made by an algorithm at step t affect the next decision it will be asked to make at t+1. The simulations allow exploration of censoring in the observation mechanism, errors from the learning algorithm, and interactions between decision policy and environment dynamics. Assessments can be made on fairness characteristics of certain decision policies based on observed data alone. This can assist with future data collection practices. Simulations can also be used along with reinforcement learning algorithms. Although ML-fairness-gym is not an officially supported Google product, its environments implement the environment API from OpenAI Gym.

Code and instructions can be found atand.

RETAIN

RETAIN uses a Reverse Time Attention Mechanism to build a clinically interpretable ML model using the EHR (Electronic Health Record) data. The algorithm uses neural attention so that, given patient records, it can make predictions while explaining how each medical code (diagnosis codes, medication codes, or procedure codes) at each visit contributes to the prediction. The algorithm also treats sequence classification as a special case, which it makes at the last timestep only. There is a potential trade-off in that the algorithm learns to pay more attention to new information than old information. This should be analyzed on a case-by-case basis.

Code and instructions can be found at.

Knowledge Graphs with Gephi

A knowledge graph comprises a graph data model ontology of attributes and other data that may provide information about relatedness. Knowledge graphs model topical coherence of information based on an assumption that information from the same context tends to belong together. Data are often contextual and knowledge graphs utilize a data interchange standard comprising tuples where each tuple can be considered a small graph. It may also comprise any other associated attributes linked to the data. The primary purpose is classification, clustering, and relatedness that is able to be presented in visual form. The knowledge graph maps heterogeneous types of knowledge. The combined approaches should result in a decrease in disambiguation errors and result in a more discerning clustering of content and inference from the highly heterogeneous data. Gephi is a visualization and exploration software platform for all kinds of graphs and networks. It is open-source and free.

Code and instructions can be found at.

Adversarial Learning with Networkx

Data may also be run on a CNN and LSTM neural network framework as a generative adversarial network (GAN) model in order to detect anomalous patterns, such as those as previously described by Zenati et al.The results can be visualized using T-SNE as one means of assessment.

Data preparation in graph neural network allows for the representation of complex data in non-Euclidean space for complex relationships and inter-dependencies.Deep learning has been extended to GNNs extract latent representations from images. Data can be converted and manipulated as .ndjson data files, followed by applying a series of 1-dimensional convolutions, followed by LSTM layers, with the sum of the outputs of all LSTM steps fed into an argmax or softmax layer to make a classification decision using known classes as a starting point.

Networkxis a useful tool with capabilities in graphing, ability to encode attributes to nodes and edges, loading and saving graphs, and ability to convert to other graph formats such DGL. The remaining steps can be run as a progression of models to evaluate the impact of applying additional modeling methods on classification accuracy and similarity.

Code and instructions for GAN-based anomaly detection can be found at,and.

IMV-LSTM

IMV-LSTM uses a mixture attention mechanism in the hidden layers of recurrent neural networks (RNNs) to capture the input variable importance in a multi-variate time series set up. Interpretable insights into the data can be found by utilizing RNNs trained on time series with target and exogenous variables, in addition to accurate prediction. The goal of this process is to capture different dynamics in multi-variable time series and distinguish the contribution of variables to the prediction.

Code and instructions can be found at.

Data Triangulation

Data triangulation is an older standard statistical analysis used more frequently in the social sciences, but it also has applicability in machine learning models, as it is used to compare multiplicative groupings of data. Instead of utilizing binary pairings, binary pairings may be made, and then compared to other paired metrics. For example, age (as integers or categorically-transformed by decades) can be compared with binary sex or categorically-transformed race enumeration. All of these factors can then be compared to other features in the dataset and cross-correlated and compared. Data triangulation also works well on qualitative data because of the ability to categorically-transform the data and cross-compare the same categorical data over different features or cross-compare it over different categories.

Machine learning and deep learning models of artificial intelligence can incorporate data triangulation in neural networks in the form of mini-batches of additional data to compare signal to noise ratios, even in noisy datasets. The theory being that the signals will show up consistently across the multiplier iterations of datasets, even if some of the data are incomplete, missing, or excessively noisy. Data triangulation compares qualitative and quantitative data and can be used across all data types and from entirely different sources. Triangulation is a correlation process that attempts to define relational metrics between data while accounting for noise, overlaps, outliers, and density. Neural networks can also be triangulated concurrently with other neural networks in order to refine classification, reduce output error, and provide a better explanation of output results (explainable AI)and.

Mitigation

What happens after bias is found? There are choices to be made that can impact data cleaning (inclusion, exclusion, averaging of missing data or parts thereof), data weighting (reweighting), feature engineering (shift, exclude, reweight), and optimization that may disregard tails (include or exclude). No model is an accurate representation of the real-world. It is merely a representation of an approximation. As such, its degree of accuracy is variable because of inputs and necessary manipulation of data in order to compare apples-to-apples

Balanced or representative data sets are commendable, but full-disclosure if whether the representation is synthetic or real-world is necessary. Balanced datasets are not enough. Resampling the data can provide a way to make assessments on representation bias and mitigate against it by retraining from the resampled dataset.

Data are what they are. The real-world is messy. Real-world data are messy. Providing commentary and analysis about bias noted in the underlying data or in the models is very important, especially where the bias can’t be mitigated without over-correcting.

Caution Against Over-Correcting

Just as over-fitting and under-fitting are not optimal for analysis, explicit removal of biases and variations are not necessarily appropriate. This type of action is tantamount to introducing new subjective bias (over-correcting) into the data or model, directly impacting outcome. It could lead to a deliberately false model, which would be inherently unethical. Similarly, caution is warranted when considering introducing additional variables to offset any biasing variables. Instead, it is recommended to provide an explanation detailing the FATE assessment so that others may understand the limitations of the data or the model. It is better to convey the understanding that the field uses representative approximations and does not provide absolutes. There is much to be gained through a better understanding of data and models that may show bias that the data, alone, does not indicate. Similarly, all machine learning models are based on linear algebra and differential equations, as well as numerical rather than categorical values. An assessment of floats is not the same as a contextual assessment of categorical data. However, it is the best approximation that may be available at any given time.

Conclusion and Future Work

Many people have asked where to find papers and code for auditing tools. This review is a response to those requests. The goal of this work is to share the information as broadly and as quickly as possible, in light of the ever-changing, challenging environment and the desire of many to want to take proactive measures in their own work for the betterment of the field.

I have endeavored to provide multiple sources of information for each tool including Github links, links to video tutorials, and even PowerPoint slide presentations when the authors made them publicly available.

The tools addressed herein have been utilized on many different datasets by multiple groups all over the world. I have personally used several of them and have found to be reliable in finding bias among real-world and publicly available datasets across multiple domains. Many of these tools are also model-agnostic.

Many talented researchers and programmers have taken their time to create open-source tools for the greater good. Lets own our ethical algorithmic responsibilities to each other. Use the tools and pay it forward.

My personal thanks to my AI ethics colleagues and to all of the hardworking researchers who continually strive to promote fundamental fairness across algorithmic domains.

Future work consists of continuing to seek and promote algorithmic fairness, audits, privacy, and human rights in the field of machine learning.

00 b1 Deb Raji. https://foundation.mozilla.org/en/blog/its-time-to-develop-the-tools-we-need-to-hold-algorithms-accountable/ (Last Accessed 18 June 2022). b2 N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstya, A Survey on Bias and Fairness in Machine Learning, arXiv 1908.09635, 2019. b3 MA Gianfrancesco, S Tamang, J Yazdany, G Schmajuk, “Potential Biases in Machine Learning Algorithms Using Elecronic Health Record Data.” November 2018, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6347576 (Last Accessed 5 February 2022). b4 N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstyan, “A Survey on Bias and Fairness in Machine Learning,” August 2019, https://arxiv.org/abs/1908.09635 (Last Accessed 5 February 2022). b5 Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, “Dissecting racial bias in an algorithm used to manage the health of populations” Science, vol 366, October 2019, pp. 447-453, 10.1126/science.aax2342, https://pubmed.ncbi.nlm.nih.gov/31649194/ (Last Accessed 5 February 2022). b6 Grgić-Hlača, N., Zafar, M.B., Gummadi, K.P. and Weller, A. 2018. Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 32, 1 (Apr. 2018). https://ojs.aaai.org/index.php/AAAI/article/view/11296 (Last Accessed 18 June 2022). b7 NeurIPS Review - A Retrospective on the NeurIPS 2021 Ethics Review Process. https://blog.neurips.cc/2021/12/03/a-retrospective-on-the-neurips-2021-ethics-review-process/ (Last accessed 3 February 2022). b8 https://gizmodo.com/irs-will-require-facial-recognition-scans-to-access-you-1848387715 (Last Accessed 3 February 2022). b9 https://epic.org/irs-to-require-facial-recognition-to-verify-identity-online/ (Last Accessed 3 February 2022). b10 ttps://www.ntu.org/foundation/detail/irs-drops-questionable-idme-facial-scanner-program (Last Accessed 18 June 2022). b11 https://www.businessinsider.com/idme-data-privacy-handling-senators-markey-menendez-wyden-concerns-2022-6 (Last Accessed 18 June 2022). b12 https://www.cyberscoop.com/id-me-ceo-backtracks-on-claims-company-doesnt-use-powerful-facial-recognition-tech/ (Last Accessed 3 February 2022). b13 https://www.justice.gov/usao-edca/pr/new-jersey-man-indicted-fraud-scheme-steal-california-unemployment-insurance-benefits (Last Accessed 3 February 2022). b14 Christoph Molnar, ”Interpretable Machine Learning. A Guide for Making Black Box Models Explainable, 2020, https://christophm.github.io/interpretable-ml-book/ (Last Accessed 5 February 2022). b15 R.K.E. Bellamy, K. Dey, M. Hind, S.C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, S. Mehta, A. Mojsilovic, S. Nagar, K.N. Ramamurthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K.R. Varshney, and Y. Zhang, ”AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias,” arXiv 1810.01943, 2018. b16 https://developer.ibm.com/articles/the-ai-360-toolkit-ai-models-explained , Last Accessed 3 February 2022). b17 AIF360 IBM, https://github.com/ibm/aif360 (Last Accessed 3 February 2022). b18 Tutorial: AI Fairness 360 Open Source Toolkit, IBM Research Trusted AI, https://aif360.mybluemix.net/resources (Last Accessed 3 February 2022). b19 M.T. Ribeiro, S. Singh, C. Guestrin, ”Anchors: High-Precision Model-Agnostic Explanations” September 2018, pp. 241-252, https://ojs.aaai.org/index.php/AAAI/article/view/11491 (Last Accessed 3 February 2022). b20 https://www.oreilly.com/content/introduction-to-local-interpretable-model-agnostic-explanations-lime/ b21 https://homes.cs.washington.edu/  marcotcr/blog/lime/ (Last Accessed 18 June 2022). b22 https://github.com/marcotcr/lime , (Last Accessed 18 June 2022). b23 https://homes.cs.washington.edu/  marcotcr/aaai18.pdf (Last Accessed 18 June 2022). b24 https://github.com/marcotcr/anchor (Last Accessed 18 June 2022). b25 M.T. Ribeiro, S. Singh, C. Guestrin, “Why Should I Trust You? Explaining the Predictions of Any Classifier", August 2016, 10.1145/2939672.2939778. b26 https://www.kdd.org/kdd2016/papers/files/rfp0573-ribeiroA.pdf (Last Accessed 3 February 2022). b27 James Wexler, M. Pushkarna, T. Bolukbasi, M. Wattenberg, F. Viegas, and J. Wilson, “The What-If Tool: Interactive probing of machine learning models,” IEEE transactions on visualization and computer graphics, vol 26, 1, IEEE, 2019, pp. 56-65. b28 https://facets.dev (Last Accessed 18 June 2022). b29 James Wexler and Andrew Zaldivar, “FAT 2020 Hands-on Tutorial: Probing ML Models for Fairness with the What-If Tool and SHAP” 2020. b30 Scott Lundberg and Su-In Lee, “A Unified Approach to Interpreting Model Predictions” SHAP, 12/2022. b31 https://pair-code.github.io/what-if-tool/fat2020.html (Last Accessed 18 June 2022). b32 https://research.google/teams/brain/pair/ (Last Accessed 18 June 2022). b33 M. Zehlike, T. Sühr, C. Castillo, I. Kitanovski, “FairSearch: A Tool For Fairness in Ranked Search Results,” Companion Proceedings of the Web Conference 2020, ACM, Apr 2020, SBN9781450370240, http://dx.doi.org/10.1145/3366424.3383534 . b34 https://dl.acm.org/doi/10.1145/3132847.3132938 (Last Accessed 18 June 2022). b35 https://arxiv.org/abs/1905.13134 (Last Accessed 18 June 2022). b36 https://github.com/fair-search (Last Accessed 18 June 2022). b37 l. Tramèr, V. Atlidakis, R. Geambasu, D. Hsu, J.P. Hubaux, M. Humbert, A. Juels, H. Lin, ”FairTest: Discovering Unwarranted Associations in Data-Driven Applications,” arXiv 1510.02377, 2015. b38 https://github.com/columbia/fairtest (Last Accessed 18 June 2022). b39 https://vatlidak.github.io/resources/eurosp17fairtest.pptx , (Last Accessed 18 June 2022). b40 https://www.youtube.com/watch?v=IZIpbXtDYT4 , (Last Accessed 18 June 2022). b41 https://arxiv.org/pdf/1811.05577.pdf (Last Accessed 18 June 2022). b42 http://www.datasciencepublicpolicy.org/our-work/tools-guides/aequitas/ [NOTE: the site is not secure.] (Last Accessed 18 June 2022). b43 https://dssg.github.io/aequitas/ (Last Accessed 18 June 2022). b44 https://github.com/dssg/aequitas (Last Accessed 18 June 2022). b45 Niels Bantilan, “Themis-ml: A Fairness-Aware Machine Learning Interface for End-To-End Discrimination Discovery and Mitigation,” vol 36, 1, Journal of Technology in Human Services, Informa UK Limited, pp. 15-30, http://dx.doi.org/10.1080/15228835.2017.1416512 . b46 https://themis-ml.readthedocs.io/en/latest/ (Last Accessed 18 June 2022). b47 https://github.com/cosmicBboy/themis-ml/blob/master/paper/main.pdf (Last Accessed 18 June 2022). b48 S.A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E.P. Hamilton, D. Roth, ”A Comparative Study of Fairness-Enhancing Interventions In Machine Learning,” Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT2019, ACM Press, 2019, http://dx.doi.org/10.1145/3287560.3287589 . (Last Accessed 18 June 2022). b49 https://arxiv.org/abs/1802.04422 (Last Accessed 18 June 2022). b50 https://github.com/algofairness/fairness-comparison (Last Accessed 18 June 2022). b51 https://paperswithcode.com/paper/a-comparative-study-of-fairness-enhancing (Last Accessed 18 June 2022). b52 D’Amour, Alexander and Srinivasan, Hansa and Atwood, James and Baljekar, Pallavi and Sculley, D. and Halpern, Yoni, “Fairness is Not Static: Deeper Understanding of Long Term Fairness via Simulation Studies”, 2020. Association for Computing Machinery. b53 https://doi.org/10.1145/3351095.3372878 (Last Accessed 18 June 2022). b54 https://github.com/google/ml-fairness-gym/ (Last Accessed 18 June 2022). b55 https://www.youtube.com/watch?v=QdW2xieKhlE (Last Accessed 18 June 2022). b56 E. Choi, T. Bahadori, A. Schuetz, W. Stewart, J. Sun, “RETAIN: Interpretable Predictive Model in Healthcare using Reverse Time Attention Mechanis” August 2016, https://arxiv.org/abs/1608.05745 (Last Accessed 18 June 2022). b57 https://github.com/mp2893/retain (Last Accessed 18 June 2022). b58 Lin et al., “Learning Entity and Relation Embeddings for Knowledge Graph Completion,” AAAI Conference on Artificial Intelligence, North America, 2015, https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9571/9523 (Last Accessed 5 February 2022). b59 Huang et al., ”Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation,” arXiv:1504.07678v1. 28 Apr 2015. b60 M. Bastian, S. Heymann, M. Jacomy ,Gephi: An Open Source Software for Exploring and Manipulating Networks. International AAAI Conference on Weblogs and Social Media. 2009. b61 https://gephi.org/users/publications/ (Last accessed 3 February 2022). b62 Zenati et al,”Efficient GAN-Based Anomaly Detection,” 2019, arxiv.org/abs/1802.06222 (Last Accessed 18 June 2022). b63 A Hagberg, D Schult, P Swart, “Exploring Network Structure, Dynamics, and Function using NetworkX”, Proceedings of the 7th Python in Science conference (SciPy 2008), G Varoquaux, T Vaught, J Millman (Eds), pp. 11-15. b64 Mufei et al, “Tutorial: Batched graph classification with Deep Graph Library,” https://docs.dgl.ai (Last Accessed 5 February 2022). b65 https://github.com/houssamzenati/Efficient-GAN-Anomaly-Detection (Last Accessed 18 June 2022). b66 https://paperswithcode.com/paper/efficient-gan-based-anomaly-detection (Last Accessed 18 June 2022). b67 https://github.com/dmlc/dgl/tree/0.8.x (Last Accessed 18 June 2022). b68 T. Guo, T. Lin, Tao, N. Antulov-Fantulin, “Exploring Interpretable LSTM Neural Networks over Multi-Variable Data,” May 2019, b69 https://arxiv.org/abs/1905.12034 (Last Accessed 18 June 2022). b70 https://paperswithcode.com/paper/exploring-interpretable-lstm-neural-networks (Last Accessed 18 June 2022). b71 Linda Heath, “Triangulation Methodology”, International Encyclopedia of the Social Behavioral Sciences (Second Edition), 2015. b72 https://www.sciencedirect.com/topics/computer-science/triangulation (Last Accessed 3 February 2022). b73 Steven Walczak, “Methodological Triangulation Using Neural Networks for Business Research,” Advances in Artificial Neural Systems, 2012, 12 pages, doi:10.1155/2012/517234, https://dl.acm.org/doi/abs/10.1155/2012/517234 (Last Accessed 5 February 2022). b74 Aldrees, A, Poland, C, Irshad, SA (2022). Auditing Algorithms: Determining Ethical Parameters of Algorithmic Decision-Making Systems in Healthcare. Information Management and Big Data. SIMBig 2021. Communications in Computer and Information Science, vol 1577. Springer, Cham. https://doi.org/10.1007/978-3-031-04447-2_20 .

Attribution

arXiv:2206.10613v1 [cs.LG]
License: cc-by-4.0