---
title: "Algorithm auditing: Get started"
author: Koen Derks
date: "`r Sys.Date()`"
output: 
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 3
vignette: >
  %\VignetteIndexEntry{Algorithm auditing: Get started}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteDepends{jfa}
  %\VignetteKeywords{algorithm, audit, bias, fairness, model, performance}
  %\VignettePackage{jfa}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
library(jfa)
```

## Introduction

Welcome to the 'Algorithm auditing' vignette of the **jfa** package. Here you
can find a detailed explanation of the functions in the package that facilitate
auditing of algorithms and predictive models. For more detailed explanations of
each function, read the other vignettes on the
[package website](https://koenderks.github.io/jfa/articles/index.html).

## Functions and intended usage

Below you can find an explanation of the available algorithm auditing functions
in **jfa**.

- [`model_fairness()`](#assess-algorithmic-fairness-with-model_fairness)

### Assess algorithmic fairness with `model_fairness()`

[![Lifecycle: experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)

The `model_fairnesS()` function aims to assess fairness in algorithmic
decision-making systems by computing and testing the equality of one of
several model-agnostic fairness metrics between protected classes based on a
set of true labels and the predictions of an algorithm. The ratio of these
metrics between an unpriveleged protected class and a priveleged protected
class is called parity, and quantifies  relative fairness in the algorithms
predictions. Available parity metrics include predictive rate parity,
proportional parity, accuracy parity, false negative rate parity, false
positive rate parity, true positive rate parity, negative predicted value
parity, specificity parity, and demographic parity. The function returns an
object of class `jfaFairness` that can be used with associated
`summary()` and `plot()` methods.

*Full function with default arguments:*

```r
model_fairness(
  data,
  protected,
  target,
  predictions,
  privileged = NULL,
  positive = NULL,
  metric = c(
    "prp", "pp", "ap", "fnrp", "fprp",
    "tprp", "npvp", "sp", "dp"
  ),
  alternative = c("two.sided", "less", "greater"),
  conf.level = 0.95,
  prior = FALSE
)
```

*Example usage:*

```{r}
model_fairness(
  data = compas,
  protected = "Ethnicity",
  target = "TwoYrRecidivism",
  predictions = "Predicted",
  privileged = "Caucasian",
  positive = "yes",
  metric = "prp"
)
```

## Benchmarks

To validate the statistical results, **jfa**'s automated
[unit tests](https://github.com/koenderks/jfa/tree/development/tests/testthat)
regularly verify the main output from the package against the following
benchmarks:

- [fairness](https://cran.r-project.org/package=fairness) (R package version 1.2.2)

## References

- Büyük, S. (2023). *Automatic Fairness Criteria and Fair Model Selection for Critical ML Tasks*, Master Thesis, Utrecht University. - [View Online](https://studenttheses.uu.nl/handle/20.500.12932/44225)
- Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. In *Proceedings of the Conference on Fairness, Accountability, and Transparency*. - [View Online](https://doi.org/10.1145/3287560.3287589)
- Pessach, D. & Shmueli, E. (2022). A review on fairness in machine learning. *ACM Computing Surveys*, 55(3), 1-44. - [View Online](https://doi.org/10.1145/3494672)
