Fairness constraints ==================== ``fair-seldonian`` ships builders for the most common fairness definitions, so a named criterion can go straight into :class:`~fair_seldonian.config.SeldonianConfig` instead of a hand-written reverse-Polish (postfix) string: .. code-block:: python from fair_seldonian import SeldonianConfig, demographic_parity config = SeldonianConfig(constraint=demographic_parity(epsilon=0.1)) Each builder takes a tolerance ``epsilon`` (smaller is stricter). The group-parity builders also take a ``groups=(g1, g0)`` pair, defaulting to ``("1", "0")``, while :func:`~fair_seldonian.constraints.fairness.error_rate` takes a single ``group``. Labels are matched against ``str(T)``, so the defaults line up with a 0/1 sensitive column. What each builder produces -------------------------- The table below is generated by calling each builder, so it shows the string the library actually emits today rather than a transcription of it. .. fs-constraint-table:: Two things the *Affine forms* column tells you. A constraint inside the affine fragment can be bounded by :ref:`variant-affine`, which is roughly twice as tight as interval arithmetic — and all five builders qualify, because they are written over the rate primitives rather than over explicit ratios. The count itself is the number of :math:`\delta` slices spent: ``equalized_odds`` needs four, which is why it is the loosest of the five at a fixed tolerance. The definitions --------------- .. fs-constraint-cards:: Choosing a definition --------------------- .. grid:: 1 1 2 2 :gutter: 3 .. grid-item-card:: Ignore the label :func:`~fair_seldonian.constraints.fairness.demographic_parity` asks only that each group be predicted positive at the same rate. Natural for **allocative** decisions, where a positive prediction grants access to something and equal access is the goal. It can be satisfied by a model that is deliberately worse for one group. .. grid-item-card:: Condition on the label :func:`~fair_seldonian.constraints.fairness.equal_opportunity` and :func:`~fair_seldonian.constraints.fairness.equalized_odds` compare rates *among people with the same true label*, so they do not penalise a model for genuinely different base rates between groups. Use them when the cost of a specific error type must be shared fairly. .. grid-item-card:: Bound the errors :func:`~fair_seldonian.constraints.fairness.error_rate` caps one group's misclassification rate — a performance bound rather than a comparison. :func:`~fair_seldonian.constraints.fairness.error_rate_parity` equalises total error without constraining which *kind* of error may differ. .. grid-item-card:: Combine them Nothing stops you pairing a parity constraint with an error bound so the model must be both *fair* and *good*. Each additional term spends its own slice of :math:`\delta`, so a combined constraint needs more data. .. warning:: These criteria are mutually incompatible in general. Except in degenerate cases — equal base rates across groups, or a perfect classifier — no model can simultaneously satisfy demographic parity and equalized odds, and no model can be both calibrated and equalized-odds fair when base rates differ. Choosing a definition is a decision about *which* notion of fairness the application needs, not a technical detail. See Barocas, Hardt & Narayanan (2023), `fairmlbook.org `_. Writing your own ---------------- The builders are conveniences. ``SeldonianConfig.constraint`` accepts any postfix expression over the base variables below, so a criterion that is not shipped can still be certified. .. fs-basevar-table:: The distinction between the three kinds matters and is easy to miss. A **cell** such as ``TP(g)`` is a fraction of the *whole* group, so every row of the group contributes and rows with the wrong label contribute zero. A **rate** such as ``TPR(g)`` is a mean over only that group's positive rows, so it carries a different — smaller — sample size, and therefore a wider interval per unit of :math:`\delta`. Two base variables are independent exactly when the rows they average over are disjoint, which is what licenses the affine bound. Operators are ``+ - * / ^`` and ``abs``, all in postfix position: .. code-block:: python from fair_seldonian import SeldonianConfig # Demographic parity, written out by hand. SeldonianConfig(constraint="PR(1) PR(0) - abs 0.1 -") # Cap group 1's false-negative rate at 15%. SeldonianConfig(constraint="FNR(1) 0.15 -") Every constraint encodes :math:`g(\theta) \leq 0`, which is why each example ends by subtracting its tolerance. :class:`~fair_seldonian.config.SeldonianConfig` runs :func:`~fair_seldonian.constraints.expression_tree.validate_constraint` on construction, so a malformed expression raises ``ValueError`` immediately rather than failing deep inside the algorithm. .. seealso:: :doc:`inequalities` for how wide each interval is, :doc:`variants` for how the intervals are combined, and :mod:`fair_seldonian.constraints.fairness` for the full API.