fair_seldonian.constraints package#

Submodules#

fair_seldonian.constraints.affine module#

Bound a constraint by compiling it to a max of affine forms.

The default path wraps a confidence interval around every node of the constraint tree and combines them with interval arithmetic. That is sound but loose, for two compounding reasons. Interval arithmetic assumes the worst about how the sub-expressions relate, even when they are means over disjoint groups and hence independent. And every leaf occurrence spends its own slice of delta, so a constraint mentioning three leaves pays a three-way union bound.

For constraints built from +, -, scaling by a constant and abs - which covers the standard gap-based fairness definitions - neither cost is necessary. Such an expression can be rewritten exactly as

g(θ)=max⁡k(ck+∑vak,vzv)g(\theta) = \max_k \left( c_k + \sum_v a_{k,v} z_v \right)

a maximum of finitely many affine forms in the base variables. Each form can then be bounded with a single interval, because all cells of one group are means over the same rows: for group g the partial sum sum_v a_v z_v is itself the mean over that group’s samples of a scalar w_i = sum_v a_v x_i^{(v)}. Distinct groups are disjoint, so conditional on the group counts their means are independent and Hoeffding applies to the weighted sum directly:

half-width=12ln⁡(1/δ′)∑grg2/ng\text{half-width} = \sqrt{\tfrac{1}{2}\ln(1/\delta')\sum_g r_g^2 / n_g}

with r_g the a-priori range of w_i within group g. Only K slices of delta are spent, one per form, instead of one per leaf occurrence.

On the constraint |TP(0) - TP(1)| - 0.2 TP(1) <= 0 this compiles to max(TP(0) - 1.2 TP(1), -TP(0) + 0.8 TP(1)) - two forms rather than three leaf intervals - and halves the slack over the true value. Measured at 50.0% across sample sizes from 10k to 160k and several parameter vectors; the ratio barely moves, because the saving comes from the delta split and the independence of the groups rather than from the data. Since a Hoeffding half-width scales as 1/n1/\sqrt{n}, halving it is worth roughly 4x the data.

class fair_seldonian.constraints.affine.AffineForm(constant=0.0, coefficients=None)[source]#

Bases: object

constant + sum_v coefficient[v] * v over base-variable tokens.

Parameters:
constant#
coefficients#
scaled(factor)[source]#
Parameters:

factor (float)

Return type:

AffineForm

plus(other)[source]#
Parameters:

other (AffineForm)

Return type:

AffineForm

exception fair_seldonian.constraints.affine.NotAffine[source]#

Bases: Exception

The constraint uses an operation this compilation cannot represent.

fair_seldonian.constraints.affine.affine_upper_bound(
root,
Y,
predicted_Y,
T,
delta,
sample_scale=1.0,
inflate=1.0,
inequality=Inequality.HOEFFDING_INEQUALITY,
)[source]#

Upper bound on the constraint via its max-of-affine-forms compilation.

The budget is split evenly across the K forms. Even splitting is not optimal - the forms have different widths, so a convex allocation would do slightly better - but the dominant saving comes from having K intervals instead of one per leaf occurrence.

Parameters:
Return type:

Bound

fair_seldonian.constraints.affine.compile_bounds(node)[source]#

Return (upper_forms, lower_forms) for the sub-expression at node.

max(upper_forms) is an exact upper envelope of the expression and min(lower_forms) a lower one. Both are returned because subtraction reads the lower envelope of its right operand.

Parameters:

node (ExprTree | None)

Return type:

tuple[list[AffineForm], list[AffineForm]]

fair_seldonian.constraints.affine.form_upper_bound(
form,
Y,
predicted_Y,
T,
delta,
sample_scale=1.0,
inflate=1.0,
inequality=Inequality.HOEFFDING_INEQUALITY,
)[source]#

One-sided upper bound on a single affine form.

Hoeffding uses the a-priori range of w_i; the t-test and empirical Bernstein use its measured variance instead, which is much tighter when the per-sample values do not fill their range. Because the groups are independent given the counts, the variance of the whole form is just the sum of the per-group variances of the mean, so the same decomposition works for all three.

sample_scale and inflate support the candidate-selection prediction of the safety-test bound: scale the group counts to the safety split’s size and widen by the usual factor of 2. As elsewhere, that prediction is a heuristic and carries no guarantee.

Parameters:
Return type:

Bound

fair_seldonian.constraints.bounds module#

fair_seldonian.constraints.bounds.eval_div_bound(l_x, u_x, l_y, u_y)[source]#
Parameters:

  • l_x (Bound | None) – lower bound of left child

  • u_x (Bound | None) – upper bound of left child

  • l_y (Bound | None) – lower bound of right child

  • u_y (Bound | None) – upper bound of right child

Returns:

lower and upper bound of div operation

Return type:

tuple[Bound | None, Bound | None]

fair_seldonian.constraints.bounds.eval_math_bound(l_x, u_x, l_y=None, u_y=None, operator=None)[source]#
Parameters:

  • l_x (Bound | None)

  • u_x (Bound | None)

  • l_y (Bound | None)

  • u_y (Bound | None)

  • operator (str | None)

Return type:

tuple[Bound | None, Bound | None]

fair_seldonian.constraints.bounds.eval_multiply_bound(l_x, u_x, l_y, u_y)[source]#
Parameters:

  • l_x (Bound | None) – lower bound of left child

  • u_x (Bound | None) – upper bound of left child

  • l_y (Bound | None) – lower bound of right child

  • u_y (Bound | None) – upper bound of right child

Returns:

lower and upper bound of multiply operation

Return type:

tuple[Bound | None, Bound | None]

fair_seldonian.constraints.expression_tree module#

class fair_seldonian.constraints.expression_tree.ExprTree(value)[source]#

Bases: object

An expression tree node of the constraint tree

Parameters:

value (str)

fair_seldonian.constraints.expression_tree.child_sides(node_value, sides, left_value, right_value)[source]#

Propagate “which endpoints do I need” from a node down to its children.

A leaf only has to pay for the endpoints that are actually read. The safety test reads a single number - the upper bound on g - so the root needs only its upper endpoint, and for constraints built from +, - and scaling by a non-negative constant that one-sidedness reaches all the way down to the leaves. Those leaves can then use ln(1/delta) instead of ln(2/delta). The half-width scales as the square root of that term, so at delta = 0.05 the interval narrows by 1 - sqrt(ln(20)/ln(40)), just under 10%.

abs breaks it: U(|x|) = max(-L(x), U(x)) reads both endpoints of its operand, so everything under an abs is two-sided. Products and quotients of two variables are treated as two-sided too, because eval_multiply_bound() and eval_div_bound() branch on the signs of both endpoints - an unguaranteed endpoint could select the wrong branch and so corrupt the endpoint that is guaranteed.

Scaling by a constant is safe to pass through: for a fixed c every branch of the multiply rule collapses to (l*c, u*c) for c >= 0 and (u*c, l*c) for c < 0, so only the corresponding endpoint of the variable child is read.

Parameters:

  • node_value (str) – this node’s token.

  • sides (tuple[bool, bool]) – the endpoints of this node that its parent needs.

  • left_value (str | None) – the left child’s token, or None.

  • right_value (str | None) – the right child’s token, or None.

Returns:

(left_sides, right_sides).

Return type:

tuple[tuple[bool, bool], tuple[bool, bool]]

fair_seldonian.constraints.expression_tree.constraint_groups(rev_polish_notation)[source]#

The distinct sensitive-attribute values a constraint refers to.

"TP(1) TP(0) - abs 0.1 -" yields ["0", "1"]. Pair this with check_constraint_groups() to confirm up front that every group the constraint names is actually present in T.

Parameters:

rev_polish_notation (str)

Return type:

list[str]

fair_seldonian.constraints.expression_tree.construct_expr_tree_base(
rev_polish_notation: str,
node_class: None = None,
) → ExprTree[source]#
fair_seldonian.constraints.expression_tree.construct_expr_tree_base(
rev_polish_notation: str,
node_class: type[_NodeT],
) → _NodeT

Returns root of constructed tree for given postfix expression

Parameters:

  • rev_polish_notation (str) – string with space as delimiter ‘ ‘

  • node_class (type[ExprTree] | None) – the tree node class to use (default: ExprTree)

Returns:

ExprTree node

Return type:

ExprTree

fair_seldonian.constraints.expression_tree.eval_expr_tree_base(t_node, Y, predicted_Y, T)[source]#

A utility function to evaluate estimate of the expression tree

Parameters:

  • t_node (ExprTree | None) – ExprTree node

  • Y (Array | None) – pandas::Series

  • predicted_Y (torch.Tensor | None) – tensor

  • T (Array | None) – pandas::Series

Returns:

estimate value: float

Return type:

Bound | None

fair_seldonian.constraints.expression_tree.eval_expr_tree_conf_interval_base(
t_node,
Y,
predicted_Y,
T,
delta,
inequality,
candidate_safety_ratio,
predict_bound,
modified_h,
sides=(False, True),
)[source]#

To evaluate confidence interval of the expression tree

Parameters:

  • t_node (ExprTree | None) – ExprTree node

  • Y (Array) – pandas::Series The true labels of the dataset

  • predicted_Y (torch.Tensor) – tensor The predicted labels of the dataset

  • T (Array) – pandas::Series The sensitive attributes of the dataset

  • delta (float) – float in [0, 1] The significance level

  • inequality (Inequality) – Enum The inequality to be used - Hoeffding/T-test

  • candidate_safety_ratio (float | None) – The candidate to safety ratio used in the experiment

  • predict_bound (bool) – Whether we are finding bound for candidate or safety data

  • modified_h (bool) – Whether modified confidence bound is used

  • sides (Sides) – which endpoints of this node’s interval the caller consumes. Defaults to the root’s requirement (upper only); see child_sides().

Returns:

upper and lower bound of the estimate of the constraint

Return type:

tuple[Bound | None, Bound | None]

fair_seldonian.constraints.expression_tree.inorder(t_node)[source]#

A utility function to log inorder traversal

Parameters:

t_node (ExprTree | None) – ExprTree node

Returns:

None

Return type:

None

fair_seldonian.constraints.expression_tree.is_func(element)[source]#
Parameters:

element (str)

Return type:

bool

fair_seldonian.constraints.expression_tree.is_mod(element)[source]#
Parameters:

element (str)

Return type:

bool

fair_seldonian.constraints.expression_tree.is_operator(element)[source]#
Parameters:

element (str)

Return type:

bool

fair_seldonian.constraints.expression_tree.validate_constraint(rev_polish_notation)[source]#

Validate a reverse-Polish (postfix) constraint string.

Checks that every token is recognized, that each operator/abs has enough operands, and that the whole expression reduces to a single value - i.e. that construct_expr_tree_base() can turn it into an evaluable tree. This is what SeldonianConfig runs on its constraint so that a malformed custom string fails immediately instead of deep inside the algorithm.

Parameters:

rev_polish_notation (str) – the postfix constraint string to validate.

Raises:

ValueError – if the string is empty or not a valid postfix expression.

Return type:

None

fair_seldonian.constraints.expression_tree_ext module#

class fair_seldonian.constraints.expression_tree_ext.ExprTree(value)[source]#

Bases: ExprTree

Extended expression tree node with delta and sidedness tracking

Parameters:

value (str)

left: ExprTree | None#
right: ExprTree | None#
delta: float#
sides: tuple[bool, bool] = (False, True)#
add_delta(delta)[source]#
Parameters:

delta (float)

Return type:

None

fair_seldonian.constraints.expression_tree_ext.change_deltas(t_node, hash_map)[source]#

Collapse repeated occurrences of a base variable onto one shared interval.

If TP(1) appears three times with budgets d/2, d/4 and d/8, the naive tree treats them as three independent intervals and pays d/2 + d/4 + d/8 for them. Building a single interval at the summed budget 7d/8 costs the same failure probability but is narrower than all three, and the constraint then sees one consistent value for the variable.

The sidedness must be merged too, and this is load-bearing: the argument above only holds if the occurrences really are one interval. Two occurrences with the same delta but different sidedness would produce two different widths, hence two failure events, and the total would silently become 2 * 7d/8. Taking the union of the endpoint requirements makes every occurrence identical.

Parameters:
Return type:

None

fair_seldonian.constraints.expression_tree_ext.construct_expr_tree(
rev_polish_notation,
delta,
check_bound,
check_constant,
)[source]#

Returns root of constructed tree for given postfix expression

Parameters:

  • rev_polish_notation (str) – string with space as delimiter ‘ ‘

  • delta (float)

  • check_bound (bool)

  • check_constant (bool)

Returns:

ExprTree node

Return type:

ExprTree

fair_seldonian.constraints.expression_tree_ext.eval_expr_tree(t_node, Y=None, predicted_Y=None, T=None)[source]#
Parameters:

  • t_node (_BaseExprTree | None)

  • Y (Array | None)

  • predicted_Y (torch.Tensor | None)

  • T (Array | None)

Return type:

Bound | None

fair_seldonian.constraints.expression_tree_ext.eval_expr_tree_conf_interval(
t_node,
Y,
predicted_Y,
T,
inequality,
candidate_safety_ratio,
predict_bound,
modified_h,
)[source]#
Parameters:
Return type:

tuple[Bound | None, Bound | None]

fair_seldonian.constraints.expression_tree_ext.inorder_ext(t_node)[source]#
Parameters:

t_node (ExprTree | None)

Return type:

None

fair_seldonian.constraints.fairness module#

Ready-to-use fairness constraints.

Each function returns a constraint in the reverse-Polish (postfix) notation that SeldonianConfig expects, so you can plug a named fairness definition straight into the algorithm without hand-writing the string:

from fair_seldonian import SeldonianConfig, demographic_parity

config = SeldonianConfig(constraint=demographic_parity(epsilon=0.1))

The constraints are written over the base variables this library exposes as primitives, of which there are three kinds:

  • cells TP(g), FP(g), FN(g), TN(g) - each a fraction of group g, the four summing to 1 within a group;

  • label-conditioned rates TPR(g), FPR(g), TNR(g), FNR(g) - TPR(g) is P(Y-hat = 1 | Y = 1, A = g), a mean over only that group’s positive rows;

  • predicted rates PR(g), NR(g) - PR(g) is P(Y-hat = 1 | A = g), equal to TP(g) + FP(g).

The builders use whichever primitive expresses the definition in the fewest leaves and without division. Demographic parity is written over PR(g) rather than TP(g) + FP(g) (two leaves instead of four), and equal opportunity over TPR(g) rather than TP(g) / (TP(g) + FN(g)). The identities hold either way, but every leaf spends its own slice of delta, and division would put the constraint outside the affine fragment.

Every constraint encodes g(theta) <= 0 and is bounded by a tolerance epsilon (smaller is stricter). Most bound a between-group gap and take a groups pair; error_rate() instead bounds a single group’s error rate and takes one group. Group labels are matched against str(T), so the defaults ("1", "0") line up with a 0/1 sensitive column.

Note

equal_opportunity and equalized_odds still need more data to certify than the others, but not because of division - none of the builders divide. Their leaves are label-conditioned, so each averages over a subset of its group (only the positives, or only the negatives) and carries a smaller sample size, which widens the interval. equalized_odds additionally spends four slices of delta rather than two.

fair_seldonian.constraints.fairness.FAIRNESS_CONSTRAINTS = {'demographic_parity': <function demographic_parity>, 'equal_opportunity': <function equal_opportunity>, 'equalized_odds': <function equalized_odds>, 'error_rate_parity': <function error_rate_parity>}#

Group-parity builders, keyed by name. Each takes (epsilon, groups) and bounds a between-group gap. error_rate() is a single-group performance bound with a different signature and is intentionally not included here.

fair_seldonian.constraints.fairness.demographic_parity(epsilon=0.1, groups=('1', '0'))[source]#

Demographic parity: equal predicted-positive rate across groups.

Bounds |P(Y-hat = 1 | A = g1) - P(Y-hat = 1 | A = g0)| <= epsilon.

Also called statistical parity or the independence criterion: it requires the prediction to be statistically independent of the sensitive attribute, so each group is predicted positive at the same rate regardless of the true label Y. Because it ignores Y, it is a natural target for allocative decisions - where a positive prediction grants access to a benefit and the goal is equal access across groups - but it can be satisfied by a model that is deliberately less accurate for one group. It is division-free and therefore the easiest of these criteria to certify.

References

  • Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. ITCS ‘12. https://arxiv.org/abs/1104.3913

  • Barocas, S., Hardt, M., & Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities (independence criterion). MIT Press. https://fairmlbook.org

Parameters:

  • epsilon (float) – maximum allowed gap between the groups’ positive rates.

  • groups (tuple[object, object]) – the two sensitive-attribute values (g1, g0) to compare.

Returns:

the constraint in postfix notation.

Return type:

str

fair_seldonian.constraints.fairness.equal_opportunity(epsilon=0.1, groups=('1', '0'))[source]#

Equal opportunity: equal true-positive rate (recall) across groups.

Bounds |TPR(g1) - TPR(g0)| <= epsilon, where TPR(g) = P(Y-hat = 1 | Y = 1, A = g).

Introduced by Hardt, Price & Srebro (2016) as the single-error-rate relaxation of equalized odds. Unlike demographic parity it is conditioned on the true label, so it only asks that qualified members (those with Y = 1) have an equal chance of a positive prediction across groups, and does not penalise a model for differing base rates between groups. It is the right target when the cost of a missed positive (a false negative) is what must be shared fairly - e.g. equal recall among applicants who truly qualify.

References

Parameters:

  • epsilon (float) – maximum allowed gap between the groups’ true-positive rates.

  • groups (tuple[object, object]) – the two sensitive-attribute values (g1, g0) to compare.

Returns:

the constraint in postfix notation.

Return type:

str

fair_seldonian.constraints.fairness.equalized_odds(epsilon=0.1, groups=('1', '0'))[source]#

Equalized odds: equal true- and false-positive rates across groups.

Bounds the sum of the two gaps by epsilon:

|TPR(g1) - TPR(g0)| + |FPR(g1) - FPR(g0)| <= epsilon

where FPR(g) = P(Y-hat = 1 | Y = 0, A = g).

Introduced by Hardt, Price & Srebro (2016) as the separation criterion: the prediction must be independent of the sensitive attribute conditional on the true label, i.e. groups must be matched on both true-positive and false-positive rate. It therefore controls unfairness for both qualified and unqualified members, and is stricter than equal_opportunity(), which bounds the true-positive gap alone. Here both gaps are required to fit within a single tolerance epsilon.

References

Parameters:

  • epsilon (float) – maximum allowed sum of the true- and false-positive gaps.

  • groups (tuple[object, object]) – the two sensitive-attribute values (g1, g0) to compare.

Returns:

the constraint in postfix notation.

Return type:

str

fair_seldonian.constraints.fairness.error_rate(epsilon=0.1, group='1')[source]#

Error rate: bound a single group’s misclassification rate.

Bounds P(Y-hat != Y | A = group) = FP(group) + FN(group) <= epsilon - one minus that group’s accuracy. Unlike the other builders this is a performance (behavioral) bound on one group rather than a between-group comparison: use it to cap how often the model errs on a chosen subpopulation (or on the whole sample, when the data is treated as a single group). Pair it with a parity constraint when you want error to be both low and equal. It is division-free.

References

  • Thomas, P. S., da Silva, B. C., Barto, A. G., Giguère, S., Brun, Y., & Brunskill, E. (2019). Preventing undesirable behavior of intelligent machines. Science, 366(6468), 999-1004 (behavioral constraints such as bounded error). https://doi.org/10.1126/science.aag3311

Parameters:

  • epsilon (float) – maximum allowed misclassification rate for the group.

  • group (object) – the sensitive-attribute value whose error rate is bounded.

Returns:

the constraint in postfix notation.

Return type:

str

fair_seldonian.constraints.fairness.error_rate_parity(epsilon=0.1, groups=('1', '0'))[source]#

Error-rate parity: equal misclassification rate across groups.

Bounds |err(g1) - err(g0)| <= epsilon, where err(g) = P(Y-hat != Y | A = g) = FP(g) + FN(g).

Also called overall accuracy equality: it asks that the model be right equally often for each group, without constraining which kind of error (false positive or false negative) may differ. It is division-free like demographic parity. Note that equal total error can still hide a group whose mistakes are mostly false negatives while another’s are mostly false positives; use equalized_odds() when the error types must match too.

References

  • Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1), 3-44 (overall accuracy equality). https://arxiv.org/abs/1703.09207

  • Barocas, S., Hardt, M., & Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities. MIT Press. https://fairmlbook.org

Parameters:

  • epsilon (float) – maximum allowed gap between the groups’ error rates.

  • groups (tuple[object, object]) – the two sensitive-attribute values (g1, g0) to compare.

Returns:

the constraint in postfix notation.

Return type:

str

fair_seldonian.constraints.inequalities module#

class fair_seldonian.constraints.inequalities.Inequality(*values)[source]#

Bases: Enum

The concentration inequality used to build confidence intervals.

HOEFFDING_INEQUALITY and EMPIRICAL_BERNSTEIN are distribution-free and give a genuine high-confidence guarantee. T_TEST assumes approximate normality of the sample mean, which makes the result quasi-Seldonian: the guarantee is only as good as that approximation.

Empirical Bernstein pays for the variance it measures rather than assuming the worst case of 1/4, so it is much tighter for base variables whose value is far from 1/2 and slightly looser at exactly 1/2. Pick it when a group’s rate is small - which is the common case for a minority group.

BETTING is the Waudby-Smith and Ramdas betting interval, which is also distribution-free and dominates the other two for bounded variables: it adapts to the observed variance like empirical Bernstein but without paying the additive penalty term. Locating each endpoint takes repeated passes over the data, so it is the slowest of the four.

T_TEST = 1#
HOEFFDING_INEQUALITY = 2#
EMPIRICAL_BERNSTEIN = 3#
BETTING = 4#
fair_seldonian.constraints.inequalities.betting_interval(x, delta, two_sided=True)[source]#

Betting confidence interval for the mean of values in [0, 1].

The interval is the set of candidate means the capital test does not reject. Rather than scan a grid - which can step straight over a narrow interval and find nothing - we anchor at the sample mean, walk outwards until the test rejects, and bisect. Each endpoint is reported on the rejected side of the boundary, so discretisation can only ever widen the interval.

Parameters:

  • x (np.ndarray)

  • delta (float)

  • two_sided (bool)

Return type:

tuple[Bound, Bound]

fair_seldonian.constraints.inequalities.check_constraint_groups(groups, T)[source]#

Raise if a group named in the constraint matches no row of T.

Group labels are compared as strings, so an integer column and a float column behave differently: str(1) == "1" but str(1.0) == "1.0". Passing T through anything that upcasts to float - DataFrame.values on a frame with any float column, for instance - therefore makes every mask empty. The bound then fails closed to +inf, the safety test rejects everything, and the run looks like a legitimate “no solution found” rather than a type error. Call this once up front so the mistake is loud.

Parameters:
Return type:

None

fair_seldonian.constraints.inequalities.conditioning_set(element)[source]#

Which rows a base variable is a mean over.

A cell such as TP(g) is a fraction of the whole of group g: every row of the group contributes, rows with the wrong label contributing zero. A rate such as TPR(g) is a mean over only the rows of group g with Y=1Y = 1. The distinction sets the sample size an inequality may use, and it decides which base variables are independent of one another: two variables are independent exactly when their conditioning sets are disjoint.

Parameters:

element (str)

Return type:

tuple[str, int | None]

fair_seldonian.constraints.inequalities.contributions(element, Y, predicted_Y, T)[source]#

Per-sample contributions to the base variable element, over its group.

TP(A), FP(A), TN(A), FN(A) are fractions of group A: the four cells sum to 1 within a group. So TP(A) estimates P(Y^=1,Y=1∣T=A)P(\hat{Y}=1, Y=1 \mid T=A) - a joint probability, not the true-positive rate P(Y^=1∣Y=1,T=A)P(\hat{Y}=1 \mid Y=1, T=A).

PR(A) and NR(A) are the predicted-positive and predicted-negative rates, TP + FP and TN + FN. They ignore the label, so they are means over the whole group too and likewise sum to 1.

TPR, FPR, TNR and FNR are the label-conditioned rates, means over a subset of the group; see conditioning_set().

Concretely, for TP(A) this returns the vector

xi=1[Yi=1]⋅p^i,i∈{T=A}x_i = \mathbb{1}[Y_i = 1] \cdot \hat{p}_i, \quad i \in \{T = A\}

whose mean is the estimate and whose length is the sample size that any concentration inequality must use. Returning the raw vector (rather than the mean alone) is deliberate: the estimate, its sample size and its variance are then guaranteed to refer to the same index set. Computing them separately invites a mismatch that is hard to see and unsound: building the interval from #{Y == 1} over all groups while the estimate divides by the size of one group makes the interval anti-conservative whenever that group is small relative to the dataset.

Conditioning on the group-assignment vector T, these ∣A∣|A| terms are i.i.d. and lie in [0,1][0, 1], which is what Hoeffding and empirical Bernstein require.

Parameters:

  • element (str) – base-variable token, e.g. "TP(A)".

  • Y (Array) – true labels.

  • predicted_Y (torch.Tensor) – predicted probability of label 1, for the whole dataset.

  • T (Array) – sensitive attribute column.

Returns:

1-D tensor of per-sample contributions, of length #{T == group}.

Return type:

torch.Tensor

fair_seldonian.constraints.inequalities.eval_estimate(element, Y, predicted_Y, T)[source]#

Point estimate of the base variable element.

This is the mean of contributions(). See that function for what the quantity means and why the two are tied together.

Parameters:

  • element (str) – base-variable token, e.g. "TP(A)".

  • Y (Array) – true labels.

  • predicted_Y (torch.Tensor) – predicted probability of label 1.

  • T (Array) – sensitive attribute column.

Returns:

scalar tensor.

Return type:

torch.Tensor

fair_seldonian.constraints.inequalities.eval_func_bound(
element,
Y,
predicted_Y,
T,
delta,
inequality,
candidate_safety_ratio,
predict_bound,
modified_h,
two_sided=True,
)[source]#

Confidence interval for a single base variable.

Parameters:

  • delta (float) – failure probability budget allocated to this node.

  • two_sided (bool) – whether the caller consumes both endpoints. A symmetric interval that must cover on both sides needs ln(2/delta); one that is only ever read from above needs ln(1/delta). Passing False when the lower endpoint is in fact used would silently double the true failure probability, so the sidedness is derived structurally from the constraint tree rather than guessed - see fair_seldonian.constraints.expression_tree.child_sides().

  • element (str)

  • Y (Array)

  • predicted_Y (torch.Tensor)

  • T (Array)

  • inequality (Inequality)

  • candidate_safety_ratio (float | None)

  • predict_bound (bool)

  • modified_h (bool)

Return type:

tuple[Bound, Bound]

fair_seldonian.constraints.inequalities.predict_hoeffding(estimate, safety_size, delta, two_sided=True)[source]#

Candidate-selection prediction of the safety-test interval.

This is a heuristic, not a bound: it guesses whether the safety test will pass so that candidate selection can avoid proposing solutions that would be rejected. It carries no guarantee and needs none - the safety test supplies the guarantee on its own. (It could not be a bound in any case: theta_c is chosen by optimizing on the candidate data, so the candidate-set estimate is not unbiased and pointwise concentration does not apply.)

Following Thomas et al. (2019), the safety-set interval is inflated by 2 to stay conservative.

Parameters:
Return type:

tuple[Bound, Bound]

Module contents#