fair_seldonian.experiments package

Submodules

fair_seldonian.experiments.plots module

fair_seldonian.experiments.plots.load_and_plot_results(file_name, ylabel, output_file, is_y_axis_prob, legend_loc)[source]

Plot results from CSV files and store the final graph.

Parameters:

  • file_name (str) – The csv file path from where the data is imported

  • ylabel (str) – The lable on the Y-axis of the graph

  • output_file (str) – The path where the graph image must be stored

  • is_y_axis_prob (bool) – Bool of whether the Y-axis is probability value or not

  • legend_loc (str) – The location of the legend

Return type:

None

fair_seldonian.experiments.results module

fair_seldonian.experiments.results.get_existing_experiment_numbers(bin_path='exp/lag_exp/bin/')[source]
Parameters:

bin_path (str)

Return type:

list[int]

fair_seldonian.experiments.results.gen_filename(n, bin_path='exp/lag_exp/bin/')[source]
Parameters:
Return type:

str

fair_seldonian.experiments.results.add_more_results(new_file_id, ms, seldonian_solutions_found, seldonian_fs, seldonian_failures_g1, seldonian_upper_bound, LS_solutions_found, LS_fs, LS_failures_g1, LS_upper_bound, bin_path='exp/lag_exp/bin/')[source]
Parameters:
Return type:

list[ndarray]

fair_seldonian.experiments.results.stderror(v)[source]
Parameters:

v (ndarray)

Return type:

float

fair_seldonian.experiments.results.save_to_csv(ms, results_qsa, results_ls, filename)[source]
Parameters:
Return type:

None

fair_seldonian.experiments.results.gather_results(bin_path='exp/lag_exp/bin/', csv_path='exp/lag_exp/csv/')[source]
Parameters:

  • bin_path (str)

  • csv_path (str)

Return type:

None

fair_seldonian.experiments.runner module

fair_seldonian.experiments.runner.store_result(theta, theta1, test_x, test_y, test_t, passed_safety_test, worker_id, n_workers, m, trial, num_trials, seldonian_type, is_baseline, config=SeldonianConfig(delta=0.05, inequality=<Inequality.HOEFFDING_INEQUALITY: 2>, constraint='TP(1) TP(0) - abs 0.25 TP(1) * -', candidate_ratio=0.4))[source]

Print and store the resultant information in a file.

Parameters:

  • theta (torch.Tensor) – The parameters of the model

  • theta1 (torch.Tensor) – The additional parameter of the model, often the last parameter

  • test_x (np.ndarray) – The features of the test dataset

  • test_y (np.ndarray) – The labels of the test dataset

  • test_t (np.ndarray) – The sensitive attribute column of the test dataset

  • passed_safety_test (bool) – Whether the safety test was passed

  • worker_id (int) – Id of the worker thread

  • n_workers (int) – Total number of worker threads

  • trial (int) – Trial number of the experiment on the worker thread

  • num_trials (int) – Total number of trials

  • seldonian_type (str) – Mode used in the experiment

  • is_baseline (bool) – Whether this is the unconstrained logistic-regression baseline

  • m (float)

  • config (SeldonianConfig)

Returns:

(solution_found, failure_g, upper_bound, fhat) tuple values

Return type:

tuple[int, int, float, float | None]

fair_seldonian.experiments.runner.run_experiments(worker_id, n_workers, ms, num_trials, m_test, N, seldonian_type, config=SeldonianConfig(delta=0.05, inequality=<Inequality.HOEFFDING_INEQUALITY: 2>, constraint='TP(1) TP(0) - abs 0.25 TP(1) * -', candidate_ratio=0.4), output_dir='exp/exp_{}/bin/')[source]

Main function that runs the experiment.

Parameters:

  • worker_id (int) – Id of the worker thread

  • n_workers (int) – Total number of worker threads

  • ms (ndarray) – Array containing the fraction values of the amount of data to be used

  • num_trials (int) – Total number of trials

  • m_test (float) – The fraction of test samples to be used from the complete dataset

  • N (int) – Number of data samples of the synthetic dataset

  • seldonian_type (str) – Mode used in the experiment

  • config (SeldonianConfig)

  • output_dir (str)

Returns:

None

Return type:

None

Module contents