seldonian.utils.hyperparam_utils.ttest_bound

ttest_bound(hyperparam_spec, bootstrap_trial_data, delta=0.1)

Compute ttest bound on the probability of passing using the bootstrap data across

bootstrap trials.

Parameters:

• bootstrap_trial_data (np.array) – Array of size n_bootstrap_samples, containing the result of each bootstrap trial.

• delta (float) – confidence level, i.e. 0.05