seldonian.utils.hyperparam_utils

Functions

bootstrap_sample_dataset(dataset, ...)

Bootstrap sample a dataset of size n_bootstrap_samples from the data points

clopper_pearson_bound(hyperparam_spec, ...)

Computes a 1-alpha clopper pearson bound on the probability of passing.

create_shuffled_dataset(dataset)

Create new dataset containing the same data as the given original dataset,

rebuild_parse_trees(spec, hyper_name, hyper_val)

Build new parse trees from existing spec, injecting new hyperparameter value.

set_spec_with_hyperparam_setting(spec, ...)

Update spec according to hyperparam_setting.

ttest_bound(hyperparam_spec, ...[, delta])

Compute ttest bound on the probability of passing using the bootstrap data across