seldonian.candidate_selection.candidate_selection.CandidateSelection

class CandidateSelection(model, candidate_dataset, n_safety, parse_trees, primary_objective, optimization_technique='barrier_function', optimizer='Powell', initial_solution=None, regime='supervised_learning', write_logfile=False, additional_datasets={}, **kwargs)

Bases: object

__init__(model, candidate_dataset, n_safety, parse_trees, primary_objective, optimization_technique='barrier_function', optimizer='Powell', initial_solution=None, regime='supervised_learning', write_logfile=False, additional_datasets={}, **kwargs)

Object for running candidate selection

Parameters:

  • model (models.model.SeldonianModel object) – The Seldonian model object

  • candidate_dataset (dataset.Dataset object) – The dataset object containing candidate data

  • n_safety (int) – The length of the safety dataset, used when predicting confidence bounds during candidate selection

  • parse_trees (List(parse_tree.ParseTree objects)) – List of parse tree objects containing the behavioral constraints

  • primary_objective (function or class method) – The objective function that would be solely optimized in the absence of behavioral constraints, i.e. the loss function

  • optimization_technique (str) – The method for optimization during candidate selection. E.g. ‘gradient_descent’, ‘barrier_function’

  • optimizer (str) – The string name of the optimizer used during candidate selection

  • initial_solution (array) – The model weights used to initialize the optimizer

  • regime (str) – The category of the machine learning algorithm, e.g., supervised_learning or reinforcement_learning

  • write_logfile (bool) – Whether to write outputs of candidate selection to disk

  • additional_datasets (dict, defaults to {}) – Specifies optional additional datasets to use for bounding the base nodes of the parse trees.

__repr__()

Return repr(self).

Methods

calculate_batches(batch_index, batch_size, epoch, n_batches)

Create a batch dataset (for the primary dataset) to be used in gradient descent. Sets self.batch_dataset. See return logic.

Parameters:

  • batch_index (int) – The batch number (0-indexed)

  • batch_size (int) – The size of the batches

  • epoch (int) – The 0-indexed epoch (needed for additional datasets, if provided)

  • n_batches (int) – The number of batches per epoch (needed for additional datasets, if provided)

Returns:

True if candidate solutions calculated using this batch are viable, False if not.

calculate_batches_addl_datasets(primary_epoch_index, primary_batch_index, n_batches)

For each additional dataset, create a batch dataset using the current batch. Uses the batch_indices list stored in the additional datasets dictionary that was precalculated to make this easy. Does not return anything.

Parameters:

  • primary_epoch_index (int) – The epoch number (0-indexed) of the primary dataset

  • primary_batch_index (int) – The batch number (0-indexed) of the primary dataset

  • n_batches (int) – The number of primary dataset batches per epoch

Returns:

None

evaluate_primary_objective(theta)

The primary objective function used for KKT/gradient descent. This is just a wrapper for self.primary_objective where the (batched) data and model are fixed, such that the only parameter is theta.

Parameters:

theta (numpy.ndarray) – model weights

Returns:

The value of the primary objective function evaluated at theta.

get_constraint_upper_bounds(theta)

The constraint functions used for KKT/gradient descent. Obtains the upper bounds of the parse trees given model weights, theta.

Parameters:

theta (numpy.ndarray) – model weights

Returns:

Array of upper bounds on the parse trees.

Return type:

array

get_importance_weights(theta)

Get an array of importance weights evaluated on the candidate dataset given model weights, theta. Only applicable for RL.

Parameters:

theta (numpy.ndarray) – model weights

Returns:

Array of upper bounds on the constraint

Return type:

array

objective_with_barrier(theta)

The objective function to be optimized if optimization_technique == ‘barrier’. Adds in a large penalty when any of the constraints are violated.

Parameters:

theta (numpy.ndarray) – model weights

Returns:

the value of the objective function evaluated at theta

precalculate_addl_dataset_batch_indices(n_epochs, n_batches, primary_batch_size)

For each additional dataset, create a list of indices corresponding to the start and end of each batch. During each iteration of gradient descent, we can look up these indices and then construct the dataset from them. Updates self.additional_datasets in place, so doesn’t return anything.

The rules here are: i) A custom batch size can optionally be used for each addl dataset, but

if it is missing, the primary batch batch is used.

  1. If we get to the end of a dataset, we wrap around back to the start of the dataseet. For this reason, when specifying the batch indices for a given batch, we can either have a list of length 2 (start,end) with no wraparound, or a list of length 4 (start1,end1,start2,end2) with wraparound.

  2. If n_batches == 1 then just use the entire additional dataset,

but don’t wrap as that would reuse datapoints in a single batch

iv) There is no concept of epoch for the addl datasets. We are simply wrapping back to the beginning when needed.

Parameters:

  • n_epochs (int) – The total number of epochs to run gradient descent

  • n_batches (int) – The number of primary dataset batches per peoch

  • primary_batch_size (int) – The batch size for the primary dataset.

Returns:

None

run(**kwargs)

Run candidate selection, either with gradient descent or barrier function techniques. Barrier function can use any black box optimizer to generate candidate solutions.

Returns:

candidate_solution, which is either an array of optimized model weights or ‘NSF’ if there was no viable candidate solution found, usually because of an error during candidate selection.

Return type:

array or str