experiments.baselines.decision_tree_leaf_tuning.DecisionTreeClassifierLeafTuningBaseline

class DecisionTreeClassifierLeafTuningBaseline(primary_objective_fn, sub_regime, adam_kwargs, dt_kwargs={})

Bases: SeldonianDecisionTree, SupervisedExperimentBaseline

__init__(primary_objective_fn, sub_regime, adam_kwargs, dt_kwargs={})

Implements a decision classifier with leaf node tuning as a baseline for binary classification tasks

Parameters:

  • primary_objective – The primary objective function to be minimized during the leaf tuning process.

  • sub_regime – Sub category of ML problem, e.g., “classification”

  • adam_kwargs – Keyword arguments to pass to the adam optimizer

  • dt_kwargs – Any keyword arguments that scikit-learn’s DecisionTreeClassifier takes.

__repr__()

Return repr(self).

Methods

adam(theta_init, **kwargs)

Perform gradient descent with adam optimizer. Hardcoded some hyperparameters just for this example baseline.

Parameters:

theta_init – The initial weights to begin gradient descent

Returns:

best_theta, the optimal weights that minimize the primary objective function

fit(features, labels, **kwargs)

A wrapper around SKLearn’s fit() method. Returns the leaf node probabilities of SKLearn’s built tree.

Parameters:

  • features (numpy ndarray) – Features

  • labels (1D numpy array) – Labels

Returns:

Leaf node probabilities (of predicting the positive class only), ordered from left to right

forward_pass(X)

Do a forward pass through the sklearn model.

Parameters:

X (numpy ndarray) – model features

Returns:

probs_pos_class: the vector of probabilities, leaf_nodes_hit: the ids of the leaf nodes that were

hit by each sample. These are needed for computing the Jacobian

get_jacobian(ans, theta, X)

Return the Jacobian d(forward_pass)_i/dtheta_j, where i run over datapoints and j run over model parameters.

Parameters:

  • ans – The result of the forward pass function evaluated on theta and X

  • theta – The weight vector, which isn’t used in this method

  • X – The features

get_leaf_node_probs()

Retrieve the leaf node probabilities from the current tree from left to right

predict(theta, X, **kwargs)

Call the autograd primitive (a workaround since our forward pass involves an external library)

Parameters:

  • theta (numpy ndarray) – model weights (not probabilities)

  • X (numpy ndarray) – model features

Return pred:

model predictions

Rtype pred:

numpy ndarray same shape as labels

set_leaf_node_values(probs)
Update the leaf node probabilities

(actually the numbers in each label)

Parameters:

probs – The vector of probabilities to set on the leaf nodes from left to right

train(X, Y)

Instantiate a new model instance and train (fit) it to the training data, X,Y. Then run Adam gradient descent on the leaf node probabilities.

Parameters:

  • X (2D np.ndarray) – features

  • Y (1D np.ndarray) – labels

Returns:

theta, the fitted model parameters

wrapped_primary_objective(theta)

The first argument of primary_objective_fn is model, which is used to get the predictions via .predict(). We want to use the current model’s predict() method (inherited) for this, so we use self for first arg.

Parameters:

theta – Model weights

Returns:

The primary objective function evaluated with these theta weights