seldonian.RL.Agents.Bases.Fourier.Fourier

class Fourier(hyperparam_and_setting_dict, env_desc)

Bases: object

__init__(hyperparam_and_setting_dict, env_desc)

Fourier basis used for linear value function approximation. See http://irl.cs.brown.edu/fb.php for a reference. Each feature is a sine or cosine function of a certain order.

Parameters:

  • hyperparameter_and_setting_dict – Specifies the environment, agent, number of episodes per trial, and number of trials

  • env_desc (Env_Description) – an object for accessing attributes of the environment

__repr__()

Return repr(self).

Methods

calculate_num_features(order, max_coupled_vars, num_obs_dims)

Calculate the number of features in a Reinforcement Learning (RL) Fourier basis.

This function computes the total number of Fourier basis features for a given order, considering the maximum number of coupled variables and the number of observation dimensions in the RL environment.

Parameters:

  • order (int) – The maximum order of the Fourier basis. This defines the highest frequency of sine and cosine functions in the basis.

  • max_coupled_vars (int) – The maximum number of variables that can be coupled in a single feature. This parameter controls the complexity of interaction terms in the Fourier basis.

  • num_obs_dims (int) – The number of dimensions in the observation space of the RL environment. Each dimension can be considered as a variable in the Fourier basis.

Return num_features:

The total number of features in the Fourier basis, which includes all possible combinations of sine and cosine functions up to the specified order, considering the limit on coupled variables and the number of observation dimensions.

construct_basis_matrix()

Construct and return the basis matrix for a Reinforcement Learning (RL) Fourier basis.

This function builds a matrix where each row represents a feature in the Fourier basis.

Return basis_matrix:

A 2D numpy array of shape (num_features, num_observation_dims), where

num_features is the total number of valid Fourier basis features and num_observation_dims is the number of dimensions in the observation space. Each entry in the matrix is an integer representing the frequency for that dimension in the Fourier basis.

get_features(obs)

Get the basis feature given an observation

Parameters:

obs – unnormalized observation

Return ret_matrix:

A matrix of features for the given observation

get_normalized_observation(obs)

Get the normalized observation given an observation

Parameters:

obs – unnormalized observation

Return norm_obs:

normalized observation