releso.agent.DDPGAgent

class releso.agent.DDPGAgent(*, save_location: Path, logger_name: str | None = None, tensorboard_log: str | None = None, policy: Literal['MlpPolicy', 'CnnPolicy', 'MultiInputPolicy'], use_custom_feature_extractor: Literal['resnet18', 'mobilenet_v2'] | None = None, cfe_without_linear: bool = False, policy_kwargs: Dict[str, Any] | None = None, type: Literal['DDPG'], learning_rate: float = 0.001, buffer_size: int = 100000, learning_starts: int = 100, batch_size: int | None = 64, tau: float = 0.005, gamma: float = 0.99, optimize_memory_usage: bool = False, seed: int | None = None, device: str = 'auto')

Bases: BaseTrainingAgent

DDPG agent definition.

DDPG definition for the stable_baselines3 implementation for this algorithm. Variable comments are taken from the stable_baselines3 documentation.

__init__(**data: Any) → None: Constructor for the ReLeSO basemodel object.

Methods

`convert_to_pathlib_add_datetime`(v)	Add timestamp to save_location, of applicable.
`get_additional_kwargs`(**kwargs)	Add additional keyword arguments for agent instantiation.
`get_agent`(environment[, normalizer_divisor])	Creates the stable_baselines version of the wanted Agent.
`get_logger`()	Gets the currently defined environment logger.
`get_next_tensorboard_experiment_name`()	Return tensorboard experiment name.
`set_logger_name_recursively`(logger_name)	Set the logger_name variable for all child elements.

Attributes

`agent_type`	What RL algorithm was used to train the agent.
`learning_rate`	The learning rate, it can be a function of the current progress remaining (from 1 to 0)
`buffer_size`	size of the replay buffer
`learning_starts`	how many steps of the model to collect transitions for before learning starts
`batch_size`	Minibatch size
`tau`	the soft update coefficient ("Polyak update", between 0 and 1)
`gamma`	Discount factor
`optimize_memory_usage`
`seed`	Seed for the pseudo random generators
`device`	Device (cpu, cuda, …) on which the code should be run.
`policy`	policy defines the network structure which the agent uses
`use_custom_feature_extractor`	If given the str identifies the Custom Feature Extractor to be added.
`cfe_without_linear`	use the custom feature extractor with out a final linear layer
`policy_kwargs`	additional arguments to be passed to the policy on creation
`tensorboard_log`	base directory of the tensorboard logs if given an experiment name with a current timestamp is also added.
`save_location`	Definition of the save location of the logs and validation results.
`logger_name`	name of the logger.

agent_type: Literal['DDPG']: What RL algorithm was used to train the agent. Needs to be know to correctly load the agent.

batch_size: int | None: Minibatch size

buffer_size: int: size of the replay buffer

device: str: Device (cpu, cuda, …) on which the code should be run. Setting it to auto, the code will be run on the GPU if possible.

gamma: float: Discount factor

get_agent(environment: GymEnvironment, normalizer_divisor: int = 1) → DDPG

Creates the stable_baselines version of the wanted Agent.

Uses all Variables given in the object (except agent_type) as the input parameters of the agent object creation.

Parameters:

environment (GymEnvironment) – The environment the agent uses to
train.
normalizer_divisor (int) – Currently not used in this function.

Returns:

Initialized DDPG agent.

Return type:

DDPG

learning_rate: float: The learning rate, it can be a function of the current progress remaining (from 1 to 0)

learning_starts: int: how many steps of the model to collect transitions for before learning starts

seed: int | None: Seed for the pseudo random generators

tau: float: the soft update coefficient (“Polyak update”, between 0 and 1)